the train_loss is on the second from left column (the far left is epoch num 30-36)

yes, that solved the errors, however the two lines "could not detect iteration reporting" and "reporting detected" a few moments later, still show up

in the meantime, I got this error message, this time regarding Trains:

it isn't an error, but just an observation

Understood. If there is something I can tweak in the reporting, I couldn't find where I tweak it since it is supposed to be related to the one line of activation of the reporting learn.callback_fns.append(partial(LearnerTensorboardWriter, base_dir=tboard_path, name=taskName)) do you have any ideas what are the options I can do to change the report of the train_loss?

Good morning Alon, since you helped me so much getting tensorboard to show results yesterday, I'm hoping you can help me understand why some results I'm getting are strange:

it is on the scale of 15K...

let me see

` Traceback (most recent call last):
File "/home/ubuntu/MultiClassLabeling/myenv/lib/python3.6/site-packages/torch/utils/tensorboard/init.py", line 2, in <module>
from tensorboard.summary.writer.record_writer import RecordWriter # noqa F401
File "/home/ubuntu/MultiClassLabeling/myenv/lib/python3.6/site-packages/trains/binding/import_bind.py", line 59, in __patched_import3
level=level)
ModuleNotFoundError: No module named 'tensorboard'

During handling of the above exception, ...

here is the result on the Tboard in Trains:

Thanks for letting me know, I'd be very happy to update.

I will try and let you know in the next experiment

This is the whole Tboard

TRAINS Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start TRAINS Monitor: Reporting detected, reverting back to iteration based reporting

Here is an example of the results from terminal:

haven't tried yet

then this:

no, I meant to change the way it is reported. I'm still interested in the train_loss graph, naturally 🙂 but obviously it is reporting something that is the inverse of the train_loss, since in the graph it is exploding, and in reality (as reported in the terminal) it is decaying to 9e-2

i did not install tensorboard 😞

This is the valin_loss, which is correct:

tried both with Firefox and Chrome, results are similar also between computers and OS (ubuntu and Windows)

this is an error during training that points out to ElasticSearch error. This might be also the cause of the delete error, what do you think SuccessfulKoala55 ?

this is what I got:
{"meta":{"id":"7cd78b67e5384e739b9aec6cdc030e6d","trx":"7cd78b67e5384e739b9aec6cdc030e6d","endpoint":{"name":"projects.delete","requested_version":"2.20","actual_version":"1.0"},"result_code":400,"result_subcode":12,"result_msg":"Validation error (error for field 'project'. field is required!)","error_stack":null,"error_data":{}},"data":{}}

Thanks Jake for your help, it's highly appreciated. This is an AWS EC2 running the clearml-server AMI (region of EC2 is us-east-1)

How do I check that? I did nothing in the settings other than the recommended defaults in the docs

the "Payload" tab contains the project id info, so it shouldn't be the cause for the delete call fail