So it is the automagic that is not working.
Can you print the following before calling Both Task.debug_simulate_remote_task
and Task.init
, Notice you have to call Task.initprint(os.environ)
I could have been more inventive as well 😄
the situation is:
I can report a scalar by calling clearml_task.logger.report_scalar
and it becomes immediately visible in the web ui
report_scalar works, report_text does not, this is very weird
maybe another warning here? https://clear.ml/docs/latest/docs/references/sdk/task/
but seriously, I am very thankful you were willing to spend so much time helping me, I am super impressed by your response time and helpfulness!
I can't make anything appear in the console part of the ui
report_text does not, this is very weird
Okay this seems to be the issue.
Just making sure the Task status is "running" and task.get_logger().report_text("something")
does not report a thing ?
Do you see it on your screen?
Can you test without the "Task.debug_simulate_remote_task / init" ?
but perhaps it is worth adding to the docs page a hint to avoid using the CLEARML_TASK_ID env variable, perhaps I am not the only one to ever try it
but perhaps it is worth adding to the docs page a hint to avoid using the CLEARML_TASK_ID env variable, perhaps I am not the only one to ever try it
Good idea, any thoughts on where ? I cannot find a trivial place to put these things
now it stopped working locally as well
At least this is consistent 🙂
How so ? Is the "main" Task still running ?
I guess we should have obfuscated the name better 😄
Just to let you know, it now works (obviously) in the k8s setting as well.
maybe not at the top but in the Task.init
description
no, I set the env variable CLEARML_TASK_ID myself
I don't know whether you have access to the backend,
Creepy , no I do not 🙂
I can't make anything appear in the console part of the ui
clearml_task.logger.report_text("some text")
should work
I can hardcode it into program if you want
Wait, how did you end up withclearml_task_id = os.environ['CLEARML_TASK_ID']
printing "01b77a220869442d80af42efce82c617" ?
This means you are running by an agent?!
no, I set the env variable CLEARML_TASK_ID myself
Do not, this is the issue 🙂
this is used internally and messing up the internal state, basically this is one of the signals for the SDK to know there is an agent taking care of things (for example logging the entire console output)
Use any other variable, for example MY_CLEARML_TASK_ID
console output:ClearML results page:
01b77a220869442d80af42efce82c617 some text 2022-03-21 22:47:16,660 - clearml.Task - INFO - Waiting to finish uploads 2022-03-21 22:47:28,217 - clearml.Task - INFO - Finished uploading
my code snippet
` from clearml import Task
import os
clearml_task_id = os.environ['CLEARML_TASK_ID']
Task.debug_simulate_remote_task(clearml_task_id)
clearml_task = Task.init(auto_connect_arg_parser=False, auto_resource_monitoring=False)
print(clearml_task.id)
clearml_task.logger.report_scalar(series='s', value='123', iteration=2, title='title')
clearml_task.logger.report_text("some text") `
yes, I don't know whether you have access to the backend, but just in case my experiment is this one:
ClearML results page: https://app.clear.ml/projects/45557a1ee1464631a9a18b0dcda4f682/experiments/01b77a220869442d80af42efce82c617/output/log