Can you check it with the latest? 1.0.2
?
SuccessfulKoala55 I can try to make one, let’s see 🙂
TimelyPenguin76 clearml 0.17.5 and clearml-agent 0.17.2
JitteryCoyote63 which version are you using?
I'm not sure... Do you have a sample code to reproduce?
SuccessfulKoala55 Am I doing/saying something wrong regarding the problem of flushing every 5 secs (See my previous message)
Ok. I just wanted to make sure I have configured my agent properly. Just want to make sure I have to set it on all agents.
Well, it's probably possible to add a hack in your code that always sets this value in the internal logger class, but how many agents are you running? If it's very important, we guess we can add a Logger interface that allows setting this properly programmatically.
So I have to specify it on every clearml-agent in the respective clearml.conf?
ReassuredTiger98 no, this is an SDK configuration parameter which is read from your clearml.conf
file
When you say it is an SDK parameter this means that I only have to specify it on the computer where I start the task from, right? So an clearml-agent would read this parameter from the task itself.
It roughly adds a line every 5 secs and do not overwrite last line
Ok, in that case it probably doesn’t work, because if the default value is 10 secs, it doesn’t match what I get in the logs of the experiment: every second the tqdm adds a new line
so the task they execute must have clearml installed?
It's not the agents, but the SDK that runs as part of the task they execute
Does that mean that agents do not read this parameter?
BTW - it's an SDK parameter, not an agent parameter
You're right, its a relatively internal parameter, and is currently not documented in the default configuration file. Perhaps it's a good idea to add it to the formal documentation... default value is 10
:
` # Carriage return (\r) support. If zero (0) \r treated as \n and flushed to backend
Carriage return flush support in seconds, flush consecutive line feeds (\r) every X (default: 10) seconds
console_cr_flush_period: 10 `
btw SuccessfulKoala55 the parameter is not documented in https://allegro.ai/clearml/docs/docs/references/clearml_ref.html#sdk-development-worker
For me this does not work (at least with nested tqdm bars, did not try single ones yet).
allows you to control the rate at which the carriage return is flushed
Add carriage return flush support using the sdk.development.worker.console_cr_flush_period configuration setting (GitHub trains Issue 181)
Can you use sdk.development.worker.console_cr_flush_period
?