Reputation
Badges 1
25 × Eureka!Well, I would like "quickly" to compare durations of several experiments ...
As they didn't all started at the same time, I have to do math to subtract the duration 🙂
It's code is like that
` (env-name) /code# cat /opt/conda/envs/env-name/bin/doit `` #!/opt/conda/envs/env-name/bin/python
-- coding: utf-8 --
import re
import sys
from doit.main import main
if name == 'main':
sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
sys.exit(main()) `
If using the launcher as is -> Can not detect the repository:launcher args
but if execute the launcher as a module -> repository is detectedpython -m launcher args
It would be nice to avoid writing "python -m" everytime...
Patching sys.argv[0] = __file__
where __file__
is the file where Task.init
happens helps with git repo discovery.
From our side everything work ok i would say if we resume from an epoch : https://demoapp.demo.clear.ml/projects/5678e22221984581b089fd110c8db1ea/compare-experiments;ids=da10aacccc97459ea13df27b9cd44561,8276746945f6424781309513cee21cf8/scalars/graph?scalars=graph
Thanks UptightCoyote42 !
Hi AgitatedDove14 thanks for the answer.
I'm using python doit: https://pydoit.org/ to schedule multiple steps. Basically, it does subprocess.Popen for each step. In each step (=subprocess) I get the task using Task.current_task()
.t0: doit -> parent proc creates new Task t1: |_ child proc: current_task() and do smth t2: |_ child proc: current_task() and do smth
My question is what happens if I launch in parallel multiple doit commands that create new Tasks. I would l...
Could you please explain in 2-3 words how current task fetches the good Task ?
Thanks! Yes, this could be great !
And if you could also update the docs with all env vars possible to set up it would awesome!
Yes, that make sense !
Could you please open a GitHub issue, so we remember to update the feature ?
sure !
=>Task.init(project_name='main_project/sub_project', task_name='test')
https://clear.ml/docs/latest/docs/fundamentals/task#task-creation
according to https://allegro.ai/clearml/docs/docs/concepts_fundamentals/concepts_fundamentals_storage.html?highlight=output_uri#configuring-clearml-for-artifact-storage there is no way to do that ?
I think it is OK for me, thanks !
AgitatedDove14 I tried and it works OK withTask.current_task()
and overall Task.init
.
Sorry for confusion.
Let me check that, thanks !
I was also started to think about closing the task and call init. Let me detail again the use-case if it was unclear:parent process |-- Task.init(...) |-- step 1 sub-process | |-- Task.init(...) | |-- task.connect |-- step 2 sub-process | |-- Task.init(...) | |-- task.connect |- end
actually, i'm using subprocess.
Popen
and step X is another python script file executed from a single main parent process.
Hi AgitatedDove14
if there is no clearml.conf on the system, then the script with Task.init("a", "b")
will create project "a" and task "b" on demoapp server.
User can also do the following:Task.set_credentials(...) Task.init("a", "b")
and now it will go to provided api_host etc, still without clearml.conf file.
Now, I would like to raise an issue if api_host is the default one (the one when without clearml.conf file) and thus tracking is going to public demoapp server.
But actua...
Updated column also can contain some more recent values and not the time when experiment ended/failed etc ...