AgitatedDove14 yeah I see this now; this was an issue because I later had to "disconnect" the remote task, so it can, itself, create new tasks (using
clearml.config.remote.override_current_task_id(None) ). I guess you might remember that discussion? 😁
EDIT: It's the discussion we had here, for reference. https://clearml.slack.com/archives/CTK20V944/p1640955599257500?thread_ts=1640867211.238900&cid=CTK20V944
So probably not needed in JitteryCoyote63 's case, we still have some use case for it.
AgitatedDove14 So I copied pasted locally the https://github.com/pytorch-ignite/examples/blob/main/tutorials/intermediate/cifar10-distributed.py from the examples of pytorch-ignite. Then I added a requirements.txt and called
clearml-task to run it on one of my agents. I adapted a bit the script (removed python-fire since it’s not yet supported by clearml).
pip install git+ https://github.com/allegroai/clearml.git
BTW: UnevenDolphin73 you should never actually do "task = clearml.Task.get_task(clearml.config.get_remote_task_id())"
You should just do "
Task.init() " it will automatically take the "get_remote_task_id" and do all sorts of internal setups, you will end up with the same object but in an ordered fashion
Yes even without any arguments give to
Task.init() , it has everything from the server
Should be added before the
if name == "main":
Yes, it should.
From you code I understand it is not ?
clearml version you are using ?
AgitatedDove14 I will try! I remember there were some issues with it, where I had to resort to this method first, but maybe things have changed since :)
JitteryCoyote63 I think I found the bug in
clearml-task it adds it at the end instead of before everything else
JitteryCoyote63 fix should be pushed later today 🙂
Meanwhile you can manually add the Task.init() call to the original script at the top, it is basically the same 🙂
So I guess the problem is that the following snippet:
from clearml import Task Task.init()Should be added before the
if __name__ == "__main__": ?
JitteryCoyote63 yes exactly, sorry, I forgot to add the
Task.get_task in my response. That's exactly what we do 😅
JitteryCoyote63 Are you calling
and this works. However, without the trick from UnevenDolphin73 , the following won’t work (return None):
if __name__ == "__main__": task = Task.current_task() task.connect(config) run() from clearml import Task Task.init()
JitteryCoyote63 I use
AgitatedDove14 So I’ll just replace
task = clearml.Task.get_task(clearml.config.get_remote_task_id()) with
Task.init() and wait for your fix 🙂
Task.init is called at a later stage of the process, so I think this relates again to the whole setup process we've been discussing both here and in #340... I promise to try ;)
AgitatedDove14 No, should I?
Do you want the task object or the task id?
The task object
AgitatedDove14 , my “uncommitted changes” ends with
... if __name__ == "__main__": task = clearml.Task.get_task(clearml.config.get_remote_task_id()) task.connect(config) run() from clearml import Task Task.init()
task = clearml.Task.get_task(clearml.config.get_remote_task_id()) worked, thanks
I remember there were some issues with it ...
I hope not 😞 Anyhow the only thing that does matter is the auto_connect arguments (meaning if you want to disable some, you should pass them when calling Task.init)
JitteryCoyote63 no you should not (unless you already have the Task.init call in your code)
clearml-data add the Task.init call at the beginning of the code in the entry point.
This means you should be able to get
Task.current_task() and get back the object.
What do you have under the "uncommitted changes" on the Task that was created?
clearml.config.get_remote_task_id() will return the Task ID not the Task object. in order to get automagic to work, one has to call
Task.init() . When executed by the agent, Task.init() will take the arguments from the Task definition on the server (it will ignore task_name, project_name etc arguments passed to the task.init call) , and apply all the patching to the different frameworks. Make sense?