I managed to get to the stage where 'Environment setup completed successfully' for the enqueued task!
However, now I encountered another issue - an error that did not occur on the manual run.
Related to
task.connect(config.model,name='model params')
I wonder if this is related to environment
Hi, Thanks! I thought it would be good to use conda since I used it on my local machine , but I understand now pip can manage everything.
Hi @<1523702969063706624:profile|PoisedShark13>
However, INSTALLED PACKAGES of my task is misses many of installed packages (any idea why?)
It automatically detects the directly imported packages, literally analyzing your code base and looking for imports
The derivative packages (i.e. the one that any of the "main" packages need, will be listed after the first time the agent installs everything)
If something specific is missing, you can manually add it with:
Task.add_requirements("package", "version")
task = Task.init(...)
How do I configure clearml-agent to use conda for virtual environment?
Is this instead of pip? if so why? specifically: None
How do I configure clearml-agent to build the virtual environment based on the desired ‘environment.yaml’ file instead of INSTALLED PACKAGES?
Actually this is not really supported, it will always revert to "installed packages" section.
The reason is that the environment.yaml ius rarely updated with the entire package list, and one of the main goals of the agent is that once it provisioned the Task, it should be able to replicate the exact setup again.
What exactly do you have in the yaml that would be challenging ?
If this is regrading cuda and pytorch, the agent automatically does the resolving and install the correct pytorch wheel based on the installed cuda drivers (both in pip and in conda)