When I launch tasks with a pipeline, they keep complaining about missing pip packages. I run it inside a docker container, and I'm sure these packages are present inside it (when I launch the container locally, run python3 and import them, it works like charm). Any ideas how to fix this?
This is odd, Can you send the full Task log? (remove any pass/user/repo that you think is sensitive)
I specifically set is as empty with
export_data['script']['requirements'] = {}
in order not to reduce overhead during launch. I have everything installed inside the container
Do you have everything inside the container Inside a venv ?
MelancholyElk85 , did you start the task using task.init() or did you start it through the pipeline?
MelancholyElk85 what do you have under "Installed Packages" for this specific Task ?
I specifically set is as empty with export_data['script']['requirements'] = {} in order not to reduce overhead during launch. I have everything installed inside the container
I understand but how do you launch the cleaml-agent itself:clearml-agent daemon --detached --queue default --docker
You're right, I re-launched the agents, now it pulls images
It doesn't install anything with pip during launch, I'm assuming it should take everything from the container itself (otherwise there would be a huge overhead). It simply fails trying to import things in the script
File "preprocess.py", line 4, in <module> from easydict import EasyDict as edict ModuleNotFoundError: No module named 'easydict'
MelancholyElk85 can you provide some logs on the pip install fails ?
MelancholyElk85 , please try the following:task_overrides={'script.branch': 'main', 'script.version_num': '', }
This doesn't seem to be running inside a container...
What's the clearml-agent launch command you are using ? (i.e. do you have --docker flag)
I have a base task for each pipeline step. When I initialize a pipeline, for each step I clone the corresponding task, modify it and add it as a step. Tasks are launched from a pipeline, not cli. I'm absolutely sure docker argument is not empty (I specify it with export_data['container']['image'] = ' http://registry.gitlab.com/cherrylabs/ml/clearml-demo:clearml ' , and it shows on Web UI)