I do not remember exactly all the steps we did, but with the help of clearml team we found solution by adding:Task.add_requirements((…)outside the pipeline code
hey Ofir
did you tried to put the repo in the decorator where you need the import ?
if you can send me some code to illustrate what you are doing, it could help me to reproduce the issue
no problem, you and your team are very responsive and help us a lot.
I try to install package from our github
great to hear that the issue is solved. btw sorry for the time it took me to come back to you
Ok. We'll try to reproduce this. If you can send a snippet/example that could help. Anyway we'll keep you updated
Thanks David,
Yes, I have the issue in pipelines and I use the decorators method for the pipelines.
The pipeline include scripts from several files.
If I put the import in the pipeline main file it works, but if the import is in another file (which is imported by the pipeline main file) it does not work.
The import is needed in one of the tasks (which is also on a different file) - I tried put ting the import on the top of the task file and also inside the task method but it didn’t work.
Also tried to add the package in the task’s decorator params and it didn’t help.
Hello Ofir,
in general matter, the agent parses the script and finds all the imports, through an intelligent analysis (it installs the ones you use/need).
It then build an env where it will install them and run (docker, venv/pip.etc).
You can also force a package/ package version
For the pipelines (and the different ways to implement them), it is a bit different
In order to answer you precisely, we would need to have a bit more detais about what you need to achieve :
Is it a pipeline that is being executed ? Is the issue with one of the steps of the pipeline? How is this step created ? (i.e. existing Task of function decorator)
Hey Ofir
We would need to know a bit more about the issue. Especially on what you try to import. Is it a file that is on your repo or does the issue occurs when you try to import a standard pypi package ?