Hi VirtuousFish83 ,

Do you have this package in the INSTALLED PACKAGES section of your task (under EXECUTION tab)?

no

You have all the other packages in this section except pyfunctional ?

I think so

Is there a way to check how clearml gets the installed packages of the current env ?

Can you try running your task with detect_with_pip_freeze option?

In your ~/clearml.conf file, change sdk.development.detect_with_pip_freeze to true

Hmm it's both better and worse, it does detect pyfunctional now (in INSTALLED PACKAGES and I can see it installed in the console logs) but it fails on
import torch ModuleNotFoundError: No module named 'torch'In the logs:
Found PyTorch version torch==1.7.1 matching CUDA version 110 2021-04-21 15:15:11 Found PyTorch version torchvision==0.8.2 matching CUDA version 110 Collecting torch==1.7.1+cu110 File was already downloaded /home/ubuntu/.clearml/pip-download-cache/cu110/torch-1.7.1+cu110-cp38-cp38-linux_x86_64.whl Successfully downloaded torch Collecting torchvision==0.8.2+cu110 File was already downloaded /home/ubuntu/.clearml/pip-download-cache/cu110/torchvision-0.8.2+cu110-cp38-cp38-linux_x86_64.whl Successfully downloaded torchvision Processing /home/ubuntu/.clearml/pip-download-cache/cu110/torch-1.7.1+cu110-cp38-cp38-linux_x86_64.whl Processing /home/ubuntu/.clearml/pip-download-cache/cu110/torchvision-0.8.2+cu110-cp38-cp38-linux_x86_64.whl

according to the logs, the issue is when installing inplace-abn packages. let me check the error

Yes I think it needs pytorch, but pytorch failed to install previously ?

VirtuousFish83
could that be that "inplace-abn" while installing the package needs torch ?

Collecting inplace-abn==1.0.12 Downloading inplace-abn-1.0.12.tar.gz (137 kB) ERROR: Command errored out with exit status 1: command: /home/ubuntu/.clearml/venvs-builds/3.8/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/setup.py'"'"'; __file__='"'"'/tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-bxycliog cwd: /tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/ Complete output (5 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/setup.py", line 4, in <module> import torch ModuleNotFoundError: No module named 'torch' ----------------------------------------

Yes the setup.py imports torch unfortunately https://github.com/mapillary/inplace_abn/blob/master/setup.py

Try to add inplace-abn to the post_packages section https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf#L79 , and it will be installed after all the other packages

Ok, btw I used https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_agent_install_configure.html which was not updated so I didn't know there was a priority_packages and post_packages

It works with post_packages