Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Sorry I Have Again Another Problem, Does Clearml Have Its Own Package Resolution System And Doesn'T Use Pip ? I Use A Lib Named Pyfunctional (

Sorry I have again another problem, does clearml have its own package resolution system and doesn't use pip ? I use a lib named pyfunctional ( https://pyfunctional.pedro.ai/ , installed with pip install pyfunctional ) but that is imported with import functional (note the different name) and that is the only one that the clearml-agent "forgets" to install (but it's both in original env, and in my repo's setup.py)

  
  
Posted 3 years ago
Votes Newest

Answers 15


Is there a way to check how clearml gets the installed packages of the current env ?

  
  
Posted 3 years ago

no

  
  
Posted 3 years ago

according to the logs, the issue is when installing inplace-abn packages. let me check the error

  
  
Posted 3 years ago

Collecting inplace-abn==1.0.12 Downloading inplace-abn-1.0.12.tar.gz (137 kB) ERROR: Command errored out with exit status 1: command: /home/ubuntu/.clearml/venvs-builds/3.8/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/setup.py'"'"'; __file__='"'"'/tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-bxycliog cwd: /tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/ Complete output (5 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-xf3qf6et/inplace-abn_15b6998cb4af4199a7692be5d3a3538f/setup.py", line 4, in <module> import torch ModuleNotFoundError: No module named 'torch' ----------------------------------------

  
  
Posted 3 years ago

It works with post_packages

  
  
Posted 3 years ago

You have all the other packages in this section except pyfunctional ?

  
  
Posted 3 years ago

Can you try running your task with detect_with_pip_freeze option?

In your ~/clearml.conf file, change sdk.development.detect_with_pip_freeze to true

  
  
Posted 3 years ago

Yes I think it needs pytorch, but pytorch failed to install previously ?

  
  
Posted 3 years ago

VirtuousFish83
could that be that "inplace-abn" while installing the package needs torch ?

  
  
Posted 3 years ago

Ok, btw I used https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_agent_install_configure.html which was not updated so I didn't know there was a priority_packages and post_packages

  
  
Posted 3 years ago

Try to add inplace-abn to the post_packages section https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf#L79 , and it will be installed after all the other packages

  
  
Posted 3 years ago

Yes the setup.py imports torch unfortunately https://github.com/mapillary/inplace_abn/blob/master/setup.py

  
  
Posted 3 years ago

Hmm it's both better and worse, it does detect pyfunctional now (in INSTALLED PACKAGES and I can see it installed in the console logs) but it fails on
import torch ModuleNotFoundError: No module named 'torch'In the logs:
Found PyTorch version torch==1.7.1 matching CUDA version 110 2021-04-21 15:15:11 Found PyTorch version torchvision==0.8.2 matching CUDA version 110 Collecting torch==1.7.1+cu110 File was already downloaded /home/ubuntu/.clearml/pip-download-cache/cu110/torch-1.7.1+cu110-cp38-cp38-linux_x86_64.whl Successfully downloaded torch Collecting torchvision==0.8.2+cu110 File was already downloaded /home/ubuntu/.clearml/pip-download-cache/cu110/torchvision-0.8.2+cu110-cp38-cp38-linux_x86_64.whl Successfully downloaded torchvision Processing /home/ubuntu/.clearml/pip-download-cache/cu110/torch-1.7.1+cu110-cp38-cp38-linux_x86_64.whl Processing /home/ubuntu/.clearml/pip-download-cache/cu110/torchvision-0.8.2+cu110-cp38-cp38-linux_x86_64.whl

  
  
Posted 3 years ago

Hi VirtuousFish83 ,

Do you have this package in the INSTALLED PACKAGES section of your task (under EXECUTION tab)?

  
  
Posted 3 years ago

I think so

  
  
Posted 3 years ago
736 Views
15 Answers
3 years ago
one year ago
Tags