Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! I'M Running A Clearml Task Remotely. It Runs A Python Machine Learning Evaluation Script In Repo A. The Script Depends On A Module Which Is Defined In Another One Of My Company'S Repos: Repo B. The File Structure Of The Module In Repo B Is As Follows

Hi! I'm running a ClearML Task remotely. It runs a Python machine learning evaluation script in repo A. The script depends on a module which is defined in another one of my company's repos: repo B.

The file structure of the module in repo B is as follows:

-module_b.py
-templates
--my_template.yml

The evaluation script calls into module_b, which tries to load my_template.yml

When I run this locally without using ClearML, the YAML file can be successfully loaded. But when I run this remotely on ClearML, I get the following error:

FileNotFoundError: [Errno 2] No such file or directory: '/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/module_b/templates/my_template.yml'

Any idea why this might be happening? I originally thought the static template files weren't being bundled up properly with module B, but I checked the setup.py file and the call to setuptools.setup() takes in this parameter: include_package_data=True

So I'd expect this to be working fine ๐Ÿค”

  
  
Posted 6 months ago
Votes Newest

Answers 6


Hi, @<1523701070390366208:profile|CostlyOstrich36> I think I could have explained myself better. So the Task is running a script in repo A. But that script depends on module B, which is defined in repo B. I have a clearml_requirements.txt file in repo A. When the remote . ClearML Task runs, it installs the dependencies listed in that .txt file, including module B.

The problem is that the remote ClearML server is unable to find the static files that should come bundled along with module B.

Is that a bit clearer? ๐Ÿคž

  
  
Posted 6 months ago

Yes, itโ€™s running in Docker and thanks for the suggestion. Iโ€™ll try a manual install ๐Ÿ‘

  
  
Posted 6 months ago

or simply create a new venv in your local PC, then install your package with pip install from repo url and see if your file is deployed properly in that venv

  
  
Posted 6 months ago

so the issue is that for some reason, the pip install by the agent don't behave the same way as your local pip install ?
Have you tried to manually install your module_b with pip install inside the machine that is running clearml-agent ? Seeing your example, looks like you are even running inside docker ?

  
  
Posted 6 months ago

Hi @<1614069770586427392:profile|FlutteringFrog26> , if I'm not mistaken ClearML doesn't support running from different repoes. You can only clone one code repository per task. Is there a specific reason these repoes are separate?

  
  
Posted 6 months ago

once you install manually your package inside the docker container, check that your file module_b/templates/my_template.yml is where it should be

  
  
Posted 6 months ago