Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Having Trouble Creating My Experiment Environment In The Clearml-Agent Worker. I Have A Set Of Dependencies Which Are Hard To Setup, So I Opted To Work In Docker Mode. My Image Contains The Python Virtualenv Needed For My Experiments, But Clearml

Hi,
I'm having trouble creating my experiment environment in the clearml-agent worker.
I have a set of dependencies which are hard to setup, so I opted to work in docker mode.
My image contains the python virtualenv needed for my experiments, but clearml creates its own virtualenv which doesn't have my packages and doesn't configure PATH correctly.
I tried:
setting the PATH & VIRTUAL_ENV variables in the docker image. If I run python in the image I get the correct environment. Turning system_site_packages both on and offAny tips on getting this to work?
Bonus points:
Currently this setup means that I clone my repository in the docker image - so the commit & changes aren't reflected in this environment. Any way to remedy this?

  
  
Posted 3 years ago
Votes Newest

Answers 8


Hi ObliviousCrocodile95 ,

Trying to understand, you want to have the clearml-agent docker mode with pre install virtual environment?

Currently this setup means that I clone my repository in the docker image - so the commit & changes aren’t reflected in this environment. Any way to remedy this?

The clearml-agent should clone your repository and apply the changes from the parent task.

  
  
Posted 3 years ago

Yes - I'm using docker mode with a pre installed virtual environment. I have to use docker mode because my dependencies (including the virtualenv) have a long and complicated setup. I can't use the "system" python for installation (again, a dependency constrain).

I'm not sure how clearml-agent cloning my repo would work, since this repo is already cloned in my docker image. I think I'll understand this part better after I manage to solve the first issue.

  
  
Posted 3 years ago

Currently the agent create venv when you start it, but system_site_packages should install the packages from the docker

  
  
Posted 3 years ago

and it should be the default for docker mode

  
  
Posted 3 years ago

Can you try installing the package on the docker’s python but not on the venv?

  
  
Posted 3 years ago

I'm not sure that's possible.
I'm using a different python version than the system python.

  
  
Posted 3 years ago

And the packages versions doesn’t match the python? Can you install the python on the system (not as venv)?

  
  
Posted 3 years ago

Packages don't match the python version.
Installing the packages in the system python is not a long term solution. Even if I do manage to set it up and solve all the dependency conflicts- eventually an update will break this beyond repair.

  
  
Posted 3 years ago