Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi There! Is There An Easy Way To Retrieve The Site-Package Directory That Was Created By An Agent From Inside A Task? Eg.

Hi there! Is there an easy way to retrieve the site-package directory that was created by an agent from inside a task? Eg.
` task = Task.init(...)
task.add_requirements("requirements.txt") # Will be installed in a virtualenv
print(task.get_site_package_dir)

'/home/ubuntu/clearml_agent_venv/lib/python3.10'. # Active virtualenv installed by clearml-agent where all dependencies from requirements where installed> I would need to retrieve this path to put it inside $PYTHONPATH (because virtualenv doesn't do it when source env/bin/activate is called) EDIT: Can I find it using sys.executable ` ?
EDIT 2: OR maybe is clearml-agent already populating this environment variable??

  
  
Posted one year ago
Votes Newest

Answers 12


The part where I'm lost is why would you need the path to the temp venv the agent creates/uses ?

let's say my task is calling a bash script, and that bash script is calling another python program, I want that last python program to be executed with the environment that was created by the agent for this specific task

  
  
Posted one year ago

Now I'm curious, what did you end up doing ?

in my repo I maintain a bash script to setup a separate python env. then in my task I spawn a subprocess and I don't pass the env variables, so that the subprocess properly picks up the separate python env

  
  
Posted one year ago

If you're just looking to reuse virtual environments, have a look here: https://clear.ml/docs/latest/docs/clearml_agent/#environment-caching

  
  
Posted one year ago

in my repo I maintain a bash script to setup a separate python env.

Hmm interesting, now I have to wonder what is the difference ? meaning why doesn't the agent build a similar one based on the requirements ?

  
  
Posted one year ago

so that one app I am using inside the Task can use the python packages installed by the agent and I can control the packages using clearml easily

That's the missing part for me, You have all the requiremnts on the Task (that you can fully control), the agent is setting a brand new venv for each Task inside a container (the venv is cahced, and you can also make the agent just use the default python without installing anything). The part where I'm lost is why would you need the path to the temp venv the agent creates/uses ?

  
  
Posted one year ago

AgitatedDove14 I eventually found a different way of achieving what I needed

  
  
Posted one year ago

But that was too complicated, I found an easier approach

  
  
Posted one year ago

I want that last python program to be executed with the environment that was created by the agent for this specific task

Well basically they all inherit the Python environment that points to the venv they started from, so at least in theory it should be transparent when the agent is spinning the initial process.

I eventually found a different way of achieving what I needed

Now I'm curious, what did you end up doing ?

  
  
Posted one year ago

Yea again I am trying to understand what I can do with what I have 😄 I would like to be able to export as an environment variable the runtime where the agent is installing, so that one app I am using inside the Task can use the python packages installed by the agent and I can control the packages using clearml easily

  
  
Posted one year ago

Do you mean what's visible in the UI, projects -> Execution: Installed Packages?

  
  
Posted one year ago

JitteryCoyote63 you mean in runtime where the agent is installing? I'm not sure I fully understand the use case?!

  
  
Posted one year ago
651 Views
12 Answers
one year ago
one year ago
Tags