Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, Coming Back With The Venv Caching: With The Following Setting:

Hi, coming back with the venv caching: with the following setting:
I call Task._update_requirements(["."]) setup.py has the following install_requires=["my-private-dep @ git+ https://github.com/my-org/my-private-dep@v1.7.0 "] in clearml-agent, pip==20.2.3 clearml-agent runs on commit f019905720529acbd316bd39b67c5ab0c02fcd55 clearml-agent has venv caching on clearml-agent has use_system_packages onWith the following scenario:
I update my-private-dep to 1.8.0 I started an experiment to testI encounter the following problem: clearml-agent did not update my-private-dep. pip recognized it as already installed and ignored it.
I need the make it install the version 1.8.0. I can ssh in the agent and install manually, but I would like this to happen automatically. Is it possible?
Note: if I disable venv caching, it works as expected (it reinstall my-private-dep)

  
  
Posted 3 years ago
Votes Newest

Answers 11


I update my-private-dep to 1.8.0

Not sure how this is connected with the venv, could you expand ?

  
  
Posted 3 years ago

I call task._update_requirements(my_reqs) regardless whether I am in the local machine or in the clearml agent, so "installed packages" section is always updated to the list my_reqs that I pass to the function, in this case ["."]

  
  
Posted 3 years ago

I mean the caching will work, but it will reinstall this repository on top of the cached copy.
make sense ?

  
  
Posted 3 years ago

yes, in setup.py I have:
..., install_requires= [ "my-private-dep @ git+ ", ... ], ...

  
  
Posted 3 years ago

replace it with:
git+No need for the repository name, this will ensure you always reinstall it (again pip feature)

  
  
Posted 3 years ago

Can you post them, I think there is something there that prevents the update (i.e. pip related).
For example:
packagename @ git+https:///....Will be translated by pip to:
If packagename is installed do nothing, if it is not installed use git+https://... to install it

  
  
Posted 3 years ago

ok, so there is no way to cache it and detect when the ref changes?

  
  
Posted 3 years ago

Yes, not sure it is connected either actually - To make it work, I had to disable both venv caching and set use_system_packages to off, so that it reinstalls the full env. I remember that we discussed this problem already but I don't remember what was the outcome, I never was able to make it update the private dependencies based on the version. But this is most likely a problem from pip that is not clever enough to parse the tag as a semantic version and check whether the installed package matches that version

  
  
Posted 3 years ago

JitteryCoyote63
Could you copy paste the "installed packages" section? the answer might be there

  
  
Posted 3 years ago

correct 🙂

  
  
Posted 3 years ago

Yes, I guess that's fine then - Thanks!

  
  
Posted 3 years ago