Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Is There A Way To Set Precedence On Package Managers? If We Set An Agent To Use

Is there a way to set precedence on package managers? If we set an agent to use poetry it will ignore local packages, even if I use the force_requirements_env_freeze
In our case, we would like to try and use pip first (for an exact copy of the local virtual environment), and only if it fails, then use poetry

  
  
Posted 2 years ago
Votes Newest

Answers 30


first try the current setup using 

pip

, and if it fails, use 

poetry

 if 

poetry.lock

 exists

I guess the order here is not clear to me (the agent does the opposite), why would you start with pip if you are using poetry ?

  
  
Posted 2 years ago

so you have a repo with poetry that some users update and some do not?
All working on the same branch ?

  
  
Posted 2 years ago

Not necessarily on the same branch, no

  
  
Posted 2 years ago

Haha, I've opened so many issues these past few days... Sure, np!

  
  
Posted 2 years ago

Task.force_requirements_env_freeze()

This might be very brittle, if users are running on a diff OS, or python versions...
I would actually go with:
you like poetry, update your lock file in git you do not use poetry, work on your own branch and delete poetry lock file
wdyt?

  
  
Posted 2 years ago

Those are for specific packages, I'm wondering about the package managers as a whole

  
  
Posted 2 years ago

Maybe it's better to approach this the other way, if one uses Task.force_requirements_env_freeze() , then the locally updated packages aren't reflected in poetry 🤔

  
  
Posted 2 years ago

My current workaround is to use poetry and tell users to delete poetry.lock if they want their environment copied verbatim

  
  
Posted 2 years ago

Yeah I figured (2) would be the way to go actually 😄

  
  
Posted 2 years ago

UnevenDolphin73 if the repo does not include a poetry file it will revert to pip

  
  
Posted 2 years ago

Should this be under the clearml or clearml-agent repo?

  
  
Posted 2 years ago

Anyway sounds good! 🙂

  
  
Posted 2 years ago

Hmmm maybe 

 I thought that was expected behavior from poetry side actually

I think this is the expected behavior, hence bug?!

  
  
Posted 2 years ago

LOL 🙂

  
  
Posted 2 years ago

for future reference this is indeed a PEP-610 related bug, f

👍

can we also set the 

poetry

 version used?....

Actually the agent assumes poetry is preinstalled (so whatever you already have on the docker) ...
That said, maybe we should install a specific version (after installing pip, we could do that if poetry is selected)
wdyt ?

  
  
Posted 2 years ago

clearml-agent  repo please 🙂

  
  
Posted 2 years ago

UnevenDolphin73 sounds great, any chance you can open a git issue on clearml-agent repo for this feature request ?

  
  
Posted 2 years ago

Hmmm maybe 🤔 I thought that was expected behavior from poetry side actually

  
  
Posted 2 years ago

I'm with on this one 🙂 it better to make a company wide decision on these things and not allow too much flexibility (just two options to choose from, and it should be enough, I think)

  
  
Posted 2 years ago

Right so it uses whatever version is available on the agent.
Yeah it would be nice to have either a poetry_version (a-la https://github.com/allegroai/clearml-agent/blob/5afb604e3d53d3f09dd6de81fe0a494dacb2e94d/docs/clearml.conf#L62 ), rename the latter to manager_version , or just install from the captured environment, etc? 🤔

  
  
Posted 2 years ago

The tl;dr is that some of our users like poetry and others prefer pip . Since pip install git+.... stores the git data, it seems trivial to first try and install based on pip , and only later on poetry , since the pip would crash with poetry as it stores git data elsewhere (in poetry.lock )

  
  
Posted 2 years ago

AgitatedDove14 for future reference this is indeed a PEP-610 related bug, fixed in https://python-poetry.org/blog/announcing-poetry-1.2.0a1/ . I see we can choose the pip version in the config, can we also set the poetry version used? Or is it updated from the lock file itself, or...?

  
  
Posted 2 years ago

Here's how it failed for us 😅
poetry stores git related data in poetry.lock , so when you pip list , you get an internal package we have with its version, but no git reference, i.e. internal_module==1.2.3 instead of internal_module @ git+https://....@commit .
Then pip actually fails (our internal module is not on pypi), but poetry suceeds

  
  
Posted 2 years ago

The other way will not work, as if you start with "pip" you cannot fail ... (if you fail it's in run time which is too late)

  
  
Posted 2 years ago

Local changes are applied before installing requirements, right?

  
  
Posted 2 years ago

Fair enough 😄
Could be nice to be able to define the fallbacks under type maybe?
type: [ poetry, pip ] (current way under the hood) vs type: [ pip, poetry ]

  
  
Posted 2 years ago

poetry

 stores git related data in ... you get an internal package we have with its version, but no git reference, i.e. 

internal_module==1.2.3

 instead of 

internal_module @H4dr1en

This seems like a bug with poetry (and I think I have run into this one), worth reporting it, no?

  
  
Posted 2 years ago

Local changes are applied before installing requirements, right?

correct

  
  
Posted 2 years ago

Or some users that update their poetry.lock and some that update manually as they prefer to resolve on their own.

  
  
Posted 2 years ago