Tada! https://github.com/allegroai/clearml-agent/issues/90

clearml-agent  repo please 🙂

Should this be under the clearml or clearml-agent repo?

Haha, I've opened so many issues these past few days... Sure, np!

UnevenDolphin73 sounds great, any chance you can open a git issue on clearml-agent repo for this feature request ?

Anyway sounds good! 🙂

Right so it uses whatever version is available on the agent.
Yeah it would be nice to have either a poetry_version (a-la https://github.com/allegroai/clearml-agent/blob/5afb604e3d53d3f09dd6de81fe0a494dacb2e94d/docs/clearml.conf#L62 ), rename the latter to manager_version , or just install from the captured environment, etc? 🤔

for future reference this is indeed a PEP-610 related bug, f

👍

can we also set the 

poetry

 version used?....

Actually the agent assumes poetry is preinstalled (so whatever you already have on the docker) ...
That said, maybe we should install a specific version (after installing pip, we could do that if poetry is selected)
wdyt ?

AgitatedDove14 for future reference this is indeed a PEP-610 related bug, fixed in https://python-poetry.org/blog/announcing-poetry-1.2.0a1/ . I see we can choose the pip version in the config, can we also set the poetry version used? Or is it updated from the lock file itself, or...?

Hmmm maybe 

 I thought that was expected behavior from poetry side actually

I think this is the expected behavior, hence bug?!

Hmmm maybe 🤔 I thought that was expected behavior from poetry side actually

poetry

 stores git related data in ... you get an internal package we have with its version, but no git reference, i.e. 

internal_module==1.2.3

 instead of 

internal_module @H4dr1en

This seems like a bug with poetry (and I think I have run into this one), worth reporting it, no?

Here's how it failed for us 😅
poetry stores git related data in poetry.lock , so when you pip list , you get an internal package we have with its version, but no git reference, i.e. internal_module==1.2.3 instead of internal_module @ git+https://....@commit .
Then pip actually fails (our internal module is not on pypi), but poetry suceeds

The other way will not work, as if you start with "pip" you cannot fail ... (if you fail it's in run time which is too late)

Fair enough 😄
Could be nice to be able to define the fallbacks under type maybe?
type: [ poetry, pip ] (current way under the hood) vs type: [ pip, poetry ]

Local changes are applied before installing requirements, right?

correct

I'm with on this one 🙂 it better to make a company wide decision on these things and not allow too much flexibility (just two options to choose from, and it should be enough, I think)

Local changes are applied before installing requirements, right?

Yeah I figured (2) would be the way to go actually 😄

Task.force_requirements_env_freeze()

This might be very brittle, if users are running on a diff OS, or python versions...
I would actually go with:
you like poetry, update your lock file in git you do not use poetry, work on your own branch and delete poetry lock file
wdyt?

Maybe it's better to approach this the other way, if one uses Task.force_requirements_env_freeze() , then the locally updated packages aren't reflected in poetry 🤔

Not necessarily on the same branch, no

Or some users that update their poetry.lock and some that update manually as they prefer to resolve on their own.

so you have a repo with poetry that some users update and some do not?
All working on the same branch ?

UnevenDolphin73 if the repo does not include a poetry file it will revert to pip

My current workaround is to use poetry and tell users to delete poetry.lock if they want their environment copied verbatim

first try the current setup using 

pip

, and if it fails, use 

poetry

 if 

poetry.lock

 exists

I guess the order here is not clear to me (the agent does the opposite), why would you start with pip if you are using poetry ?

The tl;dr is that some of our users like poetry and others prefer pip . Since pip install git+.... stores the git data, it seems trivial to first try and install based on pip , and only later on poetry , since the pip would crash with poetry as it stores git data elsewhere (in poetry.lock )

Those are for specific packages, I'm wondering about the package managers as a whole

LOL 🙂