Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi All! I Am A Bit Confused As To How The Python Environment Is Set. I Can Submit Jobs That Build The Environment And Run Perfectly Fine. But, If I Abort The Job -> Requeue It From The Gui, Then A Different Environment Is Installed (Which Has Some Package

Hi all! I am a bit confused as to how the Python environment is set. I can submit jobs that build the environment and run perfectly fine. But, if I abort the job -> requeue it from the GUI, then a different environment is installed (which has some package dependency conflicts that prevent it from running).

What is causing the environment to change on requeue, and how do I stop it?

Posted 4 months ago
Votes Newest

Answers 10

Woo, what a doozy. Thanks for the debug @<1523701205467926528:profile|AgitatedDove14> ! Will move forward with your suggestions.

Posted 4 months ago

Woo, what a doozy.

yeah those "broken" pip versions are making our life hard ...

Posted 4 months ago

the first runs perfectly fine,

Just making sure, running in an agent?

the second crashes

Running inside the same container as the first one ?

Posted 4 months ago


Posted 4 months ago

I guess what I'm confused about is that the final resolved environment is different between the first manual execution and the reproduced one -- the first runs perfectly fine, the second crashes and fails to make the environment.

Posted 4 months ago

Yes and yes

Posted 4 months ago

Hi @<1523701797800120320:profile|SteadySeagull18>

...the job -> requeue it from the GUI, then a different environment is installed

The way that it works is, in the "originating" (i.e. first manual) execution only the directly imported packages are listed (no derivative packages that re required by the original packages)
But when the agent is reproducing the job, it creates a whole clean venv for the experiment, installs the required packages, then pip resolves the derivatives, and then the agent Freezes the entire venv to be later reproduced exactly as it was.

make sense ?

Posted 4 months ago

One suggestion is to make sure all agents have the same configuration. Another is to add pip into the "installed packages" section.
(Notice that in the next release we will specifically include it there, to avoid these kind of scenarios)

Posted 4 months ago

this is very odd, can you post the log?

Posted 4 months ago

Yes that was a tricky one, basically always blame pip 🙂
One machine (original parent)

agent.package_manager.type = pip
agent.package_manager.pip_version = 

Which would not upgrade the pip and use the preinstalled Unpacking python-pip-whl (20.0.2-5ubuntu1.10)
The other one has:

agent.package_manager.type = pip
agent.package_manager.pip_version.0 = <20.2 ; python_version < '3.10'
agent.package_manager.pip_version.1 = <22.3 ; python_version >\= '3.10' 

and it installed - Successfully installed pip-20.1.1
which for some reason tried to install "pycairo" even though it was not listed anywhere

Posted 4 months ago
10 Answers
4 months ago
4 months ago