Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Am Trying Pytorch Nightly Again With Python 3.10. Works Fine Locally, But Fails On Clearml-Agent In Docker Mode.

I am trying PyTorch Nightly again with Python 3.10. Works fine locally, but fails on clearml-agent in docker mode.

ERROR: Ignored the following versions that require a different python version: 0.0.322773188 Requires-Python >=3.6, <3.10; 0.0.355168290 Requires-Python >=3.6, <3.10; 0.0.364896371 Requires-Python >=3.6, <3.10; 0.0.403778684 Requires-Python >=3.7, <3.10; 0.0.423341706 Requires-Python >=3.7, <3.10; 0.0.425341097 Requires-Python >=3.7, <3.10; 1.0.0 Requires-Python >=3.7, <=3.10; 1.0.1 Requires-Python >=3.7, <=3.10; 1.0.2 Requires-Python >=3.7, <=3.10; 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.0rc1 Requires-Python >=3.7,<3.10; 1.7.0rc2 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10
ERROR: Could not find a version that satisfies the requirement torch==2.1.0.* (from versions: 1.11.0, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 2.0.0, 2.0.0+cu118, 2.1.0.dev20230305+cu117, 2.1.0.dev20230305+cu118, 2.1.0.dev20230306+cu117, 2.1.0.dev20230306+cu118, 2.1.0.dev20230307+cu117, 2.1.0.dev20230307+cu118, 2.1.0.dev20230308+cu117, 2.1.0.dev20230308+cu118, 2.1.0.dev20230309+cu117, 2.1.0.dev20230309+cu118, 2.1.0.dev20230310+cu117, 2.1.0.dev20230310+cu118, 2.1.0.dev20230311+cu117, 2.1.0.dev20230311+cu118, 2.1.0.dev20230312+cu117, 2.1.0.dev20230312+cu118, 2.1.0.dev20230313+cu117, 2.1.0.dev20230313+cu118, 2.1.0.dev20230314+cu117, 2.1.0.dev20230314+cu118, 2.1.0.dev20230315+cu117, 2.1.0.dev20230315+cu118, 2.1.0.dev20230316+cu117, 2.1.0.dev20230316+cu118, 2.1.0.dev20230317+cu117, 2.1.0.dev20230317+cu118, 2.1.0.dev20230318+cu117, 2.1.0.dev20230318+cu118, 2.1.0.dev20230319+cu117, 2.1.0.dev20230319+cu118, 2.1.0.dev20230320+cu117, 2.1.0.dev20230320+cu118, 2.1.0.dev20230321+cu117, 2.1.0.dev20230321+cu118, 2.1.0.dev20230322+cu117, 2.1.0.dev20230322+cu118, 2.1.0.dev20230323+cu117, 2.1.0.dev20230323+cu118, 2.1.0.dev20230324+cu117, 2.1.0.dev20230324+cu118, 2.1.0.dev20230325+cu117, 2.1.0.dev20230325+cu118, 2.1.0.dev20230326+cu117, 2.1.0.dev20230326+cu118, 2.1.0.dev20230327+cu117, 2.1.0.dev20230327+cu118, 2.1.0.dev20230328+cu117, 2.1.0.dev20230328+cu118, 2.1.0.dev20230329+cu117, 2.1.0.dev20230329+cu118, 2.1.0.dev20230330+cu117, 2.1.0.dev20230330+cu118, 2.1.0.dev20230331+cu117, 2.1.0.dev20230331+cu118, 2.1.0.dev20230401+cu117, 2.1.0.dev20230401+cu118, 2.1.0.dev20230402+cu117, 2.1.0.dev20230402+cu118, 2.1.0.dev20230403+cu117, 2.1.0.dev20230403+cu118, 2.1.0.dev20230404+cu117, 2.1.0.dev20230404+cu118, 2.1.0.dev20230405+cu117, 2.1.0.dev20230405+cu118, 2.1.0.dev20230406+cu117, 2.1.0.dev20230406+cu118, 2.1.0.dev20230407+cu117, 2.1.0.dev20230407+cu118, 2.1.0.dev20230408+cu117, 2.1.0.dev20230408+cu118, 2.1.0.dev20230409+cu117, 2.1.0.dev20230409+cu118, 2.1.0.dev20230410+cu117, 2.1.0.dev20230410+cu118, 2.1.0.dev20230411+cu117, 2.1.0.dev20230411+cu118, 2.1.0.dev20230412+cu117, 2.1.0.dev20230412+cu118, 2.1.0.dev20230413+cu117, 2.1.0.dev20230413+cu118, 2.1.0.dev20230414+cu117, 2.1.0.dev20230414+cu118, 2.1.0.dev20230415+cu117, 2.1.0.dev20230415+cu118, 2.1.0.dev20230416+cu117, 2.1.0.dev20230416+cu118, 2.1.0.dev20230417+cu117, 2.1.0.dev20230417+cu118, 2.1.0.dev20230418+cu117, 2.1.0.dev20230418+cu118, 2.1.0.dev20230419+cu117, 2.1.0.dev20230419+cu118, 2.1.0.dev20230420+cu117, 2.1.0.dev20230420+cu118, 2.1.0.dev20230421+cu117, 2.1.0.dev20230421+cu118, 2.1.0.dev20230422+cu117, 2.1.0.dev20230422+cu118, 2.1.0.dev20230423+cu117, 2.1.0.dev20230423+cu118, 2.1.0.dev20230424+cu117, 2.1.0.dev20230424+cu118, 2.1.0.dev20230425+cu117, 2.1.0.dev20230425+cu118, 2.1.0.dev20230426+cu117, 2.1.0.dev20230426+cu118, 2.1.0.dev20230427+cu117, 2.1.0.dev20230427+cu118, 2.1.0.dev20230428+cu117, 2.1.0.dev20230428+cu118, 2.1.0.dev20230429+cu117, 2.1.0.dev20230429+cu118, 2.1.0.dev20230430+cu117, 2.1.0.dev20230430+cu118, 2.1.0.dev20230501+cu117, 2.1.0.dev20230501+cu118, 2.1.0.dev20230502+cu117, 2.1.0.dev20230503+cu117)
ERROR: No matching distribution found for torch==2.1.0.*

In installed packages:

torch == 2.1.0.dev20230429+cu118

Driver:

 NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0

Any idea? Latest clearml-agent!

  
  
Posted one year ago
pic
More
ReassuredTiger98
0 × 1
Report
Votes Newest

Answers 17

So this is verry odd, it looks like a pip bug:
The agent is trying to install torch==2.1.0.* because by default it ignores the 4th+ parts (they are unstable and torch have tendency to remove them) . and for some reason pip will not match 2.1.0.* with for example "2.1.0.dev20230306+cu118"
but based on the docs it should work:
see here: None

As a workaround you can always edit and change to the final url for example: so instead of:

 torch == 2.1.0.dev20230429+cu118

you should have just the the link (notice no torch package name, but make sure the python / arch is corect)
  
  
Posted one year ago
pic
More
AgitatedDove14
0 × 1
Report

seems like pip 20.1.1 has the issue, but >= 22.2.2 do not.

  
  
Posted one year ago
pic
More
NastyFox63
0 × 1
Report

Thanks for researching this issue. If you have time, you can create the issue since you are way more knowledgeable, but I can also open it if you do not have time 🙂

  
  
Posted one year ago
pic
More
ReassuredTiger98
0 × 1
Report

Full log

  
  
Posted one year ago
pic
More
ReassuredTiger98
0 × 1
Report

i am having this same issue—installing pytorch via pip. but i am not specifying a version, and the agent is not able to install pytorch.

even if i specify a version (e.g. torch<2.0 ), it fails.

i guess this is a pip problem, is there a known pip version that works correctly?

  
  
Posted one year ago
pic
More
NastyFox63
0 × 1
Report

the issue also may have been fixed somewhere between 20.1 and 22.2, i didn’t test versions in between those two

  
  
Posted one year ago
pic
More
NastyFox63
0 × 1
Report

So was the issue solved?

  
  
Posted one year ago
pic
More
AgitatedDove14
0 × 1
Report

but there was a pip_version: “<20.2” line in myclearml.conf` , which would possibly have been a default in the config file like, 2 years ago or something

  
  
Posted one year ago
pic
More
NastyFox63
0 × 1
Report

I installed as told on pytorch.org : pip3 install --pre torch torchvision torchaudio --index-url None

  
  
Posted one year ago
pic
More
ReassuredTiger98
0 × 1
Report

@<1523701868901961728:profile|ReassuredTiger98> how did you install the nightly locally ?
Can you also provide the full log?

  
  
Posted one year ago
pic
More
AgitatedDove14
0 × 1
Report

i tried lots of things, but values in the conf file (specifically the pip and cuda versions) overriding things in my code/env confused me for a long time

  
  
Posted one year ago
pic
More
NastyFox63
0 × 1
Report

Hi @<1523701868901961728:profile|ReassuredTiger98> , what do you mean by "working fine locally"?

  
  
Posted one year ago
pic
More
SuccessfulKoala55
0 × 1
Report

i noticed that the agent was downgrading to pip=20.1.1 at every attempt, so i added

Task.add_requirements("pip", "23.1.2")

and even then, it downgrades to 20.1.1?

  
  
Posted one year ago
pic
More
NastyFox63
0 × 1
Report

seems like pip 20.1.1 has the issue, but >= 22.2.2 do not.

Notice we changed the value there, it now has two versions, pne for python 3.10 < and one for python 3.10>=
The main reason is that pip changed their resolving algorithm, and the new one can break its own dependencies (i.e. pip freeze > requirements.txt -> pip install might not actually work)
None

  
  
Posted one year ago
pic
More
AgitatedDove14
0 × 1
Report

I mean that locally I was able to install the correct version without a problem.

  
  
Posted one year ago
pic
More
ReassuredTiger98
0 × 1
Report

Do you want to open an issue in pip?
Funny enough this works in:

pip3 install "torch >=2.1.0.*, <2.1.1.*" --extra-index-url
  
  
Posted one year ago
pic
More
AgitatedDove14
0 × 1
Report

ah, my mistake, that’s an issue in my conf file.

  
  
Posted one year ago
pic
More
NastyFox63
0 × 1
Report
Write your answer
1K Views
17 Answers
one year ago
one year ago
Tags
clearml