Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, I Have Seen This Error Somewhere Else In The Channel Where The Docker Agent Gets Stuck At This Warning, And Does Not Move Forward, Requirement Already Satisfied, Skipping Upgrade: Pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 In /Usr/Local/Lib/Pyt

Hi All, I have seen this error somewhere else in the channel where the docker agent gets stuck at this warning, and does not move forward,
Requirement already satisfied, skipping upgrade: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.9/dist-packages (from jsonschema<5.0.0,>=2.6.0->clearml-agent) (0.18.1)
Requirement already satisfied, skipping upgrade: orderedmultidict>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from furl<2.2.0,>=2.0.0->clearml-agent) (1.0.1)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests<2.29.0,>=2.20.0->clearml-agent) (2019.11.28)
Requirement already satisfied, skipping upgrade: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests<2.29.0,>=2.20.0->clearml-agent) (2.8)
Requirement already satisfied, skipping upgrade: charset-normalizer<4,>=2 in /usr/local/lib/python3.9/dist-packages (from requests<2.29.0,>=2.20.0->clearml-agent) (3.1.0)
Installing collected packages: distlib, platformdirs, virtualenv, clearml-agent
Successfully installed clearml-agent-1.5.2 distlib-0.3.6 platformdirs-3.2.0 virtualenv-20.22.0
WARNING: You are using pip version 20.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3.9 -m pip install --upgrade pip' command.
Many thanks!

  
  
Posted one year ago
Votes Newest

Answers 18


@<1559711593736966144:profile|SoggyCow20> how did you run the session task?

  
  
Posted one year ago

Hi @<1559711593736966144:profile|SoggyCow20> , can you attach the full log?

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> Update - I tried to also run ClearML on my other personal laptop, I still do face the same issue where the docker agent gets stuck after some pip installations, even after being left to run for 10 minutes at that state, kindly find the attached log,
this is the command
clearml-agent daemon --cpu-only --docker pytorch/pytorch --queue default

  
  
Posted one year ago

Hi @<1523701087100473344:profile|SuccessfulKoala55> !
many thanks for your reply!
kindly find the attached log

  
  
Posted one year ago

To provide you with more info, I am setting up clearml on a linux server, and the agent is also set up on the same server.
I even tried this on the local machine MACOS with an agent on the same machine, and had the same issue.

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> I found out I had some issues with the clearml_agent docker setup, lacking some environment variables. What should be the value for CLEARML_HOST_IP?

  
  
Posted one year ago

One more update, I tried running the execute_remotely from app.clear.ml with a runner on local machine and with docker. it worked fine

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> is clearml-session to initiate a queue?

  
  
Posted one year ago

The docker image getting stuck does not have to do with the pip warning, I managed to surpress that through an environment variable

  
  
Posted one year ago

I did not need to launch it however when I was using app-clearml, why is it needed when running things locally?

  
  
Posted one year ago

@<1559711593736966144:profile|SoggyCow20> I see this in the log Process terminated by user - how exactly did you run this session task?

  
  
Posted one year ago

clearml-session will automatically enqueue a task to the queue you specify

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> i stopped the runner execution by Ctrl+C in the terminal

  
  
Posted one year ago

Another Message it gets stuck at sometimes is
cp: -r not specified; omitting directory '/tmp/clearml.conf'
Using built-in ClearML default key/secret
and this is the log for it

  
  
Posted one year ago

@<1559711593736966144:profile|SoggyCow20> clearml session must be run using the clearml-session command line took, you canno't run it manually or enqueue it yourself

  
  
Posted one year ago

the command to set up the agent is
clearml-agent daemon --docker pytorch_test --queue default --foreground --cpu-only

  
  
Posted one year ago

I guess there must have been some confusion, I am not trying to run a clearml-session, or a session, I am trying to execute a task remotely, such as Clone of Scalar reporting example. When I try to clone it and execute it to a queue, the docker image is getting stuck after the pip installations. Which works if i connect to app.clear.ml, but does not work when i connect to localhost docker deployment

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> apologies for my late replies. I tried to run it through the frontend WebAPP, and I also tried to use the clearml/examples/advanced/execute_remotely.py

  
  
Posted one year ago