Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Attempting To Use

Hi,

I'm attempting to use clearml-session and have an agent started on the same machine with clearml-agent daemon . The agent starts up without issues. However when executing the clearml-session --docker my_docker command, the following issue happens at the end:

` [omitted logs]

Remote machine allocated
Setting remote environment [Task id=20d362aa966240e9ba44785c65ea6bd3]
Setup process details:
Waiting for environment setup to complete [usually about 20-30 seconds]
...........
Remote machine is ready
Setting up connection to remote session
Starting SSH tunnel
Warning: Permanently added '[192.168.0.148]:10028' (ECDSA) to the list of known hosts.
Password: 5297323bf121e3572266ef7e359342970b4aa9920c29393796a957866fa171be

Password: Error: incorrect password
Please enter password manually: incorrect password `

From reading about clearml-session and clearml-agent it seems that the SSH tunnel (started by clearml-session ) is connected to the SSH server that is started inside of the Docker container (started by the clearml-agent ). I can verify that the endpoint <AGENT_USER>@<AGENT_IP>:<AGENT_PORT> can be accessed with ssh but I don't know the password. How do I know the password?

Questions:
Any guess why the password is "incorrect" for me? In the code, I can see that the default password is training in the interactive_session_task.py ( "ssh_password": "training" ) script for the SSH server. So what is the clearml-session password used for if it is set by the clearml-agent ?
Also, on the ClearML dashboard, I can see the clearml-agent log:
` #

SSH Server running on <NAME> [IP] port 10028

LOGIN u:root p:None

Error: Cannot install code-server (not root) and could not find code-server executable, skipping. `

  
  
Posted 3 years ago
Votes Newest

Answers 8


Also, on the ClearML dashboard, I can see the 

clearml-agent

 log:

Is the clearml-agent running in docker mode ?

See https://github.com/allegroai/clearml-session/issues/3

  
  
Posted 3 years ago

clearml-agent --docker

Oh thanks. clearml-agent --help did not show the docker option, but clearml-agent daemon --help does, so I did not see it

  
  
Posted 3 years ago

Is the clearml-agent running in docker mode ?

How do you run the clearml-agent in docker mode? You execute the clearml-session in Docker mode?

  
  
Posted 3 years ago

Hi GracefulDog98

Any guess why the password is "incorrect" for me?

Basically the clearml-session CLI needs to be able to access (SSH) into the host (cleaml-agent) machine,
is that possible?

  
  
Posted 3 years ago

How do you run the 

clearml-agent

 in docker mode

clearml-agent --docker
See here:
https://clear.ml/docs/latest/docs/clearml_agent#docker-mode

  
  
Posted 3 years ago

I execute the 

clearml-session

 with 

--docker

 flag.

This is to control the docker image the agent will spin for you (think dev enviroment you want to work in, like nvidia pytorch container already having everything you need)

  
  
Posted 3 years ago

In the log for clearml-agent I can see the following, where <my_container> is the container:
docker_cmd = <my_container>:latest --network host entry_point = interactive_session.py working_dir = . Using base prefix '/usr/local' New python executable in /home/<user>/.clearml/venvs-builds/3.7/bin/python3.7 Also creating executable in /home/<user>/.clearml/venvs-builds/3.7/bin/python Installing setuptools, pip, wheel... done. 2021-08-04 16:50:03 Collecting pip<20.2 Using cached pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
But there's no log shown for pulling the container, and it seems to be using the local build. Why is that?

  
  
Posted 3 years ago

Thanks for the link. The only difference is that I execute the clearml-session with --docker flag. That seemed to be the issue in the post above?

  
  
Posted 3 years ago