Answered

Hi, I'M Attempting To Use

Hi,

I'm attempting to use clearml-session and have an agent started on the same machine with clearml-agent daemon . The agent starts up without issues. However when executing the clearml-session --docker my_docker command, the following issue happens at the end:

` [omitted logs]

Remote machine allocated
Setting remote environment [Task id=20d362aa966240e9ba44785c65ea6bd3]
Setup process details:
Waiting for environment setup to complete [usually about 20-30 seconds]
...........
Remote machine is ready
Setting up connection to remote session
Starting SSH tunnel
Warning: Permanently added '[192.168.0.148]:10028' (ECDSA) to the list of known hosts.
Password: 5297323bf121e3572266ef7e359342970b4aa9920c29393796a957866fa171be

Password: Error: incorrect password
Please enter password manually: incorrect password `

From reading about clearml-session and clearml-agent it seems that the SSH tunnel (started by clearml-session ) is connected to the SSH server that is started inside of the Docker container (started by the clearml-agent ). I can verify that the endpoint <AGENT_USER>@<AGENT_IP>:<AGENT_PORT> can be accessed with ssh but I don't know the password. How do I know the password?

Questions:
Any guess why the password is "incorrect" for me? In the code, I can see that the default password is training in the interactive_session_task.py ( "ssh_password": "training" ) script for the SSH server. So what is the clearml-session password used for if it is set by the clearml-agent ?
Also, on the ClearML dashboard, I can see the clearml-agent log:
` #

SSH Server running on <NAME> [IP] port 10028

LOGIN u:root p:None

Error: Cannot install code-server (not root) and could not find code-server executable, skipping. `

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					GracefulDog98
				
					0
					 × 1

Votes Newest

Answers 8

clearml-agent --docker

Oh thanks. clearml-agent --help did not show the docker option, but clearml-agent daemon --help does, so I did not see it

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					GracefulDog98
				
					0
					 × 1

I execute the

clearml-session

with

--docker

flag.

This is to control the docker image the agent will spin for you (think dev enviroment you want to work in, like nvidia pytorch container already having everything you need)

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

How do you run the

clearml-agent

in docker mode

clearml-agent --docker
See here:
https://clear.ml/docs/latest/docs/clearml_agent#docker-mode

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

In the log for clearml-agent I can see the following, where <my_container> is the container:
docker_cmd = <my_container>:latest --network host entry_point = interactive_session.py working_dir = . Using base prefix '/usr/local' New python executable in /home/<user>/.clearml/venvs-builds/3.7/bin/python3.7 Also creating executable in /home/<user>/.clearml/venvs-builds/3.7/bin/python Installing setuptools, pip, wheel... done. 2021-08-04 16:50:03 Collecting pip<20.2 Using cached pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
But there's no log shown for pulling the container, and it seems to be using the local build. Why is that?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					GracefulDog98
				
					0
					 × 1

Is the clearml-agent running in docker mode ?

How do you run the clearml-agent in docker mode? You execute the clearml-session in Docker mode?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					GracefulDog98
				
					0
					 × 1

Thanks for the link. The only difference is that I execute the clearml-session with --docker flag. That seemed to be the issue in the post above?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					GracefulDog98
				
					0
					 × 1

Also, on the ClearML dashboard, I can see the

clearml-agent

log:

Is the clearml-agent running in docker mode ?

See https://github.com/allegroai/clearml-session/issues/3

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Hi GracefulDog98

Any guess why the password is "incorrect" for me?

Basically the clearml-session CLI needs to be able to access (SSH) into the host (cleaml-agent) machine,
is that possible?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Write your answer

1K Views

8 Answers

3 years ago

one year ago