Hello, I Have A Server With Several Gpus That I Wish To Use To Automatically Train Models. Clearml Seems Like The Perfect Tool For My Use Case But I Am Confused As To How I Can Communicate With The Agent Daemon Running On The Gpu Server. Do I Necessarily

Answered

Hello,
I have a server with several GPUs that I wish to use to automatically train models.
ClearML seems like the perfect tool for my use case but I am confused as to how I can communicate with the agent daemon running on the gpu server.
Do I necessarily need to host a ClearML server to act as a proxy between my machine and the gpu server, or is there a way to communicate with the agent directly (for example using rabbitmq) ?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ShinyRabbit94
				
					0
					 × 1

Votes Newest

Answers 4

ShinyRabbit94 , Hey 🙂

The clearml-agent communicates with a server and receives jobs according to a "queue" connected to it. To run the agent properly you'll need to work against a ClearML server. You can either host one yourself OR use the community server. I think you can read a bit more here:

https://clear.ml/docs/latest/docs/clearml_agent#installation
https://clear.ml/docs/latest/docs/webapp/webapp_profile/#creating-clearml-credentials

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Okey thanks!

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ShinyRabbit94
				
					0
					 × 1

You'll need ES/Mongo to run the ClearML server

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Okey thank you!
If I plan using S3 for external file storage, do I still need Elasticsearch and Mongo ?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ShinyRabbit94
				
					0
					 × 1

Write your answer

963 Views

4 Answers

2 years ago

one year ago