Is Setting Up Clearml Server On Same Machine That Is Supposed To Be Our Gpu Worker A Good Idea? I Wanted To Have Every Dataset Locally Using Clearml-Data Cli, How Would Gpu Worker Handle Data Fetching, Would It Go Straight From Disc?

Answered

Is setting up ClearML server on same machine that is supposed to be our GPU worker a good idea?
I wanted to have every dataset locally using clearml-data CLI, how would gpu worker handle data fetching, would it go straight from disc?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					ObnoxiousStork61
				
					0
					 × 1

Votes Newest

Answers 3

Hi ObnoxiousStork61 ,
In general, I think setting up the server on the GPU machine running the experiment is not a good idea - the server is supposed to run in a stable environment, whereas the GPU machine is more dynamic in nature.
Regarding data, since the data is stored in a network storage, and assuming the network storage is local (i.e. in your own network), I don't think data fetching will be an issue...

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

So ideally I should setup one machine with storage for datasets for clearml-server and one GPU machine to be clearml-agent with storage enough to fetch data?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					ObnoxiousStork61
				
					0
					 × 1

Yes, one machine for server and storage, GPU machine with local storage for data fetching and caching :)

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

Write your answer

581 Views

3 Answers

3 years ago

one year ago