Hey. I Want To Run Task Remotely, But Neither The Clearml Server Nor Worker Can Clone The Repository With Training Code (They Dont Have Access). Can I Somehow Tell The Task That It Has To Use Local Copy Of The Package? I.E.:

Answered

hey. I want to run task remotely, but neither the clearml server nor worker can clone the repository with training code (they dont have access). can i somehow tell the task that it has to use local copy of the package? I.e.:
Task.force_store_standalone_script(True) Task.add_requirements(local_directory_with_setup_py_in_it) clearml_task = Task.init(....)Unfortunately I can only pass name of package or requirementst.txt to "add_requirements". What could be a workaround?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					BitterRabbit31
				
					0
					 × 1

Votes Newest

Answers 4

was rather on infrastructure level - worker doesnt have access to the repository. I found workaround of using mirror repo that is accessible by worker..

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					BitterRabbit31
				
					0
					 × 1

Why not give ClearML read-only access credentials to the repository?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					UnevenDolphin73
				
					0
					 × 1

okay, after digging a little, I think the snippet above is not valid. "standalone script" - assumes just a single python file. I still need repo where training scripts live. It just cant be cloned on remote worker. Is it possible to somehow pack local repo to be executed on remote worker?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					BitterRabbit31
				
					0
					 × 1

Hi,

can you paste the error you're getting?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Write your answer

1K Views

4 Answers

2 years ago

one year ago