Hey. I Want To Run Task Remotely, But Neither The Clearml Server Nor Worker Can Clone The Repository With Training Code (They Dont Have Access). Can I Somehow Tell The Task That It Has To Use Local Copy Of The Package? I.E.:

Answered

hey. I want to run task remotely, but neither the clearml server nor worker can clone the repository with training code (they dont have access). can i somehow tell the task that it has to use local copy of the package? I.e.:
Task.force_store_standalone_script(True) Task.add_requirements(local_directory_with_setup_py_in_it) clearml_task = Task.init(....)Unfortunately I can only pass name of package or requirementst.txt to "add_requirements". What could be a workaround?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BitterRabbit31
				
					0
					 × 1

Votes Newest

Answers 4

okay, after digging a little, I think the snippet above is not valid. "standalone script" - assumes just a single python file. I still need repo where training scripts live. It just cant be cloned on remote worker. Is it possible to somehow pack local repo to be executed on remote worker?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BitterRabbit31
				
					0
					 × 1

Hi,

can you paste the error you're getting?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

was rather on infrastructure level - worker doesnt have access to the repository. I found workaround of using mirror repo that is accessible by worker..

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BitterRabbit31
				
					0
					 × 1

Why not give ClearML read-only access credentials to the repository?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					UnevenDolphin73
				
					0
					 × 1

Write your answer

634 Views

4 Answers

one year ago