ClearML FAQ | Hi I Have A Question: I Have 2 Python Scripts: The First Python Script Is Running The 2. The Imports In The First Script Are Working. But When I Run The Programm On The Gpu I Get For Example The Mistake: No Module Named Tensorflow. This Is A Import In The

Answered

Hi I Have A Question: I Have 2 Python Scripts: The First Python Script Is Running The 2. The Imports In The First Script Are Working. But When I Run The Programm On The Gpu I Get For Example The Mistake: No Module Named Tensorflow. This Is A Import In The

Hi
I have a question:
I have 2 python scripts:
the first python script is running the 2.
The imports in the first script are working. But when I run the programm on the gpu I get for example the mistake:
No module named tensorflow. This is a import in the 2. python script.
How can I solve this?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					UnsightlySeagull42
				
					0
					 × 1

Votes Newest

Answers 4

os.system

Yes that's the culprit, it actually runs a new process and clearml assumes that there are no other scripts in the repository that are used, so it does not analyze them
A few options:
Manually add the missing requirement Task.add_requirements('package_name')make sure you call it before the Task.init
2. import the second script from the first script. This will tell clearml to analyze it as well.
3. Force the entire clearml to analyze the whole repository: https://github.com/allegroai/clearml/blob/9b962bae4b1ccc448e1807e1688fe193454c1da1/docs/clearml.conf#L162

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

The scripts are all in the git repo.
But still the same problem.
I use os.system.
Is there a better way to call the other python script?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					UnsightlySeagull42
				
					0
					 × 1

Any chance you actually run the second script with Popen (i.e. calling the python as a subprocess) ?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Hi UnsightlySeagull42
Just making sure, the two scripts are on your git repo ?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Write your answer

604 Views

4 Answers

3 years ago

one year ago

Tags

Similar posts

Hello Everybody. I Have A Question About Autoscaler. I Have An Autoscaler That Is Started By A Python Script. When I Start The Task With Task.Enqueue('Queue') In My Laptop, Where Does The Worker Get The Whole Code? Does It Clone It From Github?

Hi, I Have An Aws Autoscaler Based On The Sample Autoscaler Example Posted In Clearml Git. My Instances Need Random External Ip On Init. Is There A Way To Edit The Yaml Created By The Python Script To Do So?

Hello everyone, *Context:* I am currently facing a headache-inducing issue regarding the integration of flash attention V2 for LLM training. I am running a python script locally, that then runs remotely. Without the integration of flash attention, the co

Hi Everyone, Is It Possible On Clearml, Before Run A Task Using By Agent/Remotely We Can Implement Some Script Python? The Usecase Here I Want To Export Some Env Settings Before Task Started Execute.

Hi Everyone, I Am Working On A Script That Gets Enqueued, And Executes Remotely On Clearml-Agent Workers. I Got A "Hello World" Version Of The Script Running, The Python Env Gets Set Up, And Installs Some Basic Packages I Need From Pypi. However, I Need T

We’Re Running Clearml-Agent On K8S And I First Noticed Some Warnings From The Pod About Python 3.6..?

Hi! I'M Running A Clearml Task Remotely. It Runs A Python Machine Learning Evaluation Script In Repo A. The Script Depends On A Module Which Is Defined In Another One Of My Company'S Repos: Repo B. The File Structure Of The Module In Repo B Is As Follows

Hi, I Am Running A Script From A Git Repository. In The Repository There Is A Package That I Wrote And I Would Like The Script That I Am Running To Be Able To Import It, Thus I Need To Add The Package Path To Python Path. Repo Structure:

Hi All, I Was Trying To Use Clearml-Task To Run A Custom Docker(With Poetry To Install All The Python Dependencies And Activated The Environment) Using Clearml Gpu, But It Seems Like Clearml Always Create A Virtual Environment And Run The Python Script Fr

Hi All! I’M Currently Working On A Project Where I’M Making Use Of Clearml For Hyperparameter Tuning. In My Workflow, I Have A Python Script That I Usually Run With The Following Command: