Hello, I’M Integrating Clearml Into My Training Code To Track Experiments. I See That Clearml Logs Automatically The Python Env In The “Installed Packages” Section But The List Logged Is Very Different From The

Answered

Hello, I’m integrating ClearML into my training code to track experiments. I see that ClearML logs automatically the python env in the “installed packages” section but the list logged is very different from the conda_env.yml file that I used to build the environment. How is the “installed packages” created?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					RattyPanda61
				
					0
					 × 1

Votes Newest

Answers 4

Hi RattyLouse61 ,
I think packages are detected in runtime and it only shows the packages used by the script directly. When you run with ClearML-Agent, it will log all packages including dependencies that were used.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

RattyLouse61 , I think you can save the yml conda env file as an artifact, this way it would also be accessible by other tasks 🙂

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Thank you CostlyOstrich36 , when I’ll get back to this I’m sure I will have other questions 🙂

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					RattyPanda61
				
					0
					 × 1

Hey CostlyOstrich36 thank you 🙂 I haven’t yet started using the agent to run a task. I’m in the phase of tracking the experiments. The setup that I have is that there’s an yml conda env file (generated via conda env export ) that includes conda and pip packages. I’d like to log the content of that file in ClearML. I’ve tried to use force_requirements_env_freeze but it doesn’t do what I’d hope for. Any suggestion?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					RattyPanda61
				
					0
					 × 1

Write your answer

1K Views

4 Answers

2 years ago

one year ago