ClearML FAQ | Hello, I’M Integrating Clearml Into My Training Code To Track Experiments. I See That Clearml Logs Automatically The Python Env In The “Installed Packages” Section But The List Logged Is Very Different From The

Answered

Hello, I’M Integrating Clearml Into My Training Code To Track Experiments. I See That Clearml Logs Automatically The Python Env In The “Installed Packages” Section But The List Logged Is Very Different From The

Hello, I’m integrating ClearML into my training code to track experiments. I see that ClearML logs automatically the python env in the “installed packages” section but the list logged is very different from the conda_env.yml file that I used to build the environment. How is the “installed packages” created?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					RattyPanda61
				
					0
					 × 1

Votes Newest

Answers 4

Hey CostlyOstrich36 thank you 🙂 I haven’t yet started using the agent to run a task. I’m in the phase of tracking the experiments. The setup that I have is that there’s an yml conda env file (generated via conda env export ) that includes conda and pip packages. I’d like to log the content of that file in ClearML. I’ve tried to use force_requirements_env_freeze but it doesn’t do what I’d hope for. Any suggestion?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					RattyPanda61
				
					0
					 × 1

Hi RattyLouse61 ,
I think packages are detected in runtime and it only shows the packages used by the script directly. When you run with ClearML-Agent, it will log all packages including dependencies that were used.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

RattyLouse61 , I think you can save the yml conda env file as an artifact, this way it would also be accessible by other tasks 🙂

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Thank you CostlyOstrich36 , when I’ll get back to this I’m sure I will have other questions 🙂

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					RattyPanda61
				
					0
					 × 1

Write your answer

665 Views

4 Answers

2 years ago

one year ago

Tags

Similar posts

Hello!, Does Anyone Know Why When Running A Task In A Specific Docker Image, There Are Some Python Packages Whose Versions Doesn'T Match? For Example, I Have Installed The Package Ultralytics On 8.1.25 On The Docker Image But It'S On 8.1.11 In The Install

After Running My Pipeline On A Gcp Vm, I Am Getting The Following Error When Installing The Packages Successfully Installed Numpy-1.24.1 Torch Cuda 110 Index Page Found, Adding

Hi All, I'M A New User With Clearml-Agent. I Know It'S Supposed To Automatically Replicate The Environment Of A Task, Based On Installed Packages List. However, Installed Packages Of My Task Is Misses Many Of Installed Packages (Any Idea Why?) How Do I Co

Hi, When Running A Task To Training On The Clearml Agent It Installs Packages Not With The Correct Version That My Requirments.Txt Has, I'Ve Installed A Environment Manually For The Requirements.Txt And It Installs The Correct Versions However On The Agen

Hey Guys! We Run Tasks Using Docker Mode Environment. The Docker Image We'Re Using Has Already All The Packages Installed On The System, But I Notice That Every-Time We Run A Task, It Tries To Install The Environment Packages Over Again. Is There A Way To

One Question.. If I Had A Docker Image In The Docker Registry That Had All The Requirements Of A Task Already Installed, Would That Let The Clearml-Agents Skip The Part Of Installing Requirements For A Task? Such As Extra Python Packages

Hi All, Where Does The Installed Packages List Populate From In The Task Viewer?

Hi Everyone, Is There A Way To Avoid The Environment Setup When Running A Task Using A Worker? I Am Currently Using A Custom Docker Image That Already Has All The Require Packages Installed. I Tried Setting The Env Var

Brand New User Here. I’M Trying To Run An Optimization Task. The Tasks Resulting From The Optimization All Fail Because A Necessary Package Is Not Installed On Them. I Checked The Template Task And The List Of “Installed Packages” Indeed Does Not Have One

After Resetting And Enqueueing A Task, The Pip Installation Procedure In A Remote Agent Tries To Install Packages Different From Packages That Were Installed In The Execution Session Before The Resetting And Enquiring Of The Task. How Can I Control That