ClearML FAQ | I Just Deployed Clearml Into K8 Cluster Using Clearml Helm Package. When I Ran A Job, It Gave This Error In The Clearml Web Server (Attached Below). I Sshed Into The Pod Running The Clearml-Agent. Upon Typing Clearml-Agent Init, I Realised The Clearml.Con

Answered

I Just Deployed Clearml Into K8 Cluster Using Clearml Helm Package. When I Ran A Job, It Gave This Error In The Clearml Web Server (Attached Below). I Sshed Into The Pod Running The Clearml-Agent. Upon Typing Clearml-Agent Init, I Realised The Clearml.Con

I just deployed clearml into k8 cluster using clearml helm package. When i ran a job, it gave this error in the clearml web server (Attached below). I sshed into the pod running the clearml-agent. Upon typing clearml-agent init, i realised the clearml.conf is empty and also I am not able to configure it (it is write protected). I am only able to configure the clearml.conf in the agentservices node.
So i am unable to run a clearml-agent within the k8 system, however, things are working if the clearml-agent is outisde the k8 system (my laptop). Do let me know if there is some way to debug this ?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					DeliciousBluewhale87
				
					0
					 × 1

Votes Newest

Answers 34

Yup, tried that.. Same error also

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					DeliciousBluewhale87
				
					0
					 × 1

Did you change anything under the agent's value?
In case you didn't - please try editing the agent.clearmlWebHost and set it to the value of your webserver (use the same one you used for the agent services).
This might solve your issue.

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					FriendlySquid61
				
					0

That's the agent-services one, can you check the agent's one?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					FriendlySquid61
				
					0

Hi AgitatedDove14 , I also fiddled around by changing this line and restarted the deployment. But this just causes it revert back 0.17.2rc4 again.
python3 -m pip install clearml-agent==0.17.2rc3

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					DeliciousBluewhale87
				
					0
					 × 1

Show more results

Write your answer

22K Views

34 Answers

3 years ago

7 months ago

Tags

Similar posts

Hi All Im Trying To Save My Model Checkpoints During Runtime But Am Running Into A Confusing Snag. I'M Using The Huggingface Architecture For A Transformer. Using Their Training Module To Control Training. In The Training Args, I Have The

Hi Team, I Am Trying To Start The Clearml With Opensearch As Backend Instead Elastic Search , When I Started The Service I Am Seeing Partial Web Ui As Below Kindly Help Me In Which Module I Should Look Into It To Get Complete Functionality. No Errors I A

As Soon As I Refactor My Project Into Multiple Folders, Where On Top-Level I Put My Pipeline File, And Keep My Tasks In A Subfolder, The Clearml Agent Seems To Have Problems:

Hey There, I Am Looking Into The Self-Hosted Version Of Clearml For An Isolated Environment That Will Undergo Periodic Data Extraction Back To An Internet-Facing Environment. So Essentially I Want To Have Two Environments Existing And Periodically Update

Hello. I Am Using Hydra As Configuration Manager And I Am Using A Decorator To Specify The File And The Folder It Is Contained In (Typical Hydra Syntax). The Code Now Runs Into This Error That Says, "Primary Config Directory Not Found. Set The Environment

Hi I Am Creating A Custom Image Dataset And Does Anyone Know How To Upload Files Into Remote Uri (S3) Without Compressing The Data?

Hello Everyone, I Am Hosting My Own Clearml Server And It Is Great. However, I Ran Into A Problem Where One Of My Projects Has Become "Hidden" And I Have No Idea How. This Has Created Some Issues Where I Can No Longer Pull From The Project. How Do I Unhid

Hi All, I Am Using A Self-Hosted Clearml Instance. I'M Exploring The Report Feature And Ran Into A Bug. Steps To Reproduce:

Hi Guys, I Have Been Running The Clearml-Serving For A While Now And I Realize That From Time To Time After A Couple Of Hours The Serving Task (Control Plane) That Is Configured Through The Cli Goes Into Status Abort. This Happens Even Though All The Pods

Hi Again, I Am Trying To Execute A Pipeline Remotely, However I Am Running Into A Problem With The Steps That Require A Local Package. Basically I Have A Repo, That I Created Specifically For This Pipeline And I Have Packaged It So That I Can Split It I