ClearML FAQ | Hi, We Are Having Some Issues With Model Snapshots Uploading To The Fileserver. We Configured Sdk.Development.Default_Output_Uri To Point To Our File Server, And When We Run Some Experiment We Can See Under The Models Tab Some Url Pointing To

Answered

Hi, We Are Having Some Issues With Model Snapshots Uploading To The Fileserver. We Configured Sdk.Development.Default_Output_Uri To Point To Our File Server, And When We Run Some Experiment We Can See Under The Models Tab Some Url Pointing To

Hi, we are having some issues with model snapshots uploading to the fileserver. We configured sdk.development.default_output_uri to point to our file server, and when we run some experiment we can see under the models tab some url pointing to http://192.168.1.240:8081/uploading_file but the file is not accessible and its not clear if it was successfully stored. any clues?

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					RipeGoose2
				
					0
					 × 1

Votes Newest

Answers 6

AgitatedDove14 it is happening on an offline network, would be tricky to set it up we will try. so far the errors we observed were either:
Calling upload callback when starting upload: maximum recursion depth exceeded
Or
something like pending for upload (might be because we archived a run while it was uploading)

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					RipeGoose2
				
					0
					 × 1

it worked!

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					RipeGoose2
				
					0
					 × 1

Hi RipeGoose2
Any logs on the console ?
Could you test with a dummy example on the demoserver ?

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

RipeGoose2 That sounds familiar. Could you test with the latest RC?
pip install trains==0.16.4rc0

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

YEY 🙂

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

thanks!

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					RipeGoose2
				
					0
					 × 1

Write your answer

2K Views

6 Answers

4 years ago

2 years ago

Tags

Similar posts

Hi Everyone, We Are Using Clearml Kubernetes For Some Time Now With Great Success. However, Stability On Our Cluster Was Lacking So We Decided To Move To A Dedicated Azure Vm. How Should One Transfer The Data Of The Service To A New Location?

Hello Everyone, I'M Using Quite An Old Clearml : Webapp: 1.14.0-431 • Server: 1.14.0-431 • Api: 2.28 I Would Like To Upgrade It Because We Have Some Issues With It. The Last Time I Upgrade It There Was An Issue With Upgrading The Mongodb.. The Current Mon

Hi Everyone! I Would Need Your Opinion About How To Proceed With Offline Mode When Serving Models. We Currently Have Some Serving Endpoints Which Are Constantly Running And We Report Images And Statistics. How Do You Recommend Dealing With Offline Mode In

Hi All, We Have Clearml-Server Running On A Kube Pod, And Then A Gpu Server Running The Clearml-Agent Which We Use To Queue Jobs. For Some Reason, Our Kube Pod Restarted (We'Re Looking Into Why), But In The Process Of This Happening All Jobs On The Worke

Hi There, Our. Self-Hosted Server Is Periodically Very Slow To React In The Web Ui. We'Ve Been Debugging For Quite Some Time, And It Would Seem That Elastisearch Might Be The Culprit. Looking At The Elastisearch Index, We Have An Index Of Around 80G Of Tr

Hi, I Am Fairly New To Clearml And Would Appreciate Some Help With This Problem. My Team Currently Uses The Virtual Environment Mode To Run The Experiments. But We Also Have A Docker File That We Use For Local Debug So I Want To Consolidate Everything And

Could I Get Some Feedback From People With Experience Using Clearml Pipelines On The Best Way To Handle Caching? My Team Is Working On Configuring Clearml Pipelines For Our Data Processing Workflow. We Currently Have An Experimental Pipeline Configured F

Hey Is There A Way For One To Extend Clearml Somehow? We Have Some Custom Evaluations We Want To Do And Our Ideal Scenario Would Be To Do Them Within Clearml Itself.

Hello Everyone , I Am New Bee To Clearml And Finding Option To Accommodate Opensearch Since We Have Already Opensearch Running In Our Env, Is Opensearch Supported In Clearml Instead Of Elasticsearch ? Please Shed Some Light On That

Hey! We'Ve Set An Hyperparameter Optimization Task On One Of Our Experiments, And It Seems As Though The Optimization Itself Is Running - Yet We Don'T See The Iterations Table In The 'Plots' Section In The Ui. Providing Some Screenshots To How We'Ve Set I