I'M Getting Upload Errors When Uploading Artifacts (Using Gcs) But It Uploads For Datasets Works Just Fine.

Answered

I'm getting upload errors when uploading artifacts (using GCS) but it uploads for datasets works just fine.

clearml.storage - ERROR - Failed uploading: HTTPSConnectionPool(host=' storage.googleapis.com ', port=443): Max retries exceeded with url: /upload/storage/v1/b/files-xyz/o?uploadType=multipart (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Votes Newest

Answers 6

Version is 1.11.1

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Also have clearml-agent 1.5.2, clearml-serving 1.3.0 and clearml-session 0.4.0

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

wait_on_upload=True seems to have fixed it...

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Both uploads were from the same machine and same Task()

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

I can reproduce the problem in a notebook and post it on Github if that helps.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Hi @<1575656665519230976:profile|SkinnyBat30> , what version of ClearML are you using? Are you uploading datasets from the same machine also to GCS?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Write your answer

1K Views

6 Answers

2 years ago