I'M Getting Upload Errors When Uploading Artifacts (Using Gcs) But It Uploads For Datasets Works Just Fine.

Answered

I'm getting upload errors when uploading artifacts (using GCS) but it uploads for datasets works just fine.

clearml.storage - ERROR - Failed uploading: HTTPSConnectionPool(host=' storage.googleapis.com ', port=443): Max retries exceeded with url: /upload/storage/v1/b/files-xyz/o?uploadType=multipart (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2423)')))

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Votes Newest

Answers 6

Version is 1.11.1

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Both uploads were from the same machine and same Task()

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Hi @<1575656665519230976:profile|SkinnyBat30> , what version of ClearML are you using? Are you uploading datasets from the same machine also to GCS?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Also have clearml-agent 1.5.2, clearml-serving 1.3.0 and clearml-session 0.4.0

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

I can reproduce the problem in a notebook and post it on Github if that helps.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

wait_on_upload=True seems to have fixed it...

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SkinnyBat30
				
					0
					 × 1

Write your answer

883 Views

6 Answers

one year ago