ClearML FAQ | Hi Everyone! I Have A Question Regarding The Use Of S3 For Storage. At Our Company, We Only Use Temporary S3 Credentials (Key+Secret+Token) For Security Reasons. That Should Be No Problem On The User Side, As We Can Use The Credential Chain. However, For

Answered

Hi Everyone! I Have A Question Regarding The Use Of S3 For Storage. At Our Company, We Only Use Temporary S3 Credentials (Key+Secret+Token) For Security Reasons. That Should Be No Problem On The User Side, As We Can Use The Credential Chain. However, For

Hi everyone! I have a question regarding the use of s3 for storage. At our company, we only use temporary s3 credentials (key+secret+token) for security reasons. That should be no problem on the user side, as we can use the credential chain. However, for the web UI you can only enter permanent credentials (key+secret). Do you have any suggestions for that? Or do you plan to add support for temporary s3 credentials in the web UI?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					ArrogantFlamingo47
				
					0
					 × 1

Votes Newest

Answers 3

Thanks! Will do

  				
Posted 
	one year ago

					More
				  		
  Report
		
					ArrogantFlamingo47
				
					0
					 × 1

  				
Posted 
	one year ago

					More
				  		
  Report
		
					ArrogantFlamingo47
				
					0
					 × 1

I don't think there is such an option currently but it does make sense. Please open a GitHub feature request for this 🙂

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Write your answer

956 Views

3 Answers

one year ago

one year ago

Tags

Similar posts

Hello! Is It Possible To Use Different Credentials For Different S3 Buckets On Different Hosts? When I Try To Do Like So:

Hi. Can You Point Me In A Direction Of The Solution Please? We Are Trying To Use Spot Machines On Aws Through Autoscaler In Clearml. The Problem Is We Need To Download Each Time Huge Dataset From S3. Is There Some Cash Solution, Like Attaching Disk For Ex

Hi, I Am Trying To Save My Trained Model Weights In S3 Bucket Instead Of Using Clearml Storage When Using Clearml-Task For Ml Training Remotely. I Tried To Use --Skip-Task-Init In Clearml-Task And Set Task.Init In My Scripts, But It Doesn'T Seem To Work.

Hi, I Am Trying To Save My Trained Model Weights In S3 Bucket Instead Of Using Clearml Storage When Using Clearml-Task For Ml Training Remotely. I Tried To Use --Skip-Task-Init In Clearml-Task And Set Task.Init In My Scripts, But It Doesn'T Seem To Work.

Hi, Is There Any Way To Mount An S3 Bucket To Worker Nodes For Some Specific Use Cases Like Data Streaming? If So, How Does Clearml Tackle Data Versioning?

I’M Trying To Use Minio With Clearml As A External Storage. I Am Having Problems With The Configuration File For The Clearml Client When I Use The Output_Uri Parameter Of Task.Init What Do I Put There? I Am Currently Doing Task.Init(… Output_Uri=“S3://I

Hi, Is There A Way To Forbid Uploading Datasets Into Fileserver Or To Make S3 Storage A Default Location For Datasets? As Far As I Know, The Only Way To Upload Dataset To The S3 Storage Is To Use

Is It Possible To Avoid The Clearml-Agent For Local Installations, And Have The File Server Automatically Use An S3 Bucket? I'Ve Found

Hi, Guys! How Do You Keep Your Secrets (User, Password, Host And So On For Data Base, For Example)? I Mean Remote Experiments With Clearml. I Think There Is Two Ways: 1. Use Argparse (Or Hydra) 2. Download Them From S3 As Txt File And Parse It. What Is

Hi Everyone, I Wanna Implement Some Sort Of A Tool That Checks Every N Minutes For A New Data In Db Or Maybe S3 Bucket And Whenever The New Data Appears It Runs Some Preprocessing And Then Starts Training. Is It A Good Idea To Use Clearml Agent Services F