Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Using Clearml Python Package On Sagemaker Notebook Instance. If I Want To Edit The Clearml.Conf On The Notebook Instance, How Could I Do That?

Hi, I'm using ClearML python package on Sagemaker notebook instance. If I want to edit the clearml.conf on the notebook instance, how could I do that?

  
  
Posted 3 years ago
Votes Newest

Answers 17


Hi SquareFish25 ,

I tried the follow and succeed to upload the file:

` import os

os.environ['AWS_ACCESS_KEY_ID'] = "***"
os.environ['AWS_SECRET_ACCESS_KEY'] = "***"
os.environ['AWS_DEFAULT_REGION'] = "***"

from clearml import StorageManager
remote_file = StorageManager.upload_file(<file to upload>, 's3://bucket_name/inner_folder/file_name') `Can you try it and update if it works for you?

  
  
Posted 3 years ago

Hi TimelyPenguin76 , Thank you for your reply, I've tried env vars. The problem happened when I tried to upload the files to S3 using ClearML. os.environ['AWS_ACCESS_KEY_ID'] = "***"
os.environ['AWS_SECRET_ACCESS_KEY'] ="***"
os.environ['AWS_SESSION_TOKEN'] = "***"

  
  
Posted 3 years ago

Hi SquareFish25 , what about AWS_DEFAULT_REGION , did you add it too? Can you try with it if not?
What version of ClearML are you using?

  
  
Posted 3 years ago

We actually want to set the output_uri in Task.init to be a folder in AWS S3. And run the script on sagemaker.

  
  
Posted 3 years ago

Hi TimelyPenguin76 , I added the AWS_DEFAULT_REGION as well, and it did not work. The version of ClearML is '0.17.4'.

  
  
Posted 3 years ago

SquareFish25 Will try to reproduce it

  
  
Posted 3 years ago

SquareFish25 Still on it 🤞

  
  
Posted 3 years ago

Hey TimelyPenguin76 Any luck?

  
  
Posted 3 years ago

Hi TimelyPenguin76 , I tried and got the same error as before. One thing was different from yours, our company is using SSO to access AWS, so I added os.environ['AWS_SESSION_TOKEN'] = "***". It worked for boto3 to upload the file to S3, so I think probably SSO is not the reason.

  
  
Posted 3 years ago

Yes, "from clearml import StorageManager"

  
  
Posted 3 years ago

SquareFish25 do you have a way trying with access and secret without token? just for the checking

  
  
Posted 3 years ago

TimelyPenguin76 Unfortunately, no. Is the error because of token?

  
  
Posted 3 years ago

Hi TimelyPenguin76 , any update?

  
  
Posted 3 years ago

I suspect that

I will try to generate a new token for myself and reproduce it with it

  
  
Posted 3 years ago

I will check the aws token, just to verify, you imported the StorageManager after the os.environ calls?

  
  
Posted 3 years ago

Hi SquareFish25 ,

Which section would you like to modify? Can env vars do the trick? e.g.

os.environ["CLEARML_API_HOST"] = "***" os.environ["CLEARML_WEB_HOST"] = "***" os.environ["CLEARML_FILES_HOST"] = "***" os.environ["CLEARML_API_ACCESS_KEY"] = "***" os.environ["CLEARML_API_SECRET_KEY"] = "***"

  
  
Posted 3 years ago

I used boto3 to upload and it worked means the key should be right. But it did not work when using clearml.

  
  
Posted 3 years ago
521 Views
17 Answers
3 years ago
one year ago
Tags