Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone, I Have A Quick Question: When I Use

Hi everyone,
I have a quick question:
When I use api.files_server : s3://my_minio_instance:9000 in the clearml.conf without a bucket, I get the errors Failed creating storage object s3://my_minio_instance:9000 Reason: Missing key and secret for S3 storage access ( s3://my_minio_instance:9000 ) . When I use s3://my_minio_instance:9000/clearml with a bucket, it works. However, I also define multiple buckets in sdk.aws.s3 in clearml.conf. It seems like it does not matter, what bucket I put as a fileserver, all the buckets will work, because I put all the credentials in sdk.aws.s3 . Is there some reason I have to enter any bucket for files_server or is this a bug?

  
  
Posted 2 years ago
Votes Newest

Answers 11


Hi ReassuredTiger98 , I'm not sure I understand.
You're asking why you need to put in credentials to use minio as a files_server or why you got the error directly on port 9000 but not on s://minio_instance:9000/ <MY_BUCKET_NAME> ?

  
  
Posted 2 years ago

Latter

  
  
Posted 2 years ago

I can put anything there: s3://my_minio_instance:9000 /bucket_that_does_not_exist and it will work.

  
  
Posted 2 years ago

Just if I leave it empty, it will not work.

  
  
Posted 2 years ago

And you get the error only when there are no credentials or is this unconnected?

  
  
Posted 2 years ago

Yes. What do you mean by unconnected?

  
  
Posted 2 years ago

Unrelated 😛

  
  
Posted 2 years ago

I have set default_output_uri to s3://my_minio_instance:9000/clearml

If I set files_server to s3://my_minio_instance:9000 /bucket_that_does_not_exist it fails at uploading metrics, but model upload still works:

WARNING - Failed uploading to s3://my_minio_instance:9000/ bucket_that_does_not_exist ('NoneType' object has no attribute 'upload')

clearml.Task - INFO - Completed model upload to s3://my_minio_instance:9000/clearml

What is default_output_uri for? Only artifact upload?

  
  
Posted 2 years ago

default_output_uri is for artifacts & models while files_server is for debug samples and plots (if they are files)

  
  
Posted 2 years ago

And which is for metrics?

  
  
Posted 2 years ago

These are the errors I get if I use file_servers without a bucket ( s3://my_minio_instance:9000 )

2022-11-16 17:13:28,852 - clearml.storage - ERROR - Failed creating storage object Reason: Missing key and secret for S3 storage access ( ) 2022-11-16 17:13:28,853 - clearml.metrics - WARNING - Failed uploading to ('NoneType' object has no attribute 'upload_from_stream') 2022-11-16 17:13:28,854 - clearml.storage - ERROR - Failed creating storage object Reason: Missing key and secret for S3 storage access ( )But for the same run model upload works:
2022-11-16 17:13:28,928 - clearml.storage - INFO - Starting upload: /tmp/.clearml.upload_model_m33vzdf7.tmp => my_minio_instance:9000/clearml/mymodel.pth 2022-11-16 17:13:28,974 - clearml.Task - INFO - Completed model upload to

  
  
Posted 2 years ago