Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello! Is It Possible To Use S3 Backblaze With Clearml? Can I Use The Aws Section For That Settings? Or Another Section In Clearml.Conf Corresponding To Backblaze Would Work? Or There Is No Way To Use Backblaze In This Case At All?

Hello! Is it possible to use S3 Backblaze with clearml? Can I use the aws section for that settings? or another section in clearml.conf corresponding to backblaze would work? Or there is no way to use backblaze in this case at all?

  
  
Posted 2 months ago
Votes Newest

Answers 12


For digitalocean:
host: "(region). digitaloceanspaces.com:443 "
bucket: “(bucket name)”
key: “(key)”
secret: “(secret)”
multipart: false
secure: true
(verify commented out entirely)

So for you - make sure to add your creds that have the right scope (r/w), and try specifying the bucket .

Then in clearml tasks themselves you tell the task using output_uri=“s3://(region).digitaloceanspaces.com:443/clearml/”

(I import this as a constant from a _constants.py file across tasks)

This exact specific combination is what ive been using without an issue, but it took hours of guessing to get to it

  
  
Posted one month ago

i ran into this recently.
its a small thing but double check the port. should be 443, not 433 as in the docs (typo?) - seems you got this in the screenshot .
no region should be set .

i dont use backblaze but if it helps i can show my digitalocean spaces config . should be comparable .

  
  
Posted 2 months ago

Hi SmallTurkey79 could you please share how you did that? Here is how I tried, with 443 and no region, but still not working. The endpoint variable in clearml/storage/helper.py always returns None.
Thanks!
image

  
  
Posted one month ago

Try removing the region, it might be confusing it

  
  
Posted 2 months ago

Yes, i can get access with the boto3 client

  
  
Posted 2 months ago

Just to make sure, does Backblaze support the boto3 SDK?

  
  
Posted 2 months ago

Great, SmallTurkey79 I will check that and let you know! Appreciate that!

  
  
Posted 2 months ago

Hi CostlyOstrich36 I tried, but got
2025-01-15 16:35:13,846 - clearml.storage - ERROR - Failed uploading: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.

and I saw that endpoint_url is None. Indeed cfg.host is returning None.
I tried to hardcode the endpoint_url and got
2025-01-15 16:37:01,713 - clearml.storage - ERROR - Failed uploading: An error occurred (405) when calling the PutObject operation: Method Not Allowed
ValueError: Insufficient permissions (delete failed) for None

But I am still not sure if it is the about permissions because it was appearing the same ValueError before.

  
  
Posted 2 months ago

S3BucketConfig(bucket='mkrs-data', subdir='', host='',...)
in all cases i get that host=''.

  
  
Posted 2 months ago

Hi CostlyOstrich36 thanks! Yes, i did that like attached. I tried host: s3.us-west-004.backblazeb2.com , backblazeb2.com , with port :9000, :433.
and the log error is:
2025-01-15 13:01:58,264 - clearml.storage - ERROR - Failed uploading: Could not connect to the endpoint URL: " None "

it is trying to sue amazonaws still. Any hints on that?
Thanks!
image

  
  
Posted 2 months ago

Thanks SmallTurkey79 ! That actually solved my issue indeed!! Thanks!

  
  
Posted one month ago

Hi UpsetPanda50 , none AWS s3 solutions are also supported. Please see docs - None

  
  
Posted 2 months ago