Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, We Ran Into A Bit Of A Problem Storing Artifacts In S3. In Python 3.10 When The Files Are Large Enough For Multipart Uploading To Kick In, Something Goes Wrong In Boto3 With Threading Causing The Upload To Fail. (Exact Error Being

Hi, we ran into a bit of a problem storing artifacts in S3. In Python 3.10 when the files are large enough for multipart uploading to kick in, something goes wrong in boto3 with threading causing the upload to fail. (exact error being RuntimeError: cannot schedule new futures after interpreter shutdown in https://github.com/allegroai/clearml/blob/master/clearml/storage/helper.py#L1452 ) Temporary workaround for this would be to disable threading by setting multipart to false. This doesn't seem to work because for AWS S3 when the config is loaded in https://github.com/allegroai/clearml/blob/master/clearml/storage/helper.py#L334 and https://github.com/allegroai/clearml/blob/master/clearml/backend_config/bucket_config.py#L180 multipart is set to true by default and not loaded from the config as far as I can tell. Is this intended behavior and I'm configuring stuff wrong or is this a bug?

  
  
Posted 2 years ago
Votes Newest

Answers 3


The error originates from boto3. This seems to be a https://github.com/boto/boto3/issues/3113 with one possible solution being disabling threading. The problem does no occur on Python 3.8.10.

  
  
Posted 2 years ago

Is this error originating from boto3?

  
  
Posted 2 years ago

Does this happen on different python versions?

  
  
Posted 2 years ago
1K Views
3 Answers
2 years ago
one year ago
Tags