Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi Everyone! I Built A Task That Processes Data And In The End Generates A Large Folder Containing Images (Aprox 10Gb). I Would Like To Be Able To Upload This Folder As An Artifact But When I Try To Do This With Task.Upload_Artifact I Get This Error. Is T

Hi everyone!
I built a task that processes data and in the end generates a large folder containing images (aprox 10Gb). I would like to be able to upload this folder as an artifact but when I try to do this with task.upload_artifact I get this error. Is there a size limit for artifacts? If so, is it possible to somehow agregate multiple artifacts into a single file? Thank you in advance!

Posted one year ago
Votes Newest

Answers 4

HiĀ  SmoggyGoat53
There is a storage limit on the file server (basically 2GB per file limit), thisĀ  is the cause of the error.
You can upload the 10GB to any S3 alike solution (or a shared folder). Just set the "output_uri" on the Task (either at Task.init or with Task.output_uri = " s3://bucket ")

Posted one year ago

Also how are you uploading? Because if you don't zip the folder and upload with
task.upload_artifact('local folder', artifact_object=os.path('<PATH_TO_FOLDER>'))
This should work

Posted one year ago

Also to get past the problem via hack you can split the file via several zips through your code before uploading

Posted one year ago

Hmmm interesting. According to bytes it looks like 2GB. What type is the file?

Posted one year ago
4 Answers
one year ago
11 months ago