Unanswered
Hi.
I Have A Job That Processes Images And Creates ~5 Gb Of Processed Image Files (Lots Of Small Ones).
At The End - It Creates A
That job was using clearml 1.8.3 so I take it that setting max_workers to 1 would not make a difference?
Looking at the docs:
https://clear.ml/docs/latest/docs/references/sdk/dataset/#upload
they say that max_workers = number of cores but looking at the log it does seem like it's doing one chunk every 5 minutes (long time for 500mb upload for a node running in gcp...)
102 Views
0
Answers
one year ago
one year ago