Hi @<1668427950573228032:profile|TeenyShells80> , the parent_datasets should be a list of dataset IDs or clearml.Dataset objects, not dataset names. Maybe that is the issue

Yes, see minio instructions under this: None

have you tried copying the certificate to /usr/local/share/ca-certificates/ ?

Can you share a snippet?

great, thanks !

Can you please screenshot the INFO tab on the pipeline controller task?

Hi @<1545216070686609408:profile|EnthusiasticCow4> ! This is actually very weird. Does your pipeline fail when running the first step? What if you run the pipeline via "raw" python (i.e. by doing python3 your_script.py )?

@<1523701304709353472:profile|OddShrimp85> The way to do it is using the packages argument. Maybe clearml couldn't file the requirements file. What does os.path.exists('./requirements.txt') return?

@<1523701304709353472:profile|OddShrimp85> I believe you need to set the repo argument to point to your repository

@<1675675705284759552:profile|NonsensicalAnt77> have you tried setting secure: true and host: storage.yandexcloud.net:443 ?

just append it to None : None in Task.init

Hi SmallGiraffe94 ! Dataset.squash doesn't set as parents the ids you specify in dataset_ids . Also, notice that the current behaviour of squash is pulling the files from all the datasetes from a temp folder and re-uploading them. How about creating a new dataset with id1, id2, id3 as parents Dataset.create(..., parent_datasets=[id1, id2, id3]) instead? Would this fit your usecase?

thank Lior

Please add it to github! No other info is needed, we know what the issue is

OutrageousSheep60 1.8.4rc1 is out. Can you please try it? pip install -U clearml==1.8.4rc1

Hi @<1675675705284759552:profile|NonsensicalAnt77> ! How are you uploading the model weights without using the SDK? Can you please share a code snippet (might be useful in finding why your config doesn't work). Also, what is your clearml version?

pruning old ancestors sounds like the right move for now.

Taking a look

Hi @<1524560082761682944:profile|MammothParrot39> ! A few thoughts:
You likely know this, but the files may be downloaded to something like /home/user/.clearml/cache/storage_manager/datasets/ds_e0833955ded140a69b4c9c9d8e84986c . .clearml may be hidden and if you are using an explorer you are not able to see the directory.

If that is not the issue: are you able to download some other datasets, such as our example one: UrbanSounds example ? I'm wondering if the problem only happens fo...

Hi @<1643060801088524288:profile|HarebrainedOstrich43> ! The rc is now out and installable via pip install clearml==1.14.1rc0

PanickyMoth78 there is no env var for sdk.google.storage.pool_connections/pool_maxsize . We will likely add these env vars in a future release.
Yes, setting max_workers to 1 would not make a difference. The docs look a bit off, but it is specified that 1: if the upload destination is a cloud provider ('s3', 'gs', 'azure') .
I'm thinking now that the memory issue might also be cause because of the fact that we prepare the zips in the background. Maybe a higher max_workers wou...

@<1523703472304689152:profile|UpsetTurkey67> great, thank you! We are taking a look

Hi @<1523701523954012160:profile|ShallowCormorant89> ! This is not really supported, but you could use continue_on_fail to make sure you get to your last step: None

Hi @<1691620883078057984:profile|ConfusedSeaanemone5> ! Those are the only 3 charts that the HPO constructs and reports. You could construct other charts/plots yourself and report them when a job completes using the job_completed_callback parameter.

you could also try using gloo as the backend (it uses CPU) just to check that the subprocesses spawn properly

(We will deprecate continue_on_fail)

If the task is running remotely and the parameters are populated, then the local run parameters will not be used, instead the parameters that are already on the task will be used. This is because we want to allow users to change these parameters in the UI if they want to - so the paramters that are in the code are ignored in the favor of the ones in the UI

Hi BoredHedgehog47 ! We tried to reproduce this, but failed. What we tried is running the attached main.py which Popen s sub.py .
Can you please run main.py as well and tell us if you still encounter the bug? If not, is there anything else you can think of that could trigger this bug besides creating a subprocess?
Thank you!

Hi @<1694157594333024256:profile|DisturbedParrot38> ! We weren't able to reproduce, but you could find the source of the warning by appending the following code at the top of your script:

import traceback
import warnings
import sys

def warn_with_traceback(message, category, filename, lineno, file=None, line=None):
    log = file if hasattr(file,'write') else sys.stderr
    traceback.print_stack(file=log)
    log.write(warnings.formatwarning(message, category, filename, lineno, line))
...

does it work running this without clearml? @<1578555761724755968:profile|GrievingKoala83>