Sorry about that 😅

Hi @<1523701240951738368:profile|RoundMosquito25> ! Yes, you should be able to do that

Hi @<1523701523954012160:profile|ShallowCormorant89> ! This is not really supported, but you could use continue_on_fail to make sure you get to your last step: None

Hi @<1523705721235968000:profile|GrittyStarfish67> ! This looks like a boto3 error. You could try lowering sdk.aws.s3.boto3.max_multipart_concurrency in clearml.conf and setting max_workers=1 when calling Dataset.get_local_copy

Hi @<1539417873305309184:profile|DangerousMole43> ! You need to mark the task you want to upload an artifact to as running. You can use task.mark_started(force=True) to do so, then mark it back as completed using task.mark_completed(force=True)

Hi @<1676400486225285120:profile|GracefulSquid84> ! Each step is indeed a clearml task. You could try using the step ID. Just make sure you pass the ID to the HPO step (you can do that by simply returning the Task.current_task().id

Hi SteadySeagull18 ! The docs are correct.
How do you run the pipeline controller? Is it remotely, locally, locally with an agent? If you run os.path.exists(model.url[len("file://"):]) does it return True ?
Can you provide a minimal example that could help us reproduce the issue?
Thank you

One more question FierceHamster54 : what Python/OS/clearml version are you using?

FierceHamster54 I understand. I'm not sure why this happens then 😕 . We will need to investigate this properly. Thank you for reporting this and sorry for the time wasted training your model.

Hi @<1639799308809146368:profile|TritePigeon86> ! Please see continue_behaviour . You should be able to pass the parameter to your parent step. It is not documented yet, but it should be available in the latest version of clearml. See this for some documentation: None

(We will deprecate continue_on_fail)

Hi RoundMosquito25 ! What clearml version are you using? Do you get any error messages when you are setting floats instead of strings?

Hi DeliciousKoala34 . I was able to reproduce your issue. I'm now looking for a solution for your problem. Thank you

Hi @<1544853695869489152:profile|NonchalantOx99> ! In your clearml.conf , try to set, at the end of the whole file, outside any curly brackets, agent.package_manager.pip_version: "23.1.2"

Perfect! Can you please provide the sizes of the files of the other 2 chunks as well?

Hi FierceHamster54 ! Did you call Task.init() in train.py ?

thank Lior

@<1545216070686609408:profile|EnthusiasticCow4> a PR would be greatly appreciated. If the problem lies in _query_tasks then it should be addressed there

Hi @<1610083503607648256:profile|DiminutiveToad80> ! You need to somehow serialize the object. Note that we try different serialization methods and default to pickle if none work. If pickle doesn't work then the artifact can't be uploaded by default. But there is a way around it: you can serialize the object yourself. The recommended way to do this is using the serialization_function argument in upload_artifact . You could try using something like dill which can serialize more ob...

great, glad you found a work-around

You're correct. There are 2 main entries in the conf file: api and sdk . The dataset entry should be under sdk

@<1657556312684236800:profile|ManiacalSeaturtle63> what clearml SDK version are you using? I believe there was a bug related to pipelines not showing in the UI, but that was fixed in clearml==1.14.1

Hi @<1570220858075516928:profile|SlipperySheep79> ! What happens if you do this:

import yaml
import argparse
from my_pipeline.pipeline import run_pipeline
from clearml import Task

parser = argparse.ArgumentParser()
parser.add_argument('--config', type=str, required=True)

if __name__ == '__main__':
    if not Task.current_task():
      args = parser.parse_args()
      with open(args.config) as f:
          config = yaml.load(f, yaml.FullLoader)
    run_pipeline(config)

@<1523701168822292480:profile|ExuberantBat52> Do you have pandas installed on your machine?

Actually, datasets should have an automatic preview...

Hi @<1523701132025663488:profile|SlimyElephant79> ! Looks like this is a bug on our part. We will fix this as soon as possible

Hi @<1533620191232004096:profile|NuttyLobster9> ! PipelineDecorator.get_current_pipeline will return a PipelineDecorator instance (which inherits from PipelineController ) once the pipeline function has been called. So

pipeline = PipelineDecorator.get_current_pipeline()
pipeline(*args)

doesn't really make sense. You should likely call pipeline = build_pipeline(*args) instead

HandsomeGiraffe70 your conf file should look something like this:
` {
# ClearML - default SDK configuration

storage {
    cache {
        # Defaults to system temp folder / cache
        default_base_dir: "~/.clearml/cache"
        # default_cache_manager_size: 100
    }

    direct_access: [
        # Objects matching are considered to be available for direct access, i.e. they will not be downloaded
        # or cached, and any download request will ...

Hi JumpyDragonfly13 ! Try using get_task_log instead of download_task_log

@<1590514584836378624:profile|AmiableSeaturtle81> weren't you using https for the s3 host? maybe the issue has something to do with that?