Hi @<1523702000586330112:profile|FierceHamster54> ! This is currently not possible, but I have a workaround in mind. You could use the artifact_serialization_function parameter in your pipeline. The function should return a bytes stream of the zipped content of your data with whichever compression level you have in mind.
If I'm not mistaken, you wouldn't even need to write a deserialization function in your case, because we should be able to unzip your data just fine.
Wdyt?

Hmm, in that case you might need to write it. Doesn’t hurt trying eitherway

@<1552101474769571840:profile|DepravedLion86> You shouldn't need to call wait explicitly. What happens if you don't?

@<1523701304709353472:profile|OddShrimp85> The way to do it is using the packages argument. Maybe clearml couldn't file the requirements file. What does os.path.exists('./requirements.txt') return?

Hi @<1674226153906245632:profile|PreciousCoral74> !

Sadly, Logger.report_matplotlib_figure(…) doesn't seem to log plots. Only the automatic integration appears to behave.

What do you mean by that? report_matplotlib_figure should work. See this example on how to use it: None .
If it still doesn't work for you, could you please share a code snippet that could help us track down...

Hi @<1523715429694967808:profile|ThickCrow29> ! What do you think of this behavior when using pipelines from decorators: suppose we have the following controller:

a = step_1() # step_1 gets aborted/failed
b = step_2(a)
c = step_3() 

In this case, if abort_on_failure is set to False, then step_2 will be skipped.
If the controller uses a , doing something like:

a = step_1() # step_1 gets aborted/failed
print(a)

then an exception will be thrown.
step_3 will run...

great, thanks !

Hi @<1523701240951738368:profile|RoundMosquito25> ! Try using this function None

That would be much appreciated

Hi @<1643060801088524288:profile|HarebrainedOstrich43> ! At the moment, we don't support default arguments that are typed via a class implemented in the same module as the function.
The way pipelines work is: we copy the code of the function steps (eventually their decorator's as well if declared in the same file), then we copy all the imports in the module. Problem is, we don't copy classes.
You could have your enum in a separate file, import it and it should work

Hi @<1597762318140182528:profile|EnchantingPenguin77> ! There is no way to do that as of now

So the flow is like:
MASTER PROCESS -> (optional) calls task.init -> spawns some children CHILD PROCESS -> calls Task.init. The init is deferred even tho it should not be?
If so, we need to fix this for sure

Hi BoredHedgehog47 ! We tried to reproduce this, but failed. What we tried is running the attached main.py which Popen s sub.py .
Can you please run main.py as well and tell us if you still encounter the bug? If not, is there anything else you can think of that could trigger this bug besides creating a subprocess?
Thank you!

no problem. we will soon release a RC that solves both issues

Hi @<1633638724258500608:profile|BitingDeer35> ! Looks like the SDK doesn't currently allow to create steps/controllers with a designated cwd. You will need to call the set_script function on your step's tasks and on the controller for now.
For the controller: If you are using the PipelineDecorator, you can do something like: PipelineDecorator._singleton._task.set_script(working_dir="something") , before you are running the pipeline function. In the case of regular `PipelineControll...

thank Lior

JumpyDragonfly13 The function was auto-generated from the backend schema, so it was auto-included. This function is actually used by the UI when you press the Download full log button, but the SDK can't call it because the response is simply an octet-stream representing the log (the SDK expects a JSON containing some fields).
Anyway, you could've also done something like this to get the task log:
` from clearml import Task
t = Task.get_task(task_id="cecc9d51581d4fe3948b857d21af1ff3")
p...

SmallGiraffe94 You should use dataset_version=2022-09-07 (not version=... ). This should work for your use-case.
Dataset.get shouldn't actually accept a version kwarg, but it does because it accepts some **kwargs used internally. We will make sure to warn the users in case they pass values to **kwargs from now on.
Anyway, this issue still exists, but in another form:
Dataset.get can't get datasets with a non-semantic version, unless the version is sp...

OutrageousSheep60 Regarding number 3. , can you try going to Settings > Configuration and check the Show Hidden Projects option, then try this again? (See the attached images)

Regarding 1. , are you trying to delete the project from the UI? (I can't see an attached image in your message)

Regarding number 2. , that is indeed a bug and we will try to fix it as soon as possible

UnevenDolphin73 can't you find your task/dataset under the Datasets tab?

I don't think the version makes the task disappear. You should still see the task in the Datasets section. Maybe there is something you do with that task/dataset that makes it disappear (even tho it shouldn't)?

Please add it to github! No other info is needed, we know what the issue is

OutrageousSheep60 that is correct, each dataset is in a different subproject. That is why bug 2. happens as well

Hi @<1523715429694967808:profile|ThickCrow29> ! We identified the issue. We will soon release a fix for it

@<1523701083040387072:profile|UnevenDolphin73> are you composing the code you want to execute remotely by copy pasting it from various cells in one standalone cell?

UnevenDolphin73 The task shouldn't disappear when using use_current_task=False . There might be something else that makes it disappear.

Hi @<1676038099831885824:profile|BlushingCrocodile88> ! We will soon try to merge a PR submitted via Github that will allow you to specify a list of files to be added to the dataset. So you will then by able to do something like add_files(glob.glob(*) - glob.glob(*.ipynb))

Actually, datasets should have an automatic preview...