@<1539417873305309184:profile|DangerousMole43> , I think for this specific ability you would need to re-write your pipeline code with pipelines from decorators

The syntax should be something like from <PACKAGE> import data

RoughTiger69 thanks for the input 🙂

Which file are you referring to? Can you link it?

MuddySquid7 , we're having a look and testing it. Thanks!

SwankySeaurchin41 , I think you don't need to connect pipelines. Think of pipeline as a DAG execution. You can build it anyway you want 🙂

OddShrimp85 , Hi 🙂

I'm afraid that the only way to load contents of setup A into setup B is to perform a data merge.

This process basically requires merging the databases (mongodb, elasticsearch, files etc.). I think it's something that can be done in the paid version as a service but not in the open one.

Hi MistakenDragonfly51 , regarding your questions:
ClearML has a model repository built in. You can load an input model using InputModel module ( https://clear.ml/docs/latest/docs/references/sdk/model_inputmodel ). Also, you can fetch the models of an experiment using Task.get_models() - https://clear.ml/docs/latest/docs/references/sdk/task#get_models Can you elaborate on how this config looks in the UI when you view it?

Can you please provide stand alone code snippets that reproduce this behavior?

Try it as the first option after clearml-agent: clearml-agent --debug daemon --docker --foreground

Hi ThankfulHedgehong21 ,

What versions of ClearML & ClearML-Agent are you using?
Also, can you provide a small code snippet to play with?

I think this is what you're looking for, tell me if it helps 🙂

@<1556812486840160256:profile|SuccessfulRaven86> , to make things easier to debug, can you try running the agent locally?

You can delete locally but it should not affect the remote data.

The data itself is stored in the fileserver. Whatever you do locally does not affect the remote storage, only when creating a new version the changes should be stored (Like when using 'clearml-data sync').

And easier to manage without the need for such 'hacks' 😛

Hopefully will have updates soon

RattyLouse61 , from what I can find - Yes. I've seen some examples shared on the community as well. Is there something specific you're looking for?

Is there any chance you're filtering something and that's why you can't see it?

Hi @<1523702932069945344:profile|CheerfulGorilla72> , making sure I understand - You basically want to select an input model via the UI?

What do you mean read params file?

ExcitedSeaurchin87 , Hi 🙂

I think it's correct behavior - You wouldn't want leftover files flooding your computer.

Regarding preserving the datasets - I'm guessing that you're doing the pre-processing & training in the same task so if the training fails you don't want to re-download the data?

Hi ShallowGoldfish8 , what versions of ClearML & ClearML-Agent are you using?

Hi 🙂

Are you asking if you can share experiments between a self hosted server and http://app.clear.ml ?

In the UI check under the execution tab in the experiment view then scroll to the bottom - You will have a field called "OUTPUT" what is in there? Select an experiment that is giving you trouble?

Oh I see. Technically speaking the pipeline controller is a task of a special type of itself. So technically speaking you could provide the task ID of the controller and clone that. You would need to make sure that the relevant system tags are also applied so it would show up properly as a pipeline in the webUI.

In addition to that, you can also trigger it using the API also

What's the use case?

I can see that the old Allegro AI Trains server is not longer available:

What do you mean? You mean the AMI?
Regarding AWS deployment - I guess it really depends on your usage. Are you interested in holding the server on an EC2 instance?

I'll clarify - on the server you have two parts - the clearml folders where all the mongo/elastic/redis data sits and you have the dockers. So, downgrading would mean using previous version dockers. However if you don't have a backup of your data I don't suggest you do this since data might become corrupt (mismatching elastic versions is bad for elastic)

Any chance you have a toy script to play with?

Hi BoredHedgehog47 , You need to add
from clearml import Task task = Task.init(project_name='examples', task_name='hello world')to your code and run it once after you've ran clearml-init