I never installed trains on this environment

If the credentials don't have access tothe autoscale service obviously it won't work

Cool - so that means the fileserver which comes with the host will stay emtpy? Or is there anything else being stored there?

the ones in the export are like you said

I'm using pipe.start_locally so I imagine I don't have to .wait() right?

and then how would I register the final artifact to the pipelien? AgitatedDove14 ⬆

So could you re-explain assuming my piepline object is created by pipeline = PipelineController(...) ?

And once this is done, what is the file server IP good for? will it redirect to the bucket?

AgitatedDove14 clearml version on the Cleanup Service is 0.17.0

Maybe even a dedicated argument specifically for apt-get packages, since it is very common to need stuff like that

Oh I get it, I thought it is only a UI issue... but it actually doesn't send it O_O

it will return a Config object right?

I'll anonimyze it and paste it in a sec

got it

Config == conf_obj no?

I mean, I barely have 20 experiments

I'm really confused, I'm not sure what is wrong and what is the relationship between the templates the agent and all of those thing

For the meantime, I'm giving up on the pipeline thing and I'll write a bash script to orchestrate the execution, because I need to deliver and I'm not feeling this is going anywhere

On an end note I'd love for this to work as expected, I'm not sure what you need from me. A fully reproducible example will be hard because obviously this is proprietary code. What ...

AgitatedDove14

So nope, this doesn't solve my case, I'll explain the full use case from the beginning.

I have a pipeline controller task, which launches 30 tasks. Semantically there are 10 applications, and I run 3 tasks for each (those 3 are sequential, so in the UI it looks like 10 lines of 3 tasks).

In one of those 3 tasks that run for every app, I save a dataframe under the name "my_dataframe".
What I want to achieve is once all tasks are over, to collect all those "my_dataframe" arti...

So what's our next move FriendlySquid61 ? O_O

anyway, my ultimate goal is to create templates for other tasks... Is that possible in any other way through the CLI?

found it here:
https://github.com/allegroai/trains/blob/98ea965e6d0a39019b31e5fec146ab55da0d52e8/trains/storage/manager.py#L41

CostlyOstrich36 so why 1000:1000? My user and group are not that and so do all the otehr files I have under /opt/clearml

few minutes and I'll look at it

Okay, looks interesting but actually there is no final task, this is the pipeline layout

So how can I replace it with the MINIO?

Maybe something similar to dockers, that I could name each one of my trains agents and then refer to them by name something like

trains-agent daemon --name agent_1 ...

Then
trains-agent stop/start

I've dealt with this earlier today because I set up 2 agents, one for each GPU on a machine, and after editing configurations I wanted to restart only one of them (because the other was working) and then I noticed I don't know which one to kill

it seems apiserver_conf doesn't even change

In the larger context I'd look on how other object stores treat similar problems, I'm not that advanced in these topics.

But adding a simple force_download flag to the get_local_copy method could solve many cases I can think of, for example I'd set it to true in my case as I don't mind the times it will re-download when not necessary as it is quite small (currently I always delete the local file, but it looks pretty ugly)

yep