is this already available or only on github?

Well this will have to wait a bit... my clearml-server is causing problems

is it possible to access the children tasks of the pipeline from the pipeline object?

AgitatedDove14 worked like a charm, thanks a lot!

I think I got it, I'll ping her again if it won't succeed

Yep what 😄

very cool

the ability to exexute without an agent i was just talking about thia functionality the other day in the community channel

Sorry but its not clear

we are running the agent on the same machine AgitatedDove14 , it worked before upgrading the clearml... we never set these credentials

But what does it do?

AgitatedDove14

BTW is the if not cached_file: return cached_file is legit or a bug?

found it here:
https://github.com/allegroai/trains/blob/98ea965e6d0a39019b31e5fec146ab55da0d52e8/trains/storage/manager.py#L41

-_- why there isn't a link to source on the docs?

😄

no need to do it again, I ahve all the settings in place, I'm sure it's not a settings thing

So just to correct myself and sum up, the credentials for AWS are only in the cloud_credentials_*

I doubled checked the credentials in the configurations, and they have full EC2 access

I'll try to put it again

Thanks very much

Now something else is failing, but I'm pretty sure its on my side now... So have a good day and see you in the next question 😄

Here you go

what if i want it to use ssh creds?

AgitatedDove14

So nope, this doesn't solve my case, I'll explain the full use case from the beginning.

I have a pipeline controller task, which launches 30 tasks. Semantically there are 10 applications, and I run 3 tasks for each (those 3 are sequential, so in the UI it looks like 10 lines of 3 tasks).

In one of those 3 tasks that run for every app, I save a dataframe under the name "my_dataframe".
What I want to achieve is once all tasks are over, to collect all those "my_dataframe" arti...

AgitatedDove14

Okay, looks interesting but actually there is no final task, this is the pipeline layout

That was the issue then

Example code? I didn't see anywhere an example of filtering using project name

I assume trains passes it as is, so I think the quoting I mentioned might work

What about permissions to the machines that are being spun up? For exampel if I want the instances to have specific permissions to read/write to S3 for example, how do I mange those?