thx TimelyPenguin76
skimming over this, I can't find how to filter by project name or something similar

awesome 💪

alabaster==0.7.12 appdirs==1.4.4 apturl==0.5.2 attrs==21.2.0 Babel==2.9.1 bcrypt==3.1.7 blinker==1.4 Brlapi==0.7.0 cachetools==4.0.0 certifi==2019.11.28 chardet==3.0.4 chrome-gnome-shell==0.0.0 clearml==1.0.5 click==8.0.1 cloud-sptheme==1.10.1.post20200504175005 cloudpickle==1.6.0 colorama==0.4.3 command-not-found==0.3

👍

This is what I meant should be documented - the permissions...

😅

Hey Jake thanks for responding

?

Cool thanks 🤩

AgitatedDove14 since this is a powerful feature, I think this should be documented. I'm at a point where I want to use the AWS autoscaler and i'm not sure how.

I see in the docs that I need to supply the access+secret keys, which are associated with an IAM, but nowhere does it say what permissions does this IAM need in order to execute.

Also using the name "AWS Autoscaler" immediately suggests that behind the scene, trains uses the https://docs.aws.amazon.com/autoscaling/ec2/userguide/wha...

I have a single IAM, my question is what kind of permissions I should associate with the IAM so that the autoscaler task will work

Does that mean that teh AWS autoscaler in trains, manages EC2 auto scaling directly without using the AWS built in EC2 auto scaler?

which permissions should it have? I would like to avoid full EC2 access if possible, and only choose the necessary permissions

so putting the docs aside, what permissions should I give to the IAM associated with trains' autoscale ?

If the credentials don't have access tothe autoscale service obviously it won't work

Cool, thats more than reasonable!

SuccessfulKoala55 ?

Okay so that is a bit complicated

In our setup, the DSes don't really care about agents, the agents are being managed by our MLops team.
So essentially if you imagine it the use case looks like that:
A data scientists wants to execute some CPU heavy task. The MLops team supplied him with a queue name, and the data scientist knows that when he needs something heavy he pushes it there - the DS doesn't know nothing about where it is executed, the execution environment is fully managed by the ML...

I don't even know where trains is coming from... While using the same environment I can't even import trains, see

I'm asking that because the DSes we have are working on multiple projects, and they have only one trains.conf file, I wouldn't want them to edit it each time they switch project

No I don't have trains anywhere in my code

Hi FriendlySquid61 I did all the changes you said

Please update me 🙂

Nope, quite some time has passed since 🙂 I might be able to replicate it later... still battling with the pipeline to make it work

When I ran the clearml-task --name ... -project ... -script .... it failed saying not requiremetns was found

can you provide an example?

moreover, in each pipeline I have 10 different settings of task A -> Task b (and then task C), each run 1-2 fails randomly

💪

you know what let me try in a sec

I couldn't do it with clearml task as it was looking for a requirements file and I'm workgin with poetry