Exactly

💪

Example code? I didn't see anywhere an example of filtering using project name

First of all I wasn't aware that was an option - but I think it's preferable to be able to do it through the command line. Because I'm developing the pipeline to be executed remotely, but for debugging I run it locally.

Using what you showed I can obviously write it, and delete it once it is ready, and rewrite it when I'm debugging or adding features - but I think DX-wise it would be nicer to be able to trigger this functionality through the command line

Do i need to copy this aws scaler task to any project I want to have auto scaling on? what does it mean to enqueue hte aws scaler?

I was trying out the pipeline controller for the first time and I felt a bit of a burden that just for the sake of trying I had to launch an agent

If this includes scheduling through pipelines, in my opinion there should be an option to execute a pipeline without an agent. Sometimes for development I just want to execute a pipeline on my local machine just as I would a task...

I can easily see how this problem can reoccur

So once I enqueue it is up? Docs says I can configure the queues that the auto scaler listens to in order to spin up instances, inside the auto scale task - I wanted to make sure that this config has nothing to do to where the auto scale task was enqueued to

how do I run this wizard? is this wizard train's or aws's?

TimelyPenguin76 this fixed it, using the detect_with_pip_freeze as true solves the issue

So prior to doing any work on the trains autoscaler servcice, I should first create a auto scaling group in AWS?

I think a good idea is to add to the error message when the clearml agent fails due to import error, a suggestion ot try out with pip freeze

Okay, so let me get this straight

The autoscaling is basically an ever-running task (lets say on the services queue). Now, the actual auto scaling and which queues exist have nothign to do with that, and are configured in the auto scale task?

But does it disable the agent? or will the tasks still wait for the agent to dequeue?

What about permissions to the machines that are being spun up? For exampel if I want the instances to have specific permissions to read/write to S3 for example, how do I mange those?

👍

I'll try it

the explicity of the variable queue makes it feels like it runs somewhere else

👍

Searched for "custom plotly" and "log plotly" in search, didn't thinkg about "report plotly"

Cool thanks 🤩

when I specify --packages I shoudl manually list them all not?

Cool - so that means the fileserver which comes with the host will stay emtpy? Or is there anything else being stored there?

I guess what I want is a way to define environment variables in agents

CostlyOstrich36 so why 1000:1000? My user and group are not that and so do all the otehr files I have under /opt/clearml

so in my code, I'll use this environment variable to read from disk

okay, that's acceptable

you can use pgrep -af "trains-agent"