What about permissions to the machines that are being spun up? For exampel if I want the instances to have specific permissions to read/write to S3 for example, how do I mange those?
I mean I don't get how all the pieces add up
Do i need to copy this aws scaler task to any project I want to have auto scaling on? what does it mean to enqueue hte aws scaler?
You can start by running the wizard, it’s straightforward, just insert your values
permission management
Same as your aws credentials
In the AWS autoscaler wizard, you need to enter your credentials to AWS. The wizard will use those so it should have the same permissions
how do I run this wizard? is this wizard train's or aws's?
So prior to doing any work on the trains autoscaler servcice, I should first create a auto scaling group in AWS?
Oh... from the docs I understood that I don't have to run the script, that I can either configure it in the UI, or with the sscript (wizard) so I ignored it up until now
The AWS autoscaler is running as a https://github.com/allegroai/trains-server#trains-agent-services-- .
For services, we use a trains-agent
configure to be in services mode. Now, each “service task” you insert to the queue should run until you stop it.
So, you need the AWS autoscaler keep running, which the services agent provide you. The “service task” is still a task, so it should have a project and a task name (just one for ALL your projects).
You can always run things locally without this services agent (and you will need to control the services in this case).
What do you think?
and in the UI configuration I didn't understand where does permission management came into play
So once I enqueue it is up? Docs says I can configure the queues that the auto scaler listens to in order to spin up instances, inside the auto scale task - I wanted to make sure that this config has nothing to do to where the auto scale task was enqueued to
yep, but you need to have some initial run for the task to get into the UI (even remote execution mode)
Okay, so let me get this straight
The autoscaling is basically an ever-running task (lets say on the services
queue). Now, the actual auto scaling and which queues exist have nothign to do with that, and are configured in the auto scale task?
no, just run the wizard and start the service (enqueue it or run it locally)
So once I enqueue it is up?
If the trains-agent
is listening to this queue (services mode), yes.
Docs says I can configure the queues that the auto scaler listens to in order to spin up instances, inside the auto scale task - I wanted to make sure that this config has nothing to do to where the auto scale task was enqueued to
You are right, the auto scale task has nothing to do to where the auto scale task is enqueued
The wizard is part of trains, this is the part you need - https://allegro.ai/docs/examples/services/aws_autoscaler/aws_autoscaler/#running-using-the-script
About
I should first create a auto scaling group in AWS?
you don’t need to, but if you want, you can do it and pass the credentials to the wizard
The AWS autoscaler doesn’t related to other tasks, you can think about it as a service running in your Trains system.
and are configured in the auto scale task?
Didn’t get that 😕
Trains docs have at no point any mention on what should I do on the AWS interface... So I'm not sure at what point I should encounter this wizard
I'm going to play with it a bit and see if I can figure out how to make it work
I don't fully get it - it says it has to be enqueued
Hi WackyRabbit7
The DevOps
project is the project for the AWS autoscaler to run as a service. If you have other project that you run services from, you can change the name 🙂