Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I'M Looking To Utilize The Trains Aws Autoscaler Functionality, But After Going Through Its Docs A Few Times I Still Don'T Get It. Ultimately, My Setup Is That I Have Multiple Data Scientists Working On Static Instances, And They Have Queues Available To

I'm looking to utilize the Trains AWS autoscaler functionality, but after going through its docs a few times I still don't get it.

Ultimately, my setup is that I have multiple data scientists working on static instances, and they have queues available to them where they can push work to ( task.execute_remotely ).
I want the data scientists to be able to push to queue1 and queue2 and that the autoscaling service will spin up instances to execute these jobs and terminate them when done.

Looking at the doc, the configuration makes sense, but what I don't understand is why the configuration related to a specific task and project? I mean, why to configure it I need to go to DevOps project, and then enqueue it to the services queue? What does this project has to do anything with the project my data scientists are working on and what does this specific task has to do with their tasks?
I'm a little confused about this whole thing, and if there is a place I don't want to make errors in is exactly where code is going to control how much I spend šŸ™‚

I'd love some clarifications about how to make my setup work with that, on my own projects and queues

  
  
Posted 4 years ago
Votes Newest

Answers 21


The AWS autoscaler is running as a https://github.com/allegroai/trains-server#trains-agent-services-- .
For services, we use a trains-agent configure to be in services mode. Now, each ā€œservice taskā€ you insert to the queue should run until you stop it.

So, you need the AWS autoscaler keep running, which the services agent provide you. The ā€œservice taskā€ is still a task, so it should have a project and a task name (just one for ALL your projects).

You can always run things locally without this services agent (and you will need to control the services in this case).

What do you think?

  
  
Posted 4 years ago

Okay, so let me get this straight

The autoscaling is basically an ever-running task (lets say on the services queue). Now, the actual auto scaling and which queues exist have nothign to do with that, and are configured in the auto scale task?

  
  
Posted 4 years ago

The wizard is part of trains, this is the part you need - https://allegro.ai/docs/examples/services/aws_autoscaler/aws_autoscaler/#running-using-the-script

About

I should first create a auto scaling group in AWS?

you donā€™t need to, but if you want, you can do it and pass the credentials to the wizard

  
  
Posted 4 years ago

permission management

Same as your aws credentials

  
  
Posted 4 years ago

and in the UI configuration I didn't understand where does permission management came into play

  
  
Posted 4 years ago

So prior to doing any work on the trains autoscaler servcice, I should first create a auto scaling group in AWS?

  
  
Posted 4 years ago

Oh... from the docs I understood that I don't have to run the script, that I can either configure it in the UI, or with the sscript (wizard) so I ignored it up until now

  
  
Posted 4 years ago

Hi WackyRabbit7

The DevOps project is the project for the AWS autoscaler to run as a service. If you have other project that you run services from, you can change the name šŸ™‚

  
  
Posted 4 years ago

I mean I don't get how all the pieces add up

  
  
Posted 4 years ago

In the AWS autoscaler wizard, you need to enter your credentials to AWS. The wizard will use those so it should have the same permissions

  
  
Posted 4 years ago

Trains docs have at no point any mention on what should I do on the AWS interface... So I'm not sure at what point I should encounter this wizard

I'm going to play with it a bit and see if I can figure out how to make it work

  
  
Posted 4 years ago

What about permissions to the machines that are being spun up? For exampel if I want the instances to have specific permissions to read/write to S3 for example, how do I mange those?

  
  
Posted 4 years ago

no, just run the wizard and start the service (enqueue it or run it locally)

  
  
Posted 4 years ago

how do I run this wizard? is this wizard train's or aws's?

  
  
Posted 4 years ago

You can start by running the wizard, itā€™s straightforward, just insert your values

  
  
Posted 4 years ago

I don't fully get it - it says it has to be enqueued

  
  
Posted 4 years ago

Do i need to copy this aws scaler task to any project I want to have auto scaling on? what does it mean to enqueue hte aws scaler?

  
  
Posted 4 years ago

So once I enqueue it is up? Docs says I can configure the queues that the auto scaler listens to in order to spin up instances, inside the auto scale task - I wanted to make sure that this config has nothing to do to where the auto scale task was enqueued to

  
  
Posted 4 years ago

So once I enqueue it is up?

If the trains-agent is listening to this queue (services mode), yes.

Docs says I can configure the queues that the auto scaler listens to in order to spin up instances, inside the auto scale task - I wanted to make sure that this config has nothing to do to where the auto scale task was enqueued to

You are right, the auto scale task has nothing to do to where the auto scale task is enqueued

  
  
Posted 4 years ago

yep, but you need to have some initial run for the task to get into the UI (even remote execution mode)

  
  
Posted 4 years ago

The AWS autoscaler doesnā€™t related to other tasks, you can think about it as a service running in your Trains system.

and are configured in the auto scale task?

Didnā€™t get that šŸ˜•

  
  
Posted 4 years ago
931 Views
21 Answers
4 years ago
one year ago
Tags
Similar posts