Is it possible to have clearml launch aws instances to run clearml-agent , execute jobs and then shutdown after the jobs have been completed?
I have two relevant use cases.
I have two tasks (experiments) in a queue that need to run on two different machines. I can launch two EC2 instances with clearml-agent running on them and execute one task each. Since running a GPU instance is expensive, I only want to launch an instance when I execute a task and not have any idle time.
Is it possible to automate the above workflows with clearml-agent ? Otherwise, I'll have to create a separate process to manage these resources.
Yes, this is what I'm looking for. Excellent! Thank you!
I'm assuming your are looking for the AWS autoscaler, spinning EC2 instances up/down and running daemons on them.
https://github.com/allegroai/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py
https://clear.ml/docs/latest/docs/guides/services/aws_autoscaler
