Hi TeenyBeetle18 , there is a GCP autoscaler in the PRO tier or Scale/Enterprise
I'm afraid it is not part of the open-source version. The PRO plan is pretty cheap (15 USD/month per user + some usages for applications & storage) compared to the price of the compute you're paying for on GCP. In the long run it would be saving money on idle machines and the time of your DevOps that need to raise/lower these machines all the time 🙂
CostlyOstrich36 hello, thank you! But what if I wanna have it in open-source version? It’s only one feature I want, and I can’t convince my CTO to buy PRO tier only because of it 🙂
Hi Aleksei,
As I understood it there is a self hosted pro version (Enterprise), but that is not 15 USD / Month.
That being said we build a custom solution in AWS because in the beginning, we where not aware of the autoscaler (and after testing the autoscaler I am still not sure whether we will stick to our solution).
Basically we build a solution outside of the server Instance. A seperate Instance is polling the ClearML queues, spinning up Instances, installing the ClearML Agent on them which then registers to the queues and fetches the experiments. If the instance is Idle for a defined time it will be shut down again. This should work on any cloud and is mainly written in Terraform.
It’s sad, but due to security measures we have to use self-hosted version and it seems like PRO
plan does not provide such option
Wow, sounds great! Thank you! I’ll do some research on Terraform