Hi there, would it be possible for the autoscaler to support stopping instances instead of terminating them?
My use case is the following: I am continuing my journey with the clearml-session tool, and in case the clearml-session is running in a ec2 instance, I want to stop the instance and restart it later, so that I don’t loose my progress on my notebooks but I don’t pay for the idle instance. I can do the stopping/restarting manually, but I am thinking it would be very handy if clearml could handle it for me. Of course this would need to be refined to be properly integrated, but in general, would something like that be possible?
EDIT: Maybe this problem can be tackled from a different perspective and be seen as trying to move a clearml-session task state from one agent to another. This allows to terminate an agent when I am done for the day, and continue with another the day after
EDIT 2: For now I can think of a simple solution, which is to centralize my notebooks in a github repo, so that I can simply push my progress at the end of the day and restore it in another agent the day after. This works as far as I don’t have big artifacts living in the instance (that I wouldn’t be able to push to github). I guess this is fine as I can deal with such big artifacts separately (eg. send them to s3)