A colleague was asking about it, and especially how hard it would be to, like, save off the "best" model instead of the last

SmallDeer34 in theory no reason it will not work with it.
If you are doing a single node (from Ray's perspective)
This should just work, the challenge might be multi-node ray+cleaml (as you will have to use clearml to set the environment and ray as messaging layer (think openmpi etc.)
What did you have in mind?

save off the "best" model instead of the last

Should be relatively easy to update on the main Task the model with the best performance, no?