So, AgitatedDove14 what I really like about the approach with ClearML is that you can genuinely bring the architecture into the development process early. That has a lot of desirable outcomes, including versioning and recording of experiments, dataset versioning etc. Also it would enforce a bit more structure in project development, if things are required to fit into a bit more of a defined box (or boxes). However, it also seems to be not too prescriptive, such that I would worry that a lot of effort has to go into getting things running above what would be needed during a development cycle.

What we want to achieve is robustness and speed of deployment, but not at the expense of over being overly rigorous and prescriptive in an early development cycle, when most of the effort should be going into the development and understanding of the problem, rather than the mechanics of getting something running that is more deployment friendly. The risk here is that you spend a lot of effort on things that really don't matter if the project doesn't go anywhere. This has to be weighed up against making the process to deployment easier and more efficient. It's a balancing act, but I am starting to see how something like ClearML might tread that fine line and be useful across the range of data science projects, from the very research and development end to the model deployment end.

I find it quite difficult to explain these ideas succinctly, did I make any sense to you?