AnxiousSeal95 At first sight, the pipeline logic of ClearML seems binding with ClearML quite a bit. Back then I was considering I need something that can convert to Production pipeline (e.g. Airflow DAGs) easily, as we need pipelines not just for Experiments, Airflow seems to be the default one.

Also, clearml-data was not available when we started the development of internal framework. As for clear-agent, from my previous experience, it seems not working great with Window sometimes, and also the logic catching python environment often fail.

To use clearml-agent, data versioning need to go first since the agent would need to know where to get the data. It need to cache environment/data so the overhead of running experiments should be as low as possible. Maybe using clearml-pipeline would make it easier.

At the time I try that, it looks like it is not mature yet so I didn't spend too many time on it. And since we already have a working solution, so we do not have the motivation to switch yet.

For me, some real case study/tutorial that put every clearml-agent component together would provide some context that how ClearML thinks it is the best way to structure things. If you know some good source of documentation, please let me know! As I look up the documentation, most of the time it is a snippet that achieve a certain thing.