Hi AgitatedDove14 , I've been using Prefect for some of my data ETL needs, and so my understanding might be colored by that. But essentially Prefect also has agents to run jobs on machines where the processes run (which seems to be exactly the same model as in ClearML), and this data is highly regulated data, so it must stay on the local machine and not "accidentally" make its way to the cloud or any unapproved server. So my question was aimed at understanding whether the managed version of ClearML would be usable in this situation or not. Thank you!

Hi CrookedAlligator14

or is underlying data also accessible?

What do you mean by "underlying data" ?

But one thing that I am curious about is the Datasets section - does that also just show meta-data? Am I right in understanding that for the managed version of clearml to be completely free of storing any of my data that the file_server should be set to some other data source (e.g. S3)?

The main difference that with ClearML the agents are running on Your machines (either local or on Your cloud account) the clearml-server does not actually have access to the data streaming through it.
Does that make sense ?

Yes

But essentially Prefect also has agents to run jobs on machines where the processes run (which seems to be exactly the same model as in ClearML),

Yes ait is conceptually very similar

this data is highly regulated data, ...

The main difference that with ClearML the agents are running on Your machines (either local or on Your cloud account) the clearml-server does not actually have access to the data streaming through it.
Does that make sense ?