Basically, for a bit more context, this is part of an effort to incorporate ClearML Pipelines in a CI/CD framework.

@<1523701079223570432:profile|ReassuredOwl55> did you check these examples?
None
None
None

And I’d rather the testing/validation etc lived outside of the ClearML Pipeline itself,

Makes total sense to me.
The main point here is creating the pipeline "template", i.e. running it the first time from code. Actually you can do that manually, the pipeline from tasks serializes itself to a configuration object that you can edit/create externally, we can explore that (or maybe even create a dedicated interface