Hi all,
We are about to setup the ML infra on one of the ongoing projects where we used ClearML server for CV models training. The parts of the infra should be:

• Training- Execution of a set of complex pre-processing pipelines that extracts data from the PDF files and prepares it for the model training. Where each new data batch should be added to previously split train/test/val datasets only once.
• Run model training (we are about to use AWS GPU instances for that).
• Run the validation module that inference a newly trained model on some cases verifying it provides accurate predictions + plus metrics check.
• If the validation passed successfully deploys the model into production.- Inference- The inference steps would also be pretty the same as for training with complex pre-processing of large PDFs, so it's not merely a model inference (we used lambdas + sagemaker on other projects for that)
  Now, I'm wondering, if it's possible to do this all nicely via ClearML?
  Previously we were doing all of that via GitLab + AWS.
  The main concern in switching to ClearML is probably sophisticated preprocessing step for the inference. I just don't know how much the model serving is flexible in ClearML.

Would appreciate any guidance and materials you might have for reading. I already passed through the case studies and found this one particularly useful but did get a grasp on why there is a need for two orchestrators at a time (GitLab + ClearML).

Thank you!