Unanswered
Hey! Starting An Mlops Director Position In 2 Weeks. I'M Thinking About Architecture.
Has Anyone Ever Tried To Use Clearml As An Experiment Tracker, But Used A Different Orchestrator Like Metaflow, Airflow, Prefect, Etc.? I'M Struggling To Find Guides Or
@<1523701482157772800:profile|AnxiousSeal95> I see a lot of people here migrating data from one data source to another.
For us it was that we experimented with Clearml to get the feeling and we used clearml built in file storage to save debug images an all other artifacts.
Then we grew rapidly and we had to migrate to S3 storage.
I had to write a script that goes through elasticsearch and mongo db to point to new S3 links wher the data was migrated to.
I do however understand that migration in itself is not easy and there isnt a magical button to solve this issue. However, exposed API that could change the artifact file path prefix maybe could be useful
21 Views
0
Answers
2 months ago
2 months ago