Hi everyone,
I'm having an issue with Clearml Datasets and would like to know if this is possible.
I have a task that is executed repeatedly. This task may require data to be loaded and updated from a dataset. My question is: Is there a way to append to a CLEARMl dataset without creating new versions? If this task is created several times a day, this would otherwise lead to an increasing number of dataset versions. Is there a way to avoid this? I tried not to finalizea dataset, but then I could not download a mutable copy. If I finalize, I need to create a new version of the dataset. Is that right? If not, is there another way to share data between two tasks without using an external database? These will be created and triggered from an external UI or TaskScheduler.