try to do it with clearml-data, https://clear.ml/docs/latest/docs/clearml_data/ , you can access and share the data with it and can configure it to work with minio

TimelyPenguin76 do i need to provide access key to connect with MinIo to connect ?? cause in the documentation its shown how to insert dataset when its present locally

TimelyPenguin76 any reliable info for the solution ??

TimelyPenguin76 I just want to add my data in clearml through MinIo as I have my data over there. I don't have my data in my local. I did change the configuration file and gave access keys example I have shown you below.

TimelyPenguin76 not yet i am still finding a way to upload data from MinIo as my data is present on it...

TimelyPenguin76 yes I am having data set in MinIo from there i need to import the dataset to Clearml. So for Importing the dataset I am having a Link from MinIo now the problem is i am not able to fetch the data.

Hi SmugSnake85 , what data exactly do you have there?

Hi SmugSnake85 , so you just want it to point to your minio links?

I think I didnt understand the original question, you are looking for a dataset with your original minio links? based on those?

SuccessfulKoala55 basically I am having image data that is been stored on daily basis there. So precisely what I want to do is take all the data from their store in clear ml and then train my models here in clearml and clearml should choose which model is performing the best and run that model for predictions.

SmugSnake85 clearml-data only supports local files which it manages and uploads to a remote storage location. Managing references to remote data (as it seems you're looking for) is something supported in the clearml paid version (called hyper datasets, you can read about it here - https://clear.ml/docs/latest/docs/hyperdatasets/overview )

??

Hi SmugSnake85 , Did you upload the data with clearml-data?