Hi SmugSnake85 , what data exactly do you have there?

SuccessfulKoala55 basically I am having image data that is been stored on daily basis there. So precisely what I want to do is take all the data from their store in clear ml and then train my models here in clearml and clearml should choose which model is performing the best and run that model for predictions.

Hi SmugSnake85 , Did you upload the data with clearml-data?

TimelyPenguin76 not yet i am still finding a way to upload data from MinIo as my data is present on it...

try to do it with clearml-data, https://clear.ml/docs/latest/docs/clearml_data/ , you can access and share the data with it and can configure it to work with minio

TimelyPenguin76 do i need to provide access key to connect with MinIo to connect ?? cause in the documentation its shown how to insert dataset when its present locally

I think I didnt understand the original question, you are looking for a dataset with your original minio links? based on those?

TimelyPenguin76 yes I am having data set in MinIo from there i need to import the dataset to Clearml. So for Importing the dataset I am having a Link from MinIo now the problem is i am not able to fetch the data.

??

Hi SmugSnake85 , so you just want it to point to your minio links?

TimelyPenguin76 I just want to add my data in clearml through MinIo as I have my data over there. I don't have my data in my local. I did change the configuration file and gave access keys example I have shown you below.

TimelyPenguin76 any reliable info for the solution ??

SmugSnake85 clearml-data only supports local files which it manages and uploads to a remote storage location. Managing references to remote data (as it seems you're looking for) is something supported in the clearml paid version (called hyper datasets, you can read about it here - https://clear.ml/docs/latest/docs/hyperdatasets/overview )