EnormousCormorant39 , there are SDK methods for using the datasets. I think this will simplify your process immensely.
https://clear.ml/docs/latest/docs/references/sdk/dataset

Also here is a small example for the usage 🙂
` task = Task.init(project_name="<PROJECT_NAME>", task_name="<TASK_NAME>")

#Create dataset
ds = Dataset.create(dataset_name="<DATASET_NAME>", dataset_project="PROJECT_NAME")
ds.add_files("<PATH_TO_FILE/S>")
ds.upload()
ds.finalize() `