VivaciousBadger56 Thank you for the screenshots! I appreciate the effort. You indeed clicked on the right link, I was on mobile so had to instruct from memory 🙂
First of all: every 'object' in the ClearML ecosystem is a task. Experiments are tasks, so are dataset versions and even pipelines! Each task can be viewed using the experiment manager UI, that's just how the backend is structured. Of course we keep experiments and data separate by giving them a separate tab and different UI, but the underlying building blocks are the same. It makes it so you can easily access metadata in both datasets as in experiments using the same API 🙂
That said, it seems that isn't the case with this example for some reason. In the second screenshot on the left, we can clearly see that the 'metadata' is correctly stored in the dataset task. So the next thing I can think of is to check and make sure the system is trying to pull the right task.
Can you make sure that the self.original_dataset.id
in this snippet:
` self.original_dataset = Dataset.get(dataset_project=project_name, dataset_name='UrbanSounds example')
This will return the pandas dataframe we added in the previous task
self.metadata = Task.get_task(task_id=self.original_dataset.id).artifacts['metadata'].get() is the same ID as the one in the UI? (Check nr 1 in screenshot) You can add a simple
print(self.original_dataset.id) ` just to make sure.
I'm thinking this because in the snippet, you ask ClearML to get Dataset.get(dataset_project=project_name, dataset_name='UrbanSounds example')
the dataset with name UrbanSounds example
, but in the screenshot, you seem to have added something else to the dataset name that you erased with black (nr 2 on screenshot). Which would mean that ClearML will nog get this dataset and so cannot find the metadata.
Can you check this?