Hi
i'm trying to finalize a dataset, but although the finalize(auto_upload=True)
completes successfully (see image) - But the dataset is still in an uploading status (resulting with is_fianl() as False)
The dataset has a single parquet
file
I notices it when trying to get a local copy
I'm running clearml==1.11.1
ds.get_local_copy()
ValueError Traceback (most recent call last)
Cell In[37], line 1
----> 1 ds.get_local_copy()
File ~/..../.direnv/python-3.10.7/lib/python3.10/site-packages/clearml/datasets/dataset.py:940, in Dataset.get_local_copy(self, use_soft_links, part, num_parts, raise_on_error, max_workers)
938 self._task = Task.get_task(task_id=self._id)
939 if not self.is_final():
--> 940 raise ValueError("Cannot get a local copy of a dataset that was not finalized/closed")
941 max_workers = max_workers or psutil.cpu_count()
943 # now let's merge the parents
ValueError: Cannot get a local copy of a dataset that was not finalized/closed
Any suggestions?
