The above works for me, so if you try and the command line version does not work, there might be a bug. Please post the exact commands you use when you try it π
For the record, this is a minimal reproducible example:
Local folder structure:βββ remove_folder β βββ batch_0 β β βββ file_0_0.txt β β βββ file_0_1.txt β β βββ file_0_2.txt β β βββ file_0_3.txt β β βββ file_0_4.txt β β βββ file_0_5.txt β β βββ file_0_6.txt β β βββ file_0_7.txt β β βββ file_0_8.txt β β βββ file_0_9.txt β βββ batch_1 β βββ file_1_0.txt β βββ file_1_1.txt β βββ file_1_2.txt β βββ file_1_3.txt β βββ file_1_4.txt β βββ file_1_5.txt β βββ file_1_6.txt β βββ file_1_7.txt β βββ file_1_8.txt β βββ file_1_9.txt βββ remove_folder.ipynb
` from clearml import Dataset
Create the dataset
ds = Dataset.create(
dataset_project='issues',
dataset_name='remove_folder_test'
)
ds.add_files('remove_folder')
ds.finalize(auto_upload=True)
Create a child dataset (Create new and provide parent, but writable_copy will do this for you!)
ds = Dataset.get(
dataset_project='issues',
dataset_name='remove_folder_test',
writable_copy=True
)
print(ds.list_files())
Will print both batch_0 and batch_1 files
ds.remove_files(dataset_path='batch_0/*')
print(ds.list_files())
Will print only batch_1 files
ds.finalize(auto_upload=True)
Now check for certain with local copy
import os
os.listdir(Dataset.get(dataset_id=ds.id).get_local_copy())
Should return only 'batch_1' `
Hi ExuberantParrot61 ! Can you try using a wildcard? E.g. ds.remove_files(dataset_path='folder_to_delete/*')
Thanks will try. I was using the command line, also with wildcards.