remove this params will solve use_current_task=True,
Hello everyone!
i quite confuse about update data using sdk without download it.
first i create dataset and upload.
` dataset_path = 'dataset/batch_1'
dataset = Dataset.create(
dataset_name="dataset_name",
dataset_project="project_name",
description=description,
output_uri=" ",
)
dataset.add_files(path=dataset_path)
dataset.upload(show_progress=True, chunk_size=100, verbose=True)
dataset.finalize() then i want to update data (adding files)
dataset_path = 'dataset/batch_2'
dataset = Dataset.get(dataset_id='b8bd457374ee4e388f1bf22171d572fe')
new_dataset = Dataset.create(
dataset_name="dataset_name",
dataset_project="project_name",
description=description,
output_uri=" ",
parent_datasets=[dataset],
use_current_task=True,
)
new_dataset.add_files(dataset_path)
new_dataset.upload(verbose=True)
new_dataset.finalize() but i got message like this, it seems pending upload, and the status on ui always 'uploading'
Uploading dataset changes (941 files compressed to 16.77 MiB) to
File compression and upload completed: total size 16.77 MiB, 1 chunk(s) stored (average size 16.77 MiB) `