Hi @<1613344994104446976:profile|FancyOtter74> , can you share the code that does that?
from clearml import Task, Dataset
task = Task.init(
project_name="MyProject",
task_name="MyTask",
task_type=Task.TaskTypes.data_processing,
reuse_last_task_id=False,
output_uri="
"
)
with open("new_file.txt", "w") as file:
file.write("Hello, world!")
dataset = Dataset.create(parent_datasets=None, use_current_task=True)
dataset.add_files(".", wildcard="new_file.txt", verbose=True)
dataset.upload(verbose=True)
dataset.finalize(verbose=True)
Tried it on 1.13.1. Same problem. @<1523701087100473344:profile|SuccessfulKoala55> any advice?
Is it possible you used an older clearml version there where this was allowed?
Hi @<1613344994104446976:profile|FancyOtter74> , I think this is cause because you're creating a dataset in the same task. Therefor there is a connection between the task and the dataset and they are moved to a special folder for datasets. Is there a specific reason why you're creating both a Task & Dataset in the same code?
I did similarly at my previous work (we had open source clearml deployed). The problem I described here was not present there. I liked this approach. It was convenient that dataset_id and task_id are the same.
So you'd recommend setting use_current_task=False
when creating the dataset in this task or should this be done somehow differently?
Any updates? Should I provide any extra context?