from clearml import Task, Dataset

task = Task.init(
    project_name="MyProject",
    task_name="MyTask",
    task_type=Task.TaskTypes.data_processing,
    reuse_last_task_id=False,
    output_uri="

"
)

with open("new_file.txt", "w") as file:
    file.write("Hello, world!")

dataset = Dataset.create(parent_datasets=None, use_current_task=True)
dataset.add_files(".", wildcard="new_file.txt", verbose=True)
dataset.upload(verbose=True)
dataset.finalize(verbose=True)

So you'd recommend setting use_current_task=False when creating the dataset in this task or should this be done somehow differently?

Hi @<1613344994104446976:profile|FancyOtter74> , can you share the code that does that?

it was a year ago, so yeah.

I did similarly at my previous work (we had open source clearml deployed). The problem I described here was not present there. I liked this approach. It was convenient that dataset_id and task_id are the same.

Tried it on 1.13.1. Same problem. @<1523701087100473344:profile|SuccessfulKoala55> any advice?

Is it possible you used an older clearml version there where this was allowed?

Any updates? Should I provide any extra context?

I think that should work

clearml==1.12.2

Hi @<1613344994104446976:profile|FancyOtter74> , I think this is cause because you're creating a dataset in the same task. Therefor there is a connection between the task and the dataset and they are moved to a special folder for datasets. Is there a specific reason why you're creating both a Task & Dataset in the same code?

result: None