I think the reason is that the "original" task is already the right type. I'll make sure we fix it, and always set the system tag

but I don’t get to this line, because my task is already of type data_processing

Ohh I see now, it should have added the Tag regardless, you are correct.

also - line 77 which sets (non-system) tags is not invoked for me, thus if I define different tags for both task and dataset - then latter is being lost

but I don’t get to this line, because my task is already of type data_processing

here: https://github.com/allegroai/clearml/blob/master/clearml/datasets/dataset.py#L75

if the task is of wrong type (not data_processing) - then it’ll get both correct type and correct system tag

"system_tags": [ "development" ],

Task.get_task(..).system_tags

true 😞

system tags cannot be seen from UI. right?

I guess so, lemme double-check

Hmmm are you saying the Dataset Tasks do not have the "dataset" system_tag as well as the type ?

I do see the “Data Processing” type task in UI together with all other dataset-related features, like lineage plot

I create the dataset like this:
` project_name = "Sandbox"
task_name = "get_raw_data"
task = Task.init(project_name=project_name,
task_name=task_name,
task_type=Task.TaskTypes.data_processing,

             )

dataset = Dataset.create(use_current_task=True)

adding some files here

dataset.upload(verbose=True)

dataset.finalize(verbose=True) `

when I go into Dataset.list_datasets with the debugger and remove system_tags=[“dataset”] from api call params - I get the correct response back

and the problem is with system_tags=[“dataset”]

I see that in the end, both query functions are calling Task._query_tasks

Basically, my problem is that it returns empty result. In the same code I can get dataset by its ID and I can get the task (which created the dataset) using
Task.get_tasks()(without mentioning th ID explicitly)

Hi FiercePenguin76

should return all datasets from all projects?

Correct 🙂