Hmmm are you saying the Dataset Tasks do not have the "dataset" system_tag as well as the type ?

true 😞

Task.get_task(..).system_tags

I do see the “Data Processing” type task in UI together with all other dataset-related features, like lineage plot

system tags cannot be seen from UI. right?

I create the dataset like this:
` project_name = "Sandbox"
task_name = "get_raw_data"
task = Task.init(project_name=project_name,
task_name=task_name,
task_type=Task.TaskTypes.data_processing,

             )

dataset = Dataset.create(use_current_task=True)

adding some files here

dataset.upload(verbose=True)

dataset.finalize(verbose=True) `

I think the reason is that the "original" task is already the right type. I'll make sure we fix it, and always set the system tag

Hi FiercePenguin76

should return all datasets from all projects?

Correct 🙂

I see that in the end, both query functions are calling Task._query_tasks

"system_tags": [ "development" ],

I guess so, lemme double-check

but I don’t get to this line, because my task is already of type data_processing

Ohh I see now, it should have added the Tag regardless, you are correct.

Basically, my problem is that it returns empty result. In the same code I can get dataset by its ID and I can get the task (which created the dataset) using
Task.get_tasks()(without mentioning th ID explicitly)

when I go into Dataset.list_datasets with the debugger and remove system_tags=[“dataset”] from api call params - I get the correct response back

and the problem is with system_tags=[“dataset”]

if the task is of wrong type (not data_processing) - then it’ll get both correct type and correct system tag

here: https://github.com/allegroai/clearml/blob/master/clearml/datasets/dataset.py#L75

but I don’t get to this line, because my task is already of type data_processing

also - line 77 which sets (non-system) tags is not invoked for me, thus if I define different tags for both task and dataset - then latter is being lost