Hmmm are you saying the Dataset Tasks do not have the "dataset" system_tag as well as the type ?
if the task is of wrong type (not data_processing) - then it’ll get both correct type and correct system tag
I see that in the end, both query functions are calling Task._query_tasks
when I go into Dataset.list_datasets with the debugger and remove system_tags=[“dataset”] from api call params - I get the correct response back
Hi FiercePenguin76
should return all datasets from all projects?
Correct 🙂
I do see the “Data Processing” type task in UI together with all other dataset-related features, like lineage plot
but I don’t get to this line, because my task is already of type data_processing
Ohh I see now, it should have added the Tag regardless, you are correct.
but I don’t get to this line, because my task is already of type data_processing
and the problem is with system_tags=[“dataset”]
Basically, my problem is that it returns empty result. In the same code I can get dataset by its ID and I can get the task (which created the dataset) usingTask.get_tasks()
(without mentioning th ID explicitly)
I create the dataset like this:
` project_name = "Sandbox"
task_name = "get_raw_data"
task = Task.init(project_name=project_name,
task_name=task_name,
task_type=Task.TaskTypes.data_processing,
)
dataset = Dataset.create(use_current_task=True)
adding some files here
dataset.upload(verbose=True)
dataset.finalize(verbose=True) `
also - line 77 which sets (non-system) tags is not invoked for me, thus if I define different tags for both task and dataset - then latter is being lost
I think the reason is that the "original" task is already the right type. I'll make sure we fix it, and always set the system tag
system tags cannot be seen from UI. right?