Hey @<1545216070686609408:profile|EnthusiasticCow4> It don't selects the archived dataset by default.
I had 2 datasets on archive and 0 unarchived. When I ran the following command:
Dataset.list_datasets(dataset_project=self.task.get_project_name(), only_completed=True)
It returned two entrees for the two datasets I had on archive.
@<1539780284646428672:profile|PoisedElephant79> Are you sure you're not simply referring to the get operation? That seems to exclude archived datasets. But I don't see anything like that for the list_datasets operation.
You can check it out for deep details.
I'm aware of that but it doesn't help this situation.
or can have a look in that file, you will get clear picture of it.
@<1545216070686609408:profile|EnthusiasticCow4> you can add some tags in the dataset in previous steps and can reuse it later.
@<1539780284646428672:profile|PoisedElephant79> Sorry for not getting back with this sooner. Dataset.get() doesn't work like you suggested. In the documentation it's clear:
Get a
specific
Dataset. If multiple datasets are found, the dataset with the highest semantic version is returned. If no semantic version is found, the most recently updated dataset is returned. This functions raises an Exception in case no dataset can be found and the
auto_create=True
flag is not set
I tested it and it does return a Dataset object but it's only of one dataset. I don't think it would make sense to return several datasets when you examine the other methods.