Hello everyone,
I'm in a bit of situation where I want to optimize getting local copies of datasets, by making it so that if a parent dataset already exists locally, the child simply creates symlinks to the files it shares with the parent. However, I can't seem to find a way to get a dataset's children, which is something I need in case a child dataset already exists locally and thus I could potentially create symlinks in the parent dataset cache to the existing local files in a child's cache.
Although I was more thinking of downloading the entire parent dataset in its own cache and if there are any existing children locally, edit their caches to convert the shared files to symlinks to the now existing parent dataset cache.
(When I say cache I mean the directory get_local_copy downloads the datasets to)
Right now I'm writing a custom function to do this, but it would be great if all of this could already exist in ClearmlML 😁 .

Also, on a bit of a related note, get_dependency_graph doesn't appear to return the entire dependency graph (i.e., to the very root parent), it only links to the immediate parents.
As for what I would love to have now is something like get_children , meaning that it returns (at the very least) the immediate children of the dataset (i.e., datasets that have this dataset as their parent), best case it could return the entire descendant graph. But I'm open to currently substituting this for some less elegant solution (that would hopefully not involve indexing the entire server and resolving complete graphs that way)

Thanks!