@<1523701205467926528:profile|AgitatedDove14> when my codes get the clearml datasets, it stores in the cache e.g. /$HOME/.clearml/cache....

I wanted it to be in a mounted PV instead, so other pods (in same node) who needed same datasets can use without pulling again.

By the way, will downloading still happen if the datasets is available in the cache folder?

If it is cached, then there is no need to re-download 🙂

By the way, will downloading still happen if the datasets is available in the cache folder? Any specific settings to add to Dataset.get_local_copy()?

When you set the pod make sure you mount the clearml local cache folder to the PV
basically /root/.clearml/cache/

I have yet to figure out how to do so, would appreciate if u could give some guidance

OK let me try by adding to vol mount.

Hi @<1523701304709353472:profile|OddShrimp85>
Do you mean Dataset.get_local_copy() ?