AgitatedDove14 as always much obliged to your fast responses this is actually incredible!
Yeah a bit clearer, something like this in the docs would be really helpful 😉 At least the last part as Storagemanager is actually quite clear.
Maybe I can sum up my understanding?
So am I right in the assumption that I can manage data and the passing of such between tasks either by
Managing them in a folder structure via datasets with the potential issue of syncing a lot of data between tasks and works (obviously accounting for caching on workers between tasks) Managing them as artefacts of Tasks and passing them explicitly coupled to tasks to another task? In that case loosing some of the tracing of datasets and the nice graph? Use both as necessary simultaneously 😄
Btw I sometimes get a gzip error when I am accessing artefacts via the '.get()' part.