well there are already processes in place.. we aim at migrating everything to ClearML, but we hoped we could do it gradually
I am not aware of how clearml-dataset works, but I'll have a look 🙂
You mean as output target for artifacts?For example, for some of our models we create pdf reports, that we save in a folder in the NFS disk.
I see... because the problem it would be with permissions when creating artifacts to store in the "/shared" folder
For example, for some of our models we create pdf reports, that we save in a folder in the NFS disk
Oh, why not as artifacts ? at least you will be able to access from the web UI, and avoid VFS credential hell 🙂
Regrading clearml datasets:
https://www.youtube.com/watch?v=S2pz9jn26uI
especially for datasets (for the models and other files we were thinking to use the fileserver any way)
but I was wondering if there's any limitation in creating an image with a non root user to use as the actual worker?
SarcasticSquirrel56 non-root pods (containers) are fully supported,
I would recommend using the latest agent RC (that simplified a few things)clearml-agent==1.4.0rc3
I see... because the problem it would be with permissions when creating artifacts to store in the "/shared" folder
You mean as output target for artifacts ?
especially for datasets (for the models and other files we were thinking to use the fileserver any way)
clearml-dataset ?
Hi SarcasticSquirrel56 , running an actual worker as non-root is always an issue as many processes inside the docker image rely on having root privileges...
