ok yeah that makes sense. Thanks again John! 👍

Hi John, thanks for the clarification. I'm curious though: when uploading a dataset, it goes onto the fileserver Storage (PVC). If I want those to be stored on S3, it's a client-side setting?

Yes, it can be controlled via the local clearml.conf file

Hi @<1749602841338580992:profile|ImpressionableSparrow64> , I have a clearml-agent in my k8s cluster and I also have my clearml-api-keys stored as a secret inside my cluster and I would like to refer to it using: 'existingClearmlConfigSecret'. If my secret was created like so:

kubectl create secret generic clearml-api-secret \
  --from-literal=agentk8sglueKey="CLEARML_API_ACCESS_KEY" \
  --from-literal=agentk8sglueSecret="CLEARML_API_SECRET_KEY"

do I then just set existingClearmlConfigSecret: clearml-api-secret ?

actually never mind, I was able to figure it out.

Not really, you can even point the files_server in clearml.conf to s3. the files server is there so there would be some basic storage solution attached.

Thanks for the clarification, much appreciated.

But I then wonder, is the fileserver necessary if only using S3 for the clients and agents?

Hi @<1749602841338580992:profile|ImpressionableSparrow64> , the S3 configuration (Credentials) is always done on the client side. You don't need to configure anything server side. Also good that you configured the agent.