Not really, you can even point the files_server in clearml.conf to s3. the files server is there so there would be some basic storage solution attached.

Hi John, thanks for the clarification. I'm curious though: when uploading a dataset, it goes onto the fileserver Storage (PVC). If I want those to be stored on S3, it's a client-side setting?

Hi @<1749602841338580992:profile|ImpressionableSparrow64> , the S3 configuration (Credentials) is always done on the client side. You don't need to configure anything server side. Also good that you configured the agent.

Yes, it can be controlled via the local clearml.conf file

ok yeah that makes sense. Thanks again John! 👍

Hi @<1749602841338580992:profile|ImpressionableSparrow64> , I have a clearml-agent in my k8s cluster and I also have my clearml-api-keys stored as a secret inside my cluster and I would like to refer to it using: 'existingClearmlConfigSecret'. If my secret was created like so:

kubectl create secret generic clearml-api-secret \
  --from-literal=agentk8sglueKey="CLEARML_API_ACCESS_KEY" \
  --from-literal=agentk8sglueSecret="CLEARML_API_SECRET_KEY"

do I then just set existingClearmlConfigSecret: clearml-api-secret ?

actually never mind, I was able to figure it out.

Thanks for the clarification, much appreciated.

But I then wonder, is the fileserver necessary if only using S3 for the clients and agents?