Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Trying To Setup The Storage (Aws S3) For All Components Of Clearml. I'M Deploying Clearml On Kubernetes With The Official Helm Charts. I Can'T Find A Way To Inject The Configuration File In The Webserver / Apiserver / Fileserver. With The Agent,

Hi, I'm trying to setup the Storage (AWS S3) for ALL components of ClearML. I'm deploying ClearML on Kubernetes with the official Helm charts.

I can't find a way to inject the configuration file in the webserver / apiserver / fileserver.

With the agent, it's pretty straight forward, with existingClearmlConfigSecret or clearmlConfig, but there's nothing like that for the other components (server, api, fileserver). The only "option" is to use a PVC for persistence.

I feel like I'm misunderstanding something. Should the webserver, apiserver and fileserver all use a S3 Storage backend as well?

  
  
Posted 23 days ago
Votes Newest

Answers 8


Hi @<1749602841338580992:profile|ImpressionableSparrow64> , the S3 configuration (Credentials) is always done on the client side. You don't need to configure anything server side. Also good that you configured the agent.

  
  
Posted 23 days ago

Not really, you can even point the files_server in clearml.conf to s3. the files server is there so there would be some basic storage solution attached.

  
  
Posted 22 days ago

ok yeah that makes sense. Thanks again John! 👍

  
  
Posted 22 days ago

Thanks for the clarification, much appreciated.

But I then wonder, is the fileserver necessary if only using S3 for the clients and agents?

  
  
Posted 23 days ago

Hi John, thanks for the clarification. I'm curious though: when uploading a dataset, it goes onto the fileserver Storage (PVC). If I want those to be stored on S3, it's a client-side setting?

  
  
Posted 23 days ago

Yes, it can be controlled via the local clearml.conf file

  
  
Posted 23 days ago

Hi @<1749602841338580992:profile|ImpressionableSparrow64> , I have a clearml-agent in my k8s cluster and I also have my clearml-api-keys stored as a secret inside my cluster and I would like to refer to it using: 'existingClearmlConfigSecret'. If my secret was created like so:

kubectl create secret generic clearml-api-secret \
  --from-literal=agentk8sglueKey="CLEARML_API_ACCESS_KEY" \
  --from-literal=agentk8sglueSecret="CLEARML_API_SECRET_KEY"

do I then just set existingClearmlConfigSecret: clearml-api-secret ?

  
  
Posted one day ago

actually never mind, I was able to figure it out.

  
  
Posted one day ago