Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Everybody, I Am Using Clearml Self-Hosted. Is There A Way To Run Multiple Clearml Instances (Multiple Webservers, Apiservers And Fileservers) But With A Shared Databases (Elastic, Mongo)? Elasticsearch Requires A Lot Of Ram And It Would Be An Overki

Hello everybody,
i am using clearml self-hosted.
Is there a way to run multiple clearml instances (multiple webservers, apiservers and fileservers) but with a shared databases (elastic, mongo)?
Elasticsearch requires a lot of ram and it would be an overkill to have multiple instances. The reason i want to do that is the lack of authorization at the self hosted version

  
  
Posted 9 months ago
Votes Newest

Answers 10


Yeah I think you're right. You're basically looking for a capability to segregate users into groups so they won't have visibility between themselves?

  
  
Posted 9 months ago

I think all relevant env variables are in the docker compose. You can spin different APIservers on different machines without elastic/mongo/redis and then point them to the relevant machines that have those services running

  
  
Posted 9 months ago

I would suggest considering a Scale/Enterprise license. The amount of time you would be wasting in man hours on managing, doing hacks and patches on this and various other capabilities would most likely cost you in the long run much more than a license. Not to mention the dedicated support and other features you would be receiving.

  
  
Posted 9 months ago

hi @<1523701070390366208:profile|CostlyOstrich36> thanks for replying.
Yes but the different clearml instances must be isolated somehow. I mean that in other databases like postgresql, you can define multiple schemas or even different databases. What about elasticsearch? I'm not experienced with it.
Is there any other environment variable related to elastic except from the ones defined in the compose?(CLEARML_ELASTIC_SERVICE_HOST, CLEARML_ELASTIC_SERVICE_PORT, CLEARML_ELASTIC_SERVICE_PASSWORD)

  
  
Posted 9 months ago

ok I will think about. Thanks again

  
  
Posted 9 months ago

Hi @<1582904426778071040:profile|SteepBat69> , that's an interesting question. In theory I think it should be possible

  
  
Posted 9 months ago

Yes i know i can do that, but if all APIservers have access to the same data, then every user will be able to see the experiments of all servers.
Am i missing something here?

  
  
Posted 9 months ago

Yes, that's why I am trying to find alternatives. I know i can't have one tenant with role-based access and I was thinking if there was another way to implement a similar functionality

  
  
Posted 9 months ago

Yes I am trying to solve the authorization problem by creating multiple clearML servers but i want to avoid having one elastic search per server.
The users of each clearML server must have access to experiments of that specific server.
Imagine that each clearml server will have users belonging to the same group/team

  
  
Posted 9 months ago

Role based access controls are supported only in Scale/Enterprise versions.

  
  
Posted 9 months ago
424 Views
10 Answers
9 months ago
9 months ago
Tags