@<1585441130525233152:profile|TrickyGoose45> that's part of the server's health checks (i.e. verifying the databases are reachable) - that's totally normal
In any case @<1523701087100473344:profile|SuccessfulKoala55> @<1523701070390366208:profile|CostlyOstrich36> - is it possible to customize the database names that clearml is using or are they hard-coded in too many places to backend and auth?
@<1585441130525233152:profile|TrickyGoose45> sorry, I missed this.
This is actually configurable (although I think it was never changed in any deployment I'm aware of). In the hosts section of the apiserver deployment, the default setting is:
mongo {
backend {
host: "
"
}
auth {
host: "
"
}
}
If you change the connection strings to point to a different DB, it should work
Because running it as docker compose would imply running it on a VM. Running production stuff on a VM is not acceptable since we don't have the capacity (nor desire) to keeps VMs patched, manage redundancy (while patching and rebooting), manage secure remote access, etc.
Hence we only deploy to native managed Azure services (App service, ACA, etc.).
Plus following the principle of dependency injection I'd rather make external service dependencies explicit. For example we already know how to run redis instances, manage them, monitor them, have appropriate policies in place etc. so hiding those services somewhere in a docker-compose deployment is just not beneficial at all.
I'm aware that ClearML also comes with a helm chart to deploy on k8s but that also tries to spin up it's own dependent backend services and as such doesn't solve the problem.
Hi @<1585441130525233152:profile|TrickyGoose45> , you mean that the mongodb you set up is not part of the docker compose?
we are not using docker compose. We are deploying in Azure with each database as a standalone service
@<1576381444509405184:profile|ManiacalLizard2> , why not run it as docker compose?
Also it's a little bit disappointing how much reverse engineering (i.e. reading the code) one has to do to find out what can actually be configured. I understand that writing docs is a chore and nobody likes to do it but it would be nice if this could be improved - especially when it comes to configuring the external services
Since CosmosDB isn't 100% Mongo feature complete (even though the API is but Cosmos lacks full text indices) I have now spun up an actual Mongo service.
I can see auth and backend databases now, but also a seemingly random test one that contains "queue" and "user" collection. Not sure where that is coming from.
@<1523701070390366208:profile|CostlyOstrich36>