Thanks CooperativeFox72 , looking into it

OK, cool, let us know 🙂

yep

First, go into the MongoDB docker instance using:
sudo docker exec -it clearml-mongo /bin/bashThen, inside the docker, start the MongoDB CLI using:
mongoThen, enter these two commands:
use auth db.user.createIndex({"name": 1, "comapny": 1})

OK, yes, that clears it up 🙂

It seems the automatic MongoDB migration failed on startup

someone in my company started a training 😥 , will do it after it will finish.. and will update
Thanks you are the best 🙏

I update to the new version 0.16.1 few weeks away and it works using the elastic_upgrade.py

Obviously you have to have the server up when you do that... 🙂

No, there was a problem with the particular version migration. The temporary index creation allowed to this and all subsequent migrations to run successfully. So for now your DB is properly aligned with the latest ClearML and the future upgrades should work fine.

Hi SuccessfulKoala55 ,
I down the server:
[ec2-user@ip-172-31-26-41 ~]$ sudo docker-compose -f /opt/clearml/docker-compose.yml down WARNING: The CLEARML_HOST_IP variable is not set. Defaulting to a blank string. WARNING: The CLEARML_AGENT_GIT_USER variable is not set. Defaulting to a blank string. WARNING: The CLEARML_AGENT_GIT_PASS variable is not set. Defaulting to a blank string. Stopping clearml-webserver ... done Stopping clearml-agent-services ... done Stopping clearml-apiserver ... done Stopping clearml-redis ... done Stopping clearml-fileserver ... done Stopping clearml-mongo ... done Stopping clearml-elastic ... done Removing clearml-webserver ... done Removing clearml-agent-services ... done Removing clearml-apiserver ... done Removing clearml-redis ... done Removing clearml-fileserver ... done Removing clearml-mongo ... done Removing clearml-elastic ... done Removing network clearml_backend Removing network clearml_frontendthen try the commad:
[ec2-user@ip-172-31-26-41 ~]$ sudo docker exec -it clearml-mongo /bin/bash Error: No such container: clearml-mongo
what did I done wrong?

then start the server again and see if you get the errors in the log

trying again 🤞

Than take it down, and up again

I did it and still getting the same error 😥

what it mean? 😁

Hi CooperativeFox72 , there was a typo in the index creation instructions ("comapny" instead of "company"). Please try the following sequence in mongo shell and then starting the apiserver:
use auth db.user.createIndex({"name": 1, "company": 1})

Thanks again!! 🙏
You`r the best 🙂

the index creation:
[ec2-user@ip-172-31-26-41 ~]$ sudo docker exec -it clearml-mongo /bin/bash root@3fc365193ed0:/# mongo MongoDB shell version v3.6.5 connecting to: mongodb://127.0.0.1:27017 MongoDB server version: 3.6.5 Welcome to the MongoDB shell. For interactive help, type "help". For more comprehensive documentation, see Questions? Try the support group `
Server has startup warnings:
2021-01-25T05:58:37.309+0000 I CONTROL [initandlisten]
2021-01-25T05:58:37.309+0000 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2021-01-25T05:58:37.309+0000 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2021-01-25T05:58:37.309+0000 I CONTROL [initandlisten]

use auth
switched to db auth
db.user.createIndex({"name": 1, "comapny": 1})
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 2,
"numIndexesAfter" : 3,
"ok" : 1
}

bye
root@3fc365193ed0:/# exit `

hope this is better

SuccessfulKoala55 and AppetizingMouse58 Thanks you very much!!

I have a future question:
Does this fix should harm in future cleraml-server upgrade?
Or what the best practice to upgrade after doing it?

Is this after you've created the index using the instructions I sent?

does it ok that it looks for files in /opt/trains ? since we move all to /opt/clearml no?
File "/opt/trains/apiserver/mongo/initialize/migration.py"

Anyway, a quick fix could be to create the mongo index that's failing the imgration

Can you share the apiserver logs? Use docker logs clearml-apiserver

Of course, do that while the server is down

🙂

Oh, I'm sorry - how stupid of me...

Are you sure you previously had 0.16.1? From the log it seems you either had an empty database or that you had a Trains Server <0.14.0