Reputation
Badges 1
59 × Eureka!Woohoo! π
The instructions in the https://superuser.com/questions/278948/clear-cache-for-specific-domain-name-in-chrome/444881#444881 were not accurate, but they brought me close enough.
Here is the exact sequence of operations:
F12 --> Applications tab --> Storage --> Clear site data --> refresh login screen
Thanks everyone for your help!
Where do I see the agent print outs?
I am using an old version. It's a trains server of version 0.16.3.
I can enter my user name but even the button underneath it is blank (see below). Once clicking it, the whole screen is blank as in the 1st image that I sent.
Yes, I am using the trains server. We never took the time to update it to clearml.
The version (according to pip freeze) is 0.16.3.
I clicked Fetch/XHR and got the following (after another reboot)
I did not upgrade anything and did not do docker pull.
I am having a temporary network issue . Will send the output of the β docker inspectβ as soon as I can reconnect to my server.
SuccessfulKoala55 , here is the output of "docker inspect trains-webserver" (attached).
No other error messages but the dashboard screen is blank.
However, there is a breakthrough: I can run the dashboard from Safari (Mac browser). So the problem is only in Chrome.
Here is a snapshot of the blank screen:
Here is the developer tool Network screen capture after refreshing the page and trying to login.
I get an empty list for the 'XHR' filter.
Many errors :white_frowning_face: . Any idea what they mean?
The sequence is unclear then:
I followed the instructions in https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_es7_migration/ .
Stage 5 ("python elastic_upgrade.py") ended successfully.
Then I skipped "Upgrading to ClearML Server v.1.2. or Newer" and went straight to "Completing the Installation".
Did I do wrong? What should I do to fix it?
AgitatedDove14 SuccessfulKoala55 , after I ran elastic_update.py (stage 5 as described above), I saw there was a new folder named data/mongo_4. Doesn't it mean mongodb was already migrated?
Is there any log that maybe details the problem?
AppetizingMouse58 , SuccessfulKoala55 and AgitatedDove14 , after running the ES migration for the 2nd time the problem is solved π . Thank you all for your help! π
Is it ok to restore data/mongo from my backup, and leave all the other files that were created by elastic_upgrade.py (e.g., data/elastic_7) untouched?
What I mean, is: Do I need to run elastic_upgrade.py again, or just the mongo upgrade (clearml-server-1.2.0-migration.py)?
Update: I ran the mongo migration script (clearml-server-1.2.0-migration.py) and now I can see my projects! π
Now there is a new problem: I don't see any of the logs: console, artefacts, scalars, plots.
Can you help?
Yes I've performed the ES migration. The data is in clearml/data/elastic_7.
I will try it and keep you posted. Thanks!
In file docker-compose.yml I replaced all the strings /opt/clearml/data/elastic_7 into /home/orpat/clearml/data/elastic_7.
The upgrade is from /home/orpat/trains/data/elastic into /home/orpat/trains/data/elastic_7. Do you different paths in the log? Where?
I am not sure it matters for the following output, but anyway please note that the clearml dockers are down right now.
sigalr@momo : ~ $ curl -XGET http://localhost:9200/_cat/indices
yellow open queue_metrics_d1bd92a3b039400cbafc60a7a5b1e52b_2022-06 2F6APbQWSvajTZQ5JxXY1Q 1 1 59 0 26.2kb 26.2kb
yellow open events-plot-d1bd92a3b039400cbafc60a7a5b1e52b bZMKKCaKRXCys6VD_9oDDw 1 1 8556 0 4.1mb 4.1mb
yellow open worker_stats_d1bd92a3b039400cbafc60a7a5b1e52b_2022-06 c85DhB...
The clearml dockers are down right now because I started a new ES migration (elastic_upgrade.py). I started it before you contacted me and I don't want to break it now. So I cannot look at the console right now.
It will probably finish 30 hours from now. If the same problems repeat, we will continue this chat then.
I don't get the error any longer and the experiments get deleted as expected. So no complains on my side...