I tried as below, it can works fine:
copy(scp) /opt/trains to target machine upgrade mongo by this: https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_mongo44_migration/ mv /opt/trains /opt/clearml Increase vm.max_map_count ;Grant access to the Dockers; get lastest docker-compose.yml and pull/upBut the web ui dashboard is still weird, anyway it can work now

That's ok, i have just start the server. Could you tell me how to delete this index?

AppetizingMouse58

Also, from the log, I see your ES version is 5.6:

at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.16.jar:5.6.16]

and I'm not sure why - in v0.16, the ES version is already 7.6, which makes me think your original version was v0.15, also in the new docker-compose the version is 7.16, so maybe you are not using the updated docker-compose file?

It confused me too, but ES dir in my 0.16 version is /opt/trains/data/elastic_7

But without navigation bar, it's quiet inconvenient 😢 ...Do you known why? SuccessfulKoala55

SuccessfulKoala55 I get all the trains server experiments record in new clearml server. Maybe it's due to vm.max_map_count or access of /opt/clearml, or the failed ES upgrade...

Hi SuccessfulKoala55 I have run this in step 2

This is the second command

SuccessfulKoala55 Thanks, If restarting server wont stop running experiments, then what i say is not necessary !

AppetizingMouse58 Ok, this is the full log, here seems a error:

And actually the problem here is round doesn't work before tolist

SuccessfulKoala55 It seems ok

I know why, because i used the old apiserver.conf, i set fixed_users. After log in http://xxx:8080/login , it's ok now

Ok , i will start a new experiment to see if the error will be still there? Sorry i dont really get how to show the trains-apiserver log

AppetizingMouse58 Great, Thanks so much! You have done a great work.
Another question, how to configure elasticsearch to run as a cluster with 2 or more nodes on the same or different machine 😅

Can i backup my experiments?

AppetizingMouse58 Ok , i see, thanks!

SuccessfulKoala55 Thanks for your reply, How can i fix this?

SuccessfulKoala55
Do you mean even if the json is so large, if i use Logger.report_scatter2d() , it wont cause TransportError(429, 'circuit_breaking_exception ?

May be its due to my curl?

AppetizingMouse58 Thanks so much!, Could u tell why does this happen? If it happen next time , is there any other solution?

Hi GrumpyPenguin23 ,
Thanks your reply.
In 2, dose that mean i should develop another tool to parse the json standalone? 3 is a good solutionAnd i think what i actually need is the solution to integrate application based server,such as plotly dash or bokeh server in web ui?

GrumpyPenguin23
It seems the solution 3 is the fastest way, and i can reuse my code easily. It works now~ That's quite interesting, but the learning curve seems a bit steep for me.Do you have any material or suggestion for learning that?

I found plotly dash cannot be exported to html file, so it may cannot be used here

GrumpyPenguin23 Ok, i will open a request now!