And actually the problem here is round doesn't work before tolist

AppetizingMouse58 Thanks so much!, Could u tell why does this happen? If it happen next time , is there any other solution?

Can i backup my experiments?

AppetizingMouse58

AppetizingMouse58 After i modify the max_buckets to 200000000000, I cannot see any information in web ui. Here is the log:

SuccessfulKoala55
I have manually control the number of data under 800K, because i found the budget would be 0 if len(series_sizes) = 1, https://github.com/allegroai/trains/blob/master/trains/utilities/plotly_reporter.py#L101

Ok , i will start a new experiment to see if the error will be still there? Sorry i dont really get how to show the trains-apiserver log

Hi SuccessfulKoala55 I find why the ES version is not upgraded to 7.16, because the script provided here is 7.6 https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_es7_migration

AppetizingMouse58 Ok , i see, thanks!

HI: AgitatedDove14
2. I mean if my server break down, and i start a new server in another machine, can i migrate my backup experiments to the new server?
3. Not only change the info in web ui, can i connect to the old experiment , and report a new graph to that?

https://stackoverflow.com/questions/28287261/connection-timeout-with-elasticsearch
How to set the timeout?

SuccessfulKoala55 I get all the trains server experiments record in new clearml server. Maybe it's due to vm.max_map_count or access of /opt/clearml, or the failed ES upgrade...

But without navigation bar, it's quiet inconvenient 😢 ...Do you known why? SuccessfulKoala55

The log is produced after i go to the web ui and reproduce the error of "Failed to get Plot Charts"

AppetizingMouse58 Great, Thanks so much! You have done a great work.
Another question, how to configure elasticsearch to run as a cluster with 2 or more nodes on the same or different machine 😅

I want to use clearml, the old one's mongo is 3.65...

AgitatedDove14 OK, i see, thanks so much!

SuccessfulKoala55
Do you mean even if the json is so large, if i use Logger.report_scatter2d() , it wont cause TransportError(429, 'circuit_breaking_exception ?

AppetizingMouse58 Ok, this is the full log, here seems a error:

AppetizingMouse58 here:

Also, from the log, I see your ES version is 5.6:

at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.16.jar:5.6.16]

and I'm not sure why - in v0.16, the ES version is already 7.6, which makes me think your original version was v0.15, also in the new docker-compose the version is 7.16, so maybe you are not using the updated docker-compose file?

It confused me too, but ES dir in my 0.16 version is /opt/trains/data/elastic_7

I found plotly dash cannot be exported to html file, so it may cannot be used here

Maybe plotly dash can help, is there is any other solution?

SuccessfulKoala55 Ok, thanks, i will have a try.

GrumpyPenguin23 Ok, i will open a request now!

Cam you perhaps send the docker-compose in the current server? (已編輯)

Sorry I don't undestand what' your meaning...

CostlyOstrich36 I'm afraid even i change to ClearML, the problem still exists...

I know why, because i used the old apiserver.conf, i set fixed_users. After log in http://xxx:8080/login , it's ok now

Sorry , my poor english, it means upgrade by script.
Good news is it's fine now, I try to upgrade ES (although it fails), and i try to go through the necessary steps in this: https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_linux_mac