Reputation
Badges 1
37 × Eureka!And i try to upgrade elastic by your script, but it cannot success, the log shows :
Hi SuccessfulKoala55 I have run this in step 2
Sorry , my poor english, it means upgrade by script.
Good news is it's fine now, I try to upgrade ES (although it fails), and i try to go through the necessary steps in this: https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_linux_mac
Also, from the log, I see your ES version is 5.6:
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.16.jar:5.6.16]
and I'm not sure why - in v0.16, the ES version is already 7.6, which makes me think your original version was v0.15, also in the new docker-compose the version is 7.16, so maybe you are not using the updated docker-compose file?
It confused me too, but ES dir in my 0.16 version is /opt/trains/data/elastic_7
Hi SuccessfulKoala55 I find why the ES version is not upgraded to 7.16, because the script provided here is 7.6 https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_es7_migration
SuccessfulKoala55 I get all the trains server experiments record in new clearml server. Maybe it's due to vm.max_map_count or access of /opt/clearml, or the failed ES upgrade...
OK i will try again, this time i won't upgrade ES, But do increase vm.max_map_count and open access of /opt/clearm. Why i do again is i found the web ui is fine in deployed machine, but it's weird in my local machine:
But when i click view_all, i can see all the experiments
Cam you perhaps send the docker-compose in the current server? (已編輯)
Sorry I don't undestand what' your meaning...
I want to use clearml, the old one's mongo is 3.65...
Do you mean i send the old one to you?
I tried as below, it can works fine:
copy(scp) /opt/trains to target machine upgrade mongo by this: https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_mongo44_migration/ mv /opt/trains /opt/clearml Increase vm.max_map_count
;Grant access to the Dockers; get lastest docker-compose.yml and pull/upBut the web ui dashboard is still weird, anyway it can work now
I know why, because i used the old apiserver.conf, i set fixed_users. After log in http://xxx:8080/login , it's ok now
But without navigation bar, it's quiet inconvenient 😢 ...Do you known why? SuccessfulKoala55
@<1523701205467926528:profile|AgitatedDove14>
Hi, after rounding down numbers, the plot size decrease to 300Kb from 11M.This really works, thanks!
here is my old docker-compose
My docker-compose is lateset, download by this:curl
https://raw.githubusercontent.com/allegroai/clearml-server/master/docker/docker-compose.yml -o /opt/clearml/docker-compose.yml
Hi SuccessfulKoala55 :
I have make sure that all my data are roud to 4, but i still found my plotly data json is so large. And after checking the json ,i found there are many data with many digits, maybe those are info of plotly?
Here is my code:
` from plotly.subplots import make_subplots
import plotly.graph_objects as go
def draw_pr(self,precisions,recalls,score,distance,dataset):
score = np.round(score,4)
for i in range(4):
pre = np.around(precisions[i], 4)
recall...
SuccessfulKoala55
Even if with Logger.report_scatter2d()
the result is still so large ,and i found where the digits change: https://github.com/allegroai/trains/blob/master/trains/utilities/plotly_reporter.py#L122Tolist
will change the digits , but i haven't figure out why.
SuccessfulKoala55
I have manually control the number of data under 800K, because i found the budget would be 0 if len(series_sizes)
= 1, https://github.com/allegroai/trains/blob/master/trains/utilities/plotly_reporter.py#L101
SuccessfulKoala55 And i try to create ~/trains.conf with verify_certificate = False, but i still cannot init task, it seems doesn't work for the version i'm using.
SuccessfulKoala55 It seems ok
CostlyOstrich36 I'm afraid even i change to ClearML, the problem still exists...