Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, Is There Any Document About Migration Clearml-Server. Currently, I Have Clearml-Server Running On Servera But I Want To Move All Data (Including Artifacts, Task, Dataset) From Servera To Serverb.

Hi, is there any document about migration Clearml-server.
Currently, I have Clearml-server running on serverA but I want to move all data (including artifacts, Task, Dataset) from ServerA to ServerB.

  
  
Posted one year ago
Votes Newest

Answers 16


Hello, after did the steps you mentioned https://clearml.slack.com/archives/CTK20V944/p1659702067809619?thread_ts=1659694970.919069&cid=CTK20V944
The server is now can start properly but Clearml UI doesn’t show any experiments that I cloned from serverA. Any suggestion? thank you!

  
  
Posted one year ago

After I did
docker-compose -f docker-compose.yml down docker-compose -f docker-compose.yml up -d
Then elasticsearch container got this error

ElasticsearchException[failed to bind service]; nested: IOException[failed to test writes in data directory [/usr/share/elasticsearch/data/nodes/0/indices/mQ-x_DoZQ-iZ7OfIWGZ72g/_state] write permission is required]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes/0/indices/mQ-x_DoZQ-iZ7OfIWGZ72g/_state/.es_temp_file]; clearml-elastic | Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes/0/indices/mQ-x_DoZQ-iZ7OfIWGZ72g/_state/.es_temp_file clearml-elastic | at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) clearml-elastic | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) clearml-elastic | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) clearml-elastic | at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) clearml-elastic | at java.base/java.nio.file.Files.newByteChannel(Files.java:380) clearml-elastic | at java.base/java.nio.file.Files.createFile(Files.java:658) clearml-elastic | at org.elasticsearch.env.NodeEnvironment.tryWriteTempFile(NodeEnvironment.java:1313) clearml-elastic | at org.elasticsearch.env.NodeEnvironment.assertCanWrite(NodeEnvironment.java:1284) clearml-elastic | at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:342) clearml-elastic | at org.elasticsearch.node.Node.<init>(Node.java:427) clearml-elastic | at org.elasticsearch.node.Node.<init>(Node.java:309) clearml-elastic | at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) clearml-elastic | at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) clearml-elastic | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) clearml-elastic | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) clearml-elastic | at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) clearml-elastic | at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) clearml-elastic | at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) clearml-elastic | at org.elasticsearch.cli.Command.main(Command.java:77) clearml-elastic | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) clearml-elastic | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) clearml-elastic | For complete error details, refer to the log at /usr/share/elasticsearch/logs/clearml.log

  
  
Posted one year ago

Nice!

  
  
Posted one year ago

Looks like a permissions issue:
nested: IOException[failed to test writes in data directory [/usr/share/elasticsearch/data/nodes/0/indices/mQ-x_DoZQ-iZ7OfIWGZ72g/_state] write permission is required]; nested

  
  
Posted one year ago

I'm not entirely sure which steps you took and if you missed something. Elastic is complaining about permissions - Maybe you missed one of the steps?

  
  
Posted one year ago

I already did
chmod 777 on /opt/clearml/data or there’s other folders I need to grant the permission

  
  
Posted one year ago

I mean migrating the data from serverA to serverB.
I just replace serverB with ServerA’s /opt/clearml/data .

  
  
Posted one year ago

For example, ServerA stores file at /opt/clearml but ServeB stores at /some_path/clearml

As long as you adjust your docker-compose yaml file, should be just fine

  
  
Posted one year ago

Oh, I just realized that the mondo version between ServerA and ServerB is mismatch.
The problem was resolved by updating the mongo image to 4.0.23 as serverA.

  
  
Posted one year ago

And you have the exact same folder structure / content, and server A/B give a different set of experiments ?
(is serverB empty, meaning no experiments at all?)

  
  
Posted one year ago

Is it ok if the path of ServerA and ServerB is difference.
For example, ServerA stores file at /opt/clearml but ServeB stores at /some_path/clearml

  
  
Posted one year ago

VictoriousPenguin97 I'm assuming the exact same server version ?

  
  
Posted one year ago

Yes, the latest version.

  
  
Posted one year ago

I’ve follow the installation steps that mentioned in this page
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_linux_mac/

Then I replaced /opt/clearml/data of ServerB by ServerA /opt/clearml/data .

  
  
Posted one year ago

VictoriousPenguin97 basically spin down sereverA (this should flush all DBs) then copy /opt/clearml to the new server and spin it with docker-compose. As long as the new server is on the same address as the previous one, everything should work out of the box

  
  
Posted one year ago

Did I migrate the data correctly using the steps I took?

  
  
Posted one year ago
619 Views
16 Answers
one year ago
one year ago
Tags