Is it possible to do something so that the change of the server address is supported and the pictures are pulled up on the new server from the new server?
Do the links point to a bucket or the fileserver?
AgitatedDove14
yes, IP-based access, with DNS for some reason took 5 times longer and we abandoned it (MD is about our DNS and its settings).
Found a fix! The issue was that the field I wanted changed occurs in task not model as I did in the js script. So I used https://gist.github.com/rajiteh/1680586300d11a9c9e7a to change every occurence just in case.. hopefully I didn't break anything
Is it possible to do something so that the change of the server address is supported and the pictures are pulled up on the new server from the new server?
The link itself (full link) is stored inside the server. Can I assume the access is IP based not host based (i.e. dns) ?
AgitatedDove14 yes, that's right, he's changed
CheerfulGorilla72
yes, IP-based access,
hmm so this is the main downside of using IP based server, the links (debug images, models, artifacts) store the full URL (e.g. http://IP:8081/ http://IP:8081/... ) This means if you switched IP they will no longer work. Any chance to fix the new server to the old IP?
(the other option is somehow edit the DB with the links, I guess doable but quite risky)
CostlyOstrich36
we have a render server and a file server - one machine, unfortunately I'm not so familiar with clearml yet to set it all up separately.
AgitatedDove14
I suspect that it will be difficult to assign the old IP - I will consult with the admin guys.
Architecturally it sounds right to work with the base, though I don't have that much experience. I correctly understand, What then it is necessary to replace the address in a line of a way for a file?
I am facing a similar issue and followed the steps in https://clearml.slack.com/archives/CTK20V944/p1618473431491800?thread_ts=1618471688.490400&cid=CTK20V944 but it didn't seem to work. The script in my case was,db.model.find({uri:{$regex:/^http/}}).forEach(function(e,i) { e.uri = e.uri.replace("http://<oldip>:8081/"," http://<newip>:8081/"); db.model.save(e); });
CheerfulGorilla72 Yes, the link changes are reflected in the UI and artifact downloads worked as intended
when the old server is up - all the pictures in the new server are also opened from the old server, if you click on open at the link address
GorgeousSeagull44
Cool!
Please tell me how to run it, I have never run JS before.
GorgeousSeagull44 I think this should have worked (basically replacing all the links on the mongo DB with the new IP)
Is it possible to do something so that the change of the server address is supported and the pictures are pulled up on the new server from the new server?
CheerfulGorilla72 could it be the server address has changed when migrating ?
No errors. Output of mongo backend script.js :MongoDB shell version v4.4.13 connecting to: Implicit session: session { "id" : UUID("a29b4c5e-0325-40e4-bb13-c9a80123b4e5") } MongoDB server version: 4.4.9
CheerfulGorilla72 Just realized your thread refers more to the debug images while I was referring to saved artifacts. You might find https://github.com/allegroai/clearml-server/issues/83 more useful
About migration - we saved the data archives and copied them to a new server, extracting them to the appropriate folders and setting the necessary rights, and rebuilding the docker image and launching the container
Before all this, we migrated to the new version according to the instructions and everything went well, all the data after the restart was displayed correctly.
And only after that we began the process of switching to new hardware - with a large disk.
GorgeousSeagull44 did you restart docker compose?
Just follow the steps in https://clearml.slack.com/archives/CTK20V944/p1618473431491800?thread_ts=1618471688.490400&cid=CTK20V944 . But use the attached script instead.
docker-compose -f /opt/clearml/docker-compose.yml down docker-compose -f /opt/clearml/docker-compose.yml pull docker-compose -f /opt/clearml/docker-compose.yml up -d