@<1558986867771183104:profile|ShakyKangaroo32> - if you check the api section in your client-side clearml.conf, the value for files_server
there should be the same one that you set in the .env
file on the server. Can you check that they are indeed the same?
If they are the same - can you please send me the output of the following command in the server:sudo docker logs -n30 async_delete
@<1574931903440490496:profile|CrookedBear44> - if you logged in with email - you can also login with one of the other options available in the log in page.
To reset your email's password please do the following:
- Go to the login page ( None )
- Press
Continue with Email
- Go to
Log In
tab - Press
Don't remember your password
- Fill your email and press
SEND EMAIL
Please make sure to check your spam folder
Let me know if you need any help.
@<1566596960691949568:profile|UpsetWalrus59> - please note that if you report the plots as two separate series of the same metric - it should work better
@<1523701087100473344:profile|SuccessfulKoala55> - yes, plots are reported every iteration.
@<1564060248291938304:profile|AmusedParrot89> - the plot comparison indeed compares the latest iteration of the experiments. I will see if this can be better indicated somewhere
@<1564060248291938304:profile|AmusedParrot89> - let me check this and get back to you
@<1564060248291938304:profile|AmusedParrot89> - I see the logic in displaying the last iteration per metric in the compare screen. We will need to think if this won't cause any other issues.
In the mean time - may I ask you to open a github issue - so it will be easier to track?
Hi DepressedChimpanzee34 .
Currently supporting plotly animations is not planned for the upcoming versions. I would suggest opening a feature request in github, or add it to the UI change request thread: https://github.com/allegroai/clearml/issues/81 .
Thanks,
Oren.
Hi DepressedChimpanzee34 . Thanks for the info.
Plotly animation is currently not supported in the web app. We're looking into adding this into a future version. Can you describe the use case of animations in your system?
ShinyRabbit94 - The github issue reported by SuperiorPanda77 is a display issue - meaning the files are deleted, and only the "undefined" display is incorrect. This should be fixed in the next release (1.4).
In your case the files are not deleted from the filesystem - so I assume this is a different issue. Can you provide more info so that I can try to reproduce:
How are the artifacts created? Are the links in the popup describing the un-deleted files correct? Can you add a screenshot?
SuperiorPanda77 - thanks for updating. So indeed these may be similar issues. I will re-check this and udpate
@<1558986867771183104:profile|ShakyKangaroo32> - are you running the server using docker-compose?
If so - please add the following to the .env
file in the same directory as the compose:CLEARML_FILES_HOST=http://
<YOUR IP or HOSTNAME>
:8081
Then restart the service by running sudo docker-compose up -d
Please update if this worked, or if you have any questions/issues
Hi @<1526371986278715392:profile|VivaciousReindeer64> - please try this file:
Hi @<1526371986278715392:profile|VivaciousReindeer64> ,
I replicated your environment, and found the following solution:
- In the docker-compose, add the following to the webserver section:
volumes:
- /opt/clearml/config:/mnt/external_files/configs
- In the host machine, create a file
/opt/clearml/config/configuration.json
containing the following:
{
"displayedServerUrls": {
"filesServer": "
"
}
}
- restart the docker-compose: `sudo docker-...
VivaciousReindeer64 - you can try to add to the following to the webserver service environment section:
WEBSERVER__fileBaseUrl= https://192.168.1.145:8081
Hi EnviousStarfish54 .
I'm trying to make sure I understand the scenario. What I undestood is that you add a custom column (metric) to the experiments table, sort by it and then refresh with F5. I wasn't able to reproduce this on the Demo site ( https://demoapp.trains.allegro.ai/projects/*/experiments?columns=selected&columns=type&columns=name&columns=tags&columns=status&columns=project.name&columns=users&columns=started&columns=last_update&columns=last_iteration&columns=m.5451af93e0bf68a4ab...
@<1523701790044852224:profile|FreshParrot56> - Indeed a bug. We will fix this in the upcoming version - due in a few weeks
EnviousStarfish54 - Got it!
I'll look into it and update you. Thanks for reporting!
@<1526371986278715392:profile|VivaciousReindeer64> - Yes - please send me the docker-compose and the log of the webserver
Hi ThoughtfulGorilla90 - when did you perform the deletion. It might take up to 24 hours for the application to get the updated size
Hi ThoughtfulGorilla90 , how did you delete the experiments/models - was it from the web app or using the SDK?
The artifacts (debug samples, models, artifacts) deletion is currently performed done from the browser, after the task object is deleted from the backend. So it might take time for all the artifacts to be deleted - please make sure not to close your browser.
Note that only artifacts saved on fileserver are deleted - exteranal artifacts (in the cloud or on a local filesystem) are not deleted.
We are working on moving the deletion to the server side, to avoid these kind of issues. Should be in ...
sudo docker logs clearml-webserver
Make sure restart the compose ( docker-compose up -d
)
you can re-direct it to a file or just copy with the clipboard
@<1526371986278715392:profile|VivaciousReindeer64> - please check the following:
- What do you get if you go to http//192.168.1.145:8080/configuration.json?
- Can you check the log of the webserver docker (using
sudo docker logs clearml-webserver
) - especially the beginning? Does it say anything about the fileBaseUrl?
MotionlessMonkey27 - was this resolved?
NastySeahorse61 - can you check if this was updated? What does you usage page say?
@<1526371986278715392:profile|VivaciousReindeer64> - I'll check it on my env
@<1526371986278715392:profile|VivaciousReindeer64> - I added this and it worked:
- WEBSERVER__fileBaseUrl="
"
Maybe the quotes are required.