@<1564060248291938304:profile|AmusedParrot89> - let me check this and get back to you

@<1558986867771183104:profile|ShakyKangaroo32> - if you check the api section in your client-side clearml.conf, the value for files_server there should be the same one that you set in the .env file on the server. Can you check that they are indeed the same?
If they are the same - can you please send me the output of the following command in the server:
sudo docker logs -n30 async_delete

Thanks DepressedChimpanzee34 . We'll update if/when it is supported.

@<1558986867771183104:profile|ShakyKangaroo32> - are you running the server using docker-compose?
If so - please add the following to the .env file in the same directory as the compose:
CLEARML_FILES_HOST=http:// <YOUR IP or HOSTNAME> :8081
Then restart the service by running sudo docker-compose up -d
Please update if this worked, or if you have any questions/issues

We limit the allowed calls per IP - to make sure the server is not blocked accidentally. We enabled over 1000 calls per minute.

@<1526371986278715392:profile|VivaciousReindeer64> - Yes - please send me the docker-compose and the log of the webserver

@<1566596960691949568:profile|UpsetWalrus59> - please note that if you report the plots as two separate series of the same metric - it should work better

you can re-direct it to a file or just copy with the clipboard

Make sure restart the compose ( docker-compose up -d )

Hi @<1570583227918192640:profile|FloppySwallow46> . We've update the rate limits. Can you please check if the issue is still occurring?

JitteryCoyote63 - thanks for the info. We are trying to reproduce this, and will update

Hi CooperativeFox72 . Thanks for reporting this issue.
Can you add the following information:
Is this on the hosted community server ( http://app.community.clear.ml ) or a server that you installed? If this is a server the you installed locally - which server version are you running Which browser, and what resolution are you using?Thanks,
Oren.

NastySeahorse61 - can you check if this was updated? What does you usage page say?

Hi EnviousStarfish54 .
I'm trying to make sure I understand the scenario. What I undestood is that you add a custom column (metric) to the experiments table, sort by it and then refresh with F5. I wasn't able to reproduce this on the Demo site ( https://demoapp.trains.allegro.ai/projects/*/experiments?columns=selected&columns=type&columns=name&columns=tags&columns=status&columns=project.name&columns=users&columns=started&columns=last_update&columns=last_iteration&columns=m.5451af93e0bf68a4ab...

AbruptWorm50 - just to make sure there is no misunderstanding - the last image you sent is on the "training" queue and not on the "services" queue. Are there free agents running on that queue?

Looking at the 2nd image you sent, I see that in addition to "services" queue, you also have queues called "training" and "training*_2" - and the experiments you circled is in the "training" queue. In that image - there are no experiments in the services queue.
If you press on the "services" queue (like you did in the first image) you can view the experiments in the queue and the workers. Can you check if there is a situation where there are tasks pending in that queue while one of the work...

EnviousStarfish54 - Got it!
I'll look into it and update you. Thanks for reporting!

sudo docker logs clearml-webserver

Do you still get the new version notification?

No problem. Glad it's working now

ShinyRabbit94 - The github issue reported by SuperiorPanda77 is a display issue - meaning the files are deleted, and only the "undefined" display is incorrect. This should be fixed in the next release (1.4).
In your case the files are not deleted from the filesystem - so I assume this is a different issue. Can you provide more info so that I can try to reproduce:
How are the artifacts created? Are the links in the popup describing the un-deleted files correct? Can you add a screenshot?

@<1526371986278715392:profile|VivaciousReindeer64> - I'll check it on my env

@<1564060248291938304:profile|AmusedParrot89> - I see the logic in displaying the last iteration per metric in the compare screen. We will need to think if this won't cause any other issues.
In the mean time - may I ask you to open a github issue - so it will be easier to track?

Hi JealousSealion33 . We are planning on releasing a clearml k8s package very soon. I hope this is something that you will be able to use. If not, could you please elaborate on the issue you are facing and share the error you are receiving?

HandsomeGiraffe70 - please try to update to clearml 1.6.1 - this should solve the issue. Please update us if this worked

Hi DrainedHippopotamus42 - I just answered in Slack (didn't see the post here...). We found the issue and it will be resolved in the next saas update. For the time being I added more storage to your workspace. Once you run a task that reports metrics - the problem should disappear.
Please let me know if this works for you

NutritiousBear41 - can you go to localhost:8080/version.json (or change to the relevant address of your web app) and see what it outputs?

Hi JitteryCoyote63 . Just to make sure we fully understand the scenario - you had a project with several experiments, some of which were archived. Then you went to archive and selected all of the experiments and deleted them using the bulk delete operation. And then you saw that the project was empty even when you were not in archive mode. Is that correct?

The artifacts (debug samples, models, artifacts) deletion is currently performed done from the browser, after the task object is deleted from the backend. So it might take time for all the artifacts to be deleted - please make sure not to close your browser.
Note that only artifacts saved on fileserver are deleted - exteranal artifacts (in the cloud or on a local filesystem) are not deleted.
We are working on moving the deletion to the server side, to avoid these kind of issues. Should be in ...

@<1566596960691949568:profile|UpsetWalrus59> - if you could also paste the payload and response of the call to events.get_multi_task_plots - from the network tab of the browser's dev-tools (F12), this might also help understand