We have not performed any upgrade lately. Also, we are using the community server.
Simply running a task on ClearML is very slow at the moment.
Is there anything specific I can further test from my side?
Hi, the above snippet is again taking minutes to run 😞 ... A few days ago it seemed snappier, but not anymore. Anything I can try on my side?
Hi @<1534344465790013440:profile|UnsightlyAnt34> , I'm not sure, but I think there's some network issue going on since I've seen report of gateway errors happening here and there
I was able to resolve the issue. I am currently using clearml on wsl2 and my machine is connected to a vpn that allows me to connect on to the clearml instance hosted on AWS. You were right it was a network issue, I was able to resolve it by modifying my /etc/resolv.conf
file.
I am having the same problem on both the self hosted and free community clearml.
Hi @<1534344462161940480:profile|QuaintSeal61> , have you upgraded to a new version? Are you self hosted or using the community server? Also, can you elaborate on which part of it is slow? 🙂
@<1534344465790013440:profile|UnsightlyAnt34> we've identified this issue and resolved it, it was indeed related to the network ingress 😞
Hi John, thanks for the quick reply.
Portugal, and this is happening for different users (me and Filipe), from different networks,
Hi,
Using ClearML Free plan, this simple sample code takes 5min to run:
from clearml import Task
print('creating task')
task_ = Task.init(project_name='test_dev', task_name='task')
print('hello world')
For this test I am not running anything else
Hi @<1534344465790013440:profile|UnsightlyAnt34> , what country are you running the code from? Maybe @<1523701087100473344:profile|SuccessfulKoala55> might have some insight?
@<1523701168822292480:profile|ExuberantBat52> if you experienced it on both the self hosted and the community servers, might it be related to network issues?
Hi @<1523701087100473344:profile|SuccessfulKoala55> , I am still facing the same problem, takes minutes to run that sample script on free clearml. I have tested it in several networks :(. If I run it in a Agent in an Azure VM (West Europe), it is as fast as expected.
That's what I was thinking. But I am still having issues on the self hosted version. I think it may be an unrelated issue though. I will do some debugging and report back.
On a separate note, does clearml have a set of acceptance tests that you usually go through before a release?