Sorry, its actually
task.update_requirements(["."]) 

AgitatedDove14 So I’ll just replace task = clearml.Task.get_task(clearml.config.get_remote_task_id()) with Task.init() and wait for your fix 🙂

Ok so it seems that the single quote is the reason, using double quotes works

edited the aws_auto_scaler.py, actually I think it’s just a typo, I just need to double the brackets

Ho I see, I think we are now touching a very important point:
I thought that torch wheels already included cuda/cudnn libraries, so you don't need to care about the system cuda/cudnn version because eventually only the cuda/cudnn libraries extracted from the torch wheels were used. Is this correct? If not, then does that mean that one should use conda to install the correct cuda/cudnn cudatoolkit?

and just run the same code I run production

Sorry, I was actually able to fix it (using 1.1.3) not sure what was the problem 😄

Awesome, thanks WackyRabbit7 , AgitatedDove14 !

Hi SuccessfulKoala55 , How can I now if I log in in this free access mode? I assume it is since in the login page I only see login field, not password field

Awesome, huge thanks to the team!

Thanks a lot for the solution SuccessfulKoala55 ! I’ll try that if the solution “delete old bucket, wait for its name to be available, recreate it with the other aws account, transfer the data back” fails

yes 🙂

CostlyOstrich36 , this also happens with clearml-agent 1.1.1 on a aws instance…

I see 3 agents in the "Workers" tab

so most likely one hard requirement installs the version 2 of pyjwt while setting up the experiment

I am trying to upload an artifact during the execution

I mean, inside a parent, do not show the project [parent] if there is nothing inside

and then call task.connect_configuration probably

But according the the example, this syntax should be supported right?

What is this cleanup service? where is it available?

I added the pass_hashed and restarted the server, still get the same problem

So the problem comes when I do
my_task.output_uri = " s3://my-bucket , trains in the background checks if it has access to this bucket and it is not able to find/ read the creds

Hi AgitatedDove14 , sorry somehow this message got lost 😄
clearml version is the latest at the time, 1.7.1 Yes, I always see the "model uploaded completed" for such stuck tasks I am using python 3.8.10

This is what I get, when I am connected and when I am logged out (by clearing cache/cookies)

AgitatedDove14 According to the dependency order you shared, the original message of this thread isn't solved: the agent mentionned used output from nvcc (2) before checking the nvidia driver version (1)

Yes it would be very valuable to be able to tweak that param, currently it's quite annoying because it's set to 30 mins, so when a worker is killed by the autoscaler, I have to wait 30 mins before the autoscaler spins up a new machine because the autoscaler thinks there is already enough agents available, while in reality the agent is down

but it is set here, right?

AgitatedDove14 Yes that might work, also the first one (with conda) might work as well, I will give it a try, thanks!

They indeed do auto-rotate when you limit the size of the logs

Hi AgitatedDove14 , coming by after a few experiments this morning:
Indeed torch 1.3.1 does not support cuda, I tried with 1.7.0 and it worked, BUT trains was not able to pick the right wheel when I updated the torch req from 1.3.1 to 1.7.0: It downloaded wheel for cuda version 101. But in the experiment log, the agent correctly reported the cuda version (111). I then replaced the torch==1.7.0 with the direct https link to the torch wheel for cuda 110, and that worked (I also tried specifyin...