I don't think the problem is setting that variable, I think it has something to do with it but not that obvious... Because it did work for me in the past, since then we docker-compose up/downed a few times, changed some other things etc... Can't figure out what made it get to this point

but remember, it didnt work also with the default one (nvidia/cuda)

(I changed it in the settings)

I guess not many tensorflowers running agents around here if this wasn't brought up already

I assume it has nothing to do with my client version

I ran the wizard liek a milion times

and I'm using the same credentials

No absolutely not. Yes I do have a GOOGLE_APPLICATION_CREDENTIALS environment variable set, but nowhere do we save anything to GCS. The only usage is in the code which reads from BigQuery

How do I get from the node to the task object?

😅

the worst part of debugging this is waiting for the docker to install tensorflow each time over and over again 😞

AgitatedDove14
So I couldn't kill the service agent myself (permission denied, I'm not sudo). What I did is I docker-compose down ed, commented out only the environment variable of GOOGLE_APPLICATION_CREDENTIALS from the clearml services agent service and upped the docker-compose again. I enqueued the Cleanup Service and now it works. Really weird, looks like the setting of GOOGLE_APPLICATION_CREDENTIALS causes an error when set even though I'm 100% is it not used for storag...

nvidia/cuda:10.1-base-ubuntu18.04

Yep, the warnings no longer appear

That was the issue then

🤞

🤔 is the "installed packages" part editable? good to know

Isn't it a bit risky manually changing a package version? what if it won't be compatible with the rest?

and the machine I have is 10.2.

I also tried nvidia/cuda:10.2-base-ubuntu18.04 which is the latest

But I'm naive enough to believe that 10.2 is compatible with 10.1 as it is a minor upgrade

glad I managed to help back in some way

oops m ybad

Okay, I'll try that

🤞

I don't have ifconfig

the path to the JSON file

I'll try to put it again

Trains docs have at no point any mention on what should I do on the AWS interface... So I'm not sure at what point I should encounter this wizard

I'm going to play with it a bit and see if I can figure out how to make it work

Now I remind you that using the same credentials exactly, the auto scaler task could launch instances before

the ones in the export are like you said

yep