Ok, seems like the problem is solved.
These uncommited changes were already applied to the local branch, but the git apply error wasn't very informative.

Thanks!

I got the same issue as well last night.

You are right Idan,
I consulted our Private ClearML channel.
you cannot insert these environment variables any other place,
only in init script.
Here is the full quote:

the environment setting you added to your vault is only applied inside the instance when the agent starts running there, not as part of the command that starts the instance.
The most common DevOps practice for having these kind of variables in the init script but not completely exposed to the naked eye is by adding something like

export MY_ENV_VAR=$(echo '<base64-encoded secret>' | base64 --decode)

to the init script （編集済み)

1.2.0

That helps a lot!
Thanks Martin.
Although I didn't understand why you mentioned torch in my case?
Since I don't use it directly, I guess somewhere along the way multiprocessing does get activated (in HPO)
I would guess it relates to parallelization of Tasks execution of the HyperParameterOptimizer class?

+1 for this 🙂

Yes, Thank you!

GrumpySeaurchin29 Thank you!

Using an autoscaler service(on 24/7 EC2 machine)
that triggers EC2 workers (with an AMI I saved prior to activation)

Hope that helps

The thing is this.
My optimizer works a bit different.
my "optimized task" is actually a task that gets a specific
Hyper parameters and then enqueus more tasks (each one on different object)

My task runs just fine.
But no GPU.
(When it demands GPU it collapses).
Looking at the VM features on GCP UI it seems no GPU was defined for the VM.

Wouldn't this work?
https://gist.github.com/pemagrg1/f959c19ec18fee3ce2ff9b3b86b67c16

I am not a staff member. But it seems like something quite trivial with not much effort.
if you can avoid conda and don't need the c++ dependencies that conda takes care of
(and since you can convert to pip fomat , you probably can).