ERROR: No matching distribution found for pandas==1.2.4 (from -r /tmp/cached-reqs1i9hihyv.txt (line 14))

I will investigate a bit more and then check if I can recover

thank you for your time and support, I appreciate it!

it woudl be the same with a docker container and -v then 🙂 np

Seems like this is not the best solution:

why not?

yea the api server configuration also went away

but the PV seems to be just a path to the labeled node

no that's for sure not

so if the node went down and then some other node came up, the data is lost

seems like I miss configured something of course. /secrets is mounted to the agent but not to the pod template 🙂

I have another problem at the moment (some packages that are not inside the cache seem to have be missing and then everything fails)

at the end it's just another env var

Could it be it was never allocated to begin with ?

what do you mean?

yea

actually I should rephrase my question

AgitatedDove14 so basically I am using my own docker image with all of our internal dependencies already installed, including python packages. Some of these packages (e.g green-common ) should be loaded automatically from the dockerimage.

I thought that maybe the virtualenv should https://stackoverflow.com/questions/12079607/make-virtualenv-inherit-specific-packages-from-your-global-site-packages --system-site-packages https://stackoverflow.com/questions/12079607/make-virtualenv-...

Do you mind if I send it in a private chat?

That's a cool idea. Then you pass the tolerations definition through a different pod template?

ERROR: Could not find a version that satisfies the requirement pandas==1.2.4 (from -r /tmp/cached-reqs1i9hihyv.txt (line 14)) (from versions: 0.1, 0.2b0, 0.2b1, 0.2, 0.3.0b0, 0.3.0b2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0rc1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0rc1, 0.8.0rc2, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0rc1, 0.19.0, 0.19...

I see that there's a system_site_packages: false,

oh yea, that is true regarding mitm attacks

For example, the attached package is being skipped automatically but it is needed while running the actual script

I hope so

You can either use the StrictHostKeyChecking=no or generate a known_hosts file. I don't know about other options

Now I suspect what happened is it stayed on another node, and your k8s never took care of that

that's an interesting theory

Yes. Sorry, I have used the wrong backup for some reason

you can actually inject a known_hosts file to your docker container/k8s pod through a volume

I don't know, I only see the error

I tried to add it to post_packages and run it locally, for some reason it didn't install googlecloud-storage. However it is possible that I have an old clearml pip package installed

yea, it's working 🙂