About nodeSelector you are right, one is for the agent pod while the other is used to spawn task pods

# Point to the internal API server hostname APISERVER= `

Path to ServiceAccount token

SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount

Read this Pod's namespace

NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)

Read the ServiceAccount bearer token

TOKEN=$(cat ${SERVICEACCOUNT}/token)

Reference the internal certificate authority (CA)

CACERT=${SERVICEACCOUNT}/ca.crt

Explore the API with TOKEN

curl --cacert ${CACERT} --header "Authorization: Bearer ${TOKEN}" -X GET ${A...

ok let me check this one

additionalConfigs: auth.conf: | auth { # Fixed users login credentials # No other user will be able to login fixed_users { enabled: true pass_hashed: false users: [ { username: "jane" password: "12345678" name: "Jane Doe" }, { username: "john" password: "12345678" name: "John Doe" }, ] } }

ok this is weird, in apiserver we should see call for deletion request. I need to consult with some people because I don’t think this is infra config related.

with that in place k8s should be able to provisione pvc

can you also show output of kubectl get po of the namespace where you installaed clearml?

yes, exactly, agent creates and manages task pod lifecycle

oh yass

kubectl get svc -n clearml ?

did you tried to create a debug pod with a mount using ceph storageclass? you can start from here https://downey.io/notes/dev/ubuntu-sleep-pod-yaml/ then add the pvc and the mount. then you should exec into the pod and try to write a dummy file on the mount; I suspect the problem is there

wohoo

oh now I understand

can you change the path in ALB healthcheck pls?

trying a new one from scratch

yes

btw I think it's related clearml-task usage; did you try to create a pipeline with decorators creating it with clearml-task before?

😄

I;m already testing this patch

but TaskTypes is under Task

pretty strange, I also noticed in example from line 2
from clearml import TaskTypes

same issue