If you have ALB you will just need to add some annotations on ingress rules depending on your setup. btw for now , since you already have everything in place, I suggest to just add values to /etc/hosts and see if it works

in some second it should became green

Hi @<1523701717097517056:profile|ScantMoth28> , disabling atm we are not supporting Istio but I’m more than willing to look at a proposal like yours. Let’s discuss this on a new issue on github pls so we can keep track of it and find a good way to implement. thanks

kubectl get ingress -A

other wise yes, if this is not an option, you can also mount what is already existing so pls open an issue in new repo helm chart and we can find a solution

yep but this is not how it should work with inpod

btw a good practice is to keep infrastructural stuff decoupled from applications. What about using https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner ? After applying that chart you can simply use the generated storage class; wdyt?

I guess the message may be mistaken. Pls share kubectl get svc of the namespace you installed clearml

I don’t think it’s related how agent talk with apiserver or fileserver. It’s more related the fact agent pod internal kubectl cannot contact kubernetes apiserver

from https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/

but it;s just a quick guess, not sure if i’m right

you can workaround the issue mlunting the kubeconfig but I guess the issue is someway to be investigated

just my two cents

I think we can find a solution pretty quickly after some checks. Can you pls open an issue on new helm chart repo so I can take care of it in some day?

exactly

accessing apiserver from a pod doesn’t require kubeconfig

this means network issues at some level

I guess yes but honestly I’m not sure you will get the right results

agent is running the command inside the pod like you did execing into pod and manually launching it. If one is returning 127 while manually you are ok it looks to me the command issued is not the same. what is chart version you are using?

Or do you want to dinamically mount directly an nfs endpoint? (I understood you need this one)

probably
is not accessible from your pod

try this into the pod

can you pls check the content of file /root/clearml.conf inside agent pod. I would like to understand how config file looks, ty.

Just a quick suggestion since I have some more insight on the situation. Maybe you can look at Velero, it should be able to migrate data. If not you can simply create a new fresh install, scale everything to zero, then create some debug pod mounting old and new pvc and copy data between the two. More complex to say it than do it.

btw it's on the filesystem

ok, for mayor version upgrade my suggestion is to backup the data somewhere and do a clean install after removing the pvc/pv

pls fix also fileServerUrlReference: anf webServerUrlReference:

pretty strange, I also noticed in example from line 2
from clearml import TaskTypes

how do you point tasks to git repo?

Sure, OddShrimp85 , until you need to specifically bind any pod to a node, nodeSelector is not needed. In fact, the new chart will leave to k8s the right to share the load on the worker nodes. About pvc you simply need to declare the Storageclass at k8s level so it can take care of creating the PVC too. How many workers do you have in your setup?