Okay, makes sense. So there is no copying of the data to the pod, it is simply references via the EFS

So this is an additional config file with enterprise? Is this new config file deployable via helm charts?

Also, how do I associate that new queue with a worker?

Just curious, if https://github.com/allegroai/clearml-helm-charts/blob/19a6785a03b780c2d22da1e79bcd69ac9ffcd839/charts/clearml-agent/values.yaml#L50 is a value I can set, where is it used? It would be great if it overrides the Dataset.get embedded url parsed from my clearml conf file

that is the containerinit logs from k8glueagent

I'm getting import errors when I do, from project.utils import tool all from the working directory of project

When I exec into the pod, it says I need sudo, but wondering if extra_docker_shell_script is executed as sudo already?

I don't know how to get past this? My k8 pods shouldn't need to reach out to the public file server URL.

So I'd create the queue in the UI, then update the helm yaml as above, and install? How would I add a 3rd queue?

Basically, when I have the clearml sdk imported into my python module and I Task.init I'm getting that error only when running the experiment in the task pod.

Does the file on the EFS get downloaded to the k8 pod local volume?

` SysPath: ['/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi/imagery/models/training', '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py/sfi', '/home/npuser/.clearml/venvs-builds/3.7/task_repository/commons-imagery-models-py', '/usr/lib64/python37.zip', '/usr/lib64/python3.7', '/usr/lib64/python3.7/lib-dynload', '/home/npuser/.clearml/venvs-builds/3.7/lib64/python3.7/site-packages', '/home/npuser/.clearml/venvs-builds/3.7/l...

On a somewhat related note to k8s, do you know where I can change this host name? I got this error when my task is fetching a dataset.
2022-09-23 15:09:45,318 - clearml.storage - ERROR - Could not download

Not yet AgitatedDove14 Perhaps we can pair on this Monday.

SuccessfulKoala55 It looks like it should eval to True?

I think if I use the local service URL this problem is fixed

Seems like its just missing the brackets

err maybe not, I dont know where its being fetched

Figured this out, the value is parsed from my local clearml.conf file

Gotcha, and the agent default runtime mode is docker correct? So I could install all my system dependencies in my own docker image?

Does the clearml module parse the python packages? If I'm using a private pypi artifact server, would I set the PIP_INDEX_URL on the workers so they could retrieve those packages when that experiment is cloned and re-ran?

Also what is the purpose of the aws block in the clearml.conf? Where are those values used?

I guess I'm confused on venv mode vs docker mode. It seems like I'm passing in my own docker image which is then used at run time?

How does a task specify which docker image it needs?

Wouldnt that be docker mode?

Yep got it, I was under the impression I could set those values in the UI but I now see they are parsed from my local workstation

Ah I see now

Yea that is a similar bug, needs the same fix

Thanks that worked, I had to set the AWS_PROFILE as well

It will then parse the above information from my local workstation?