Got it thanks TimelyPenguin76 .

AgitatedDove14 - this was an interesting one. I think I have found the issue, but verifying the fix as of now.

One of the devs was using shutil.copy2 to copy parts of dataset to a temporary directory in a with block - something like:

with TemporaryDirectory(dir=temp_dir) as certificates_directory: for file in test_paths: shutil.copy2(f"{dataset_local}/{file}", f"{certificates_directory}/file")
My suspicion is since copy2 copies with full data and symlin...

` if project_name is None and Task.current_task() is not None:
project_name = Task.current_task().get_project_name()

if project_name is None and not Task.running_locally():
task = Task.init()
project_name = task.get_project_name() `

Would be good to have frequentish releases if possible 🙂

AgitatedDove14 - worked with mutable copy! So was definitely related to the symlinks in some form

Yeah concerns make sense.

The underlying root issue is unnecessary models being added or at least what I think are unnecessary and even happening when you load a model to test.

Do people use ClearML with huggingface transformers? The code is std transformers code.

Will create an issue.

From the code - it’s supposed to not cache if task override is different? I also have task_override that adds a version which changes each run

Yeah sure

Beyond this have the UI running, have to start playing with it. Any suggestions for agents with k8s?

another place I was using was to see if i am in a pipeline task

How can I ensure that additional tasks aren’t created for a notebook unless I really want to?

workflow

don’t know what’s happening there

Basic question - i am running clearml agent in a ubuntu ec2 machine. Does it use docker by default? I thought it uses docker only if I add the --docker flag?

Is there some doc or relevant code on exactly what's happening? The behaviour has been random

Ref of dvc doing about the same - https://github.com/iterative/dvc/blob/master/dvc/fs/s3.py#L127-L134

Yeah got it

Only allowed to have ssh key, not username-password

AlertBlackbird30 - got it running. Few comments:

Nodeport is set by default despite being parameter in values.yml. For example:` webserver:
extraEnvs: []

service:
type: NodePort
port: 80 `2. Ingress was using 8080 for webserver but service was 80
3. Had to change path in ingress to “/*” instead of “/” to get it working for me

Yeah getting a better sense

As of now my understanding is agent and worker are just interchangeable words

Yeah sounds good to me

But ok the summary is I guess it doesn’t work in a k8s env

BTW when I started using s3, I was thinking I needed to specify ouput_uri for each task. Soon realized that you just need the prefix where you want to put it into, and clearml will take care of project etc being appended to the path. So for most usecases, a single output uri set in conf should work.

Thanks that works. Had to use Task.completed() for my version

I do it to get project name

Yes using clearml-data.

Can I pass a s3 path to ds.add_files() essentially so that I can directly store a dataset without having to get the files to local and then upload again. Makes sense?

In this case I have data and then set of pickles created from the data

Ok couldn’t see it in the docs - https://clear.ml/docs/latest/docs/references/sdk/task