Sorry if it was confusing. Was asking if people have setup pipelines automatically triggered on update to datasets

AgitatedDove14 - where does automation.controller.PipelineController fit in?

now if dataset1 is updated, i want process to update dataset2

Trying to understand these, maybe playing around will help

I just want to change git remote like https://gitserver.com/path/to.git -> mailto:git@gitserver.com :path/to.git

I was thinking such limitations will exist only for published

When did this PipelineDecorator come. Looks interesting 🙂

You mean the job with the exact same arguments ?

Yes

From the code - it’s supposed to not cache if task override is different? I also have task_override that adds a version which changes each run

Or WARNING should be param not found, but using General/param etc

He/Him 🙂

I don’t use the UI. Which param do you mean?

AgitatedDove14 - any thoughts on this?

Can I disable this cache?

Is it not possible to say just look at my requirements.txt file and the imports in the script?

Cool, didn't know it was disabled. This exact reason was why I created a wrapper over ClearML for my use so that people don't ever accidentally talk to demo server

Here’s an example error I get trying it out on one of the example models:
Error: Requested Model project=ClearML Examples name=autokeras imdb example with scalars tags=None not found. 'config.pbtxt' could not be inferred. please provide specific config.pbtxt definition.

All right, that’s the only way? 🙂

Ok so that’s nothing more than what I would configured in the clearml config then

Thoughts AgitatedDove14 SuccessfulKoala55 ? Some help would be appreciated.

The description says this though

A section name associated with the connected object. Default: 'General'

I always have my notebooks in git repo but suddenly it's not running them correctly.

I have a wrapper SDK over clearml that includes default conf and others are loaded from secret manager / env vars as needed

This is the command that is running:

` ['docker', 'run', '-t', '-e', 'NVIDIA_VISIBLE_DEVICES=none', '-e', 'CLEARML_WORKER_ID=clearml-services:service:c606029d77784c69a30edfdf4ba291a5', '-e', 'CLEARML_DOCKER_IMAGE=', '-v', '/tmp/.clearml_agent.72r6h9pl.cfg:/root/clearml.conf', '-v', '/root/.clearml/apt-cache:/var/cache/apt/archives', '-v', '/root/.clearml/pip-cache:/root/.cache/pip', '-v', '/root/.clearml/pip-download-cache:/root/.clearml/pip-download-cache', '-v', '/root/.clearml/cache:/clea...

As in if there are jobs, first level is new pods, second level is new nodes in the cluster.

AgitatedDove14 - any pointers on how to run gpu tasks with k8s glue. How to control the queue and differentiate tasks that need cpu vs gpu in this context

Running multiple k8s_daemon rightt? k8s_daemon("1xGPU") and k8s_daemon('cpu') right?

AgitatedDove14 aws autoscaler is not k8s native right? That's sort of the loose point I am coming at.

For different workloads, I need to habe different cluster scaler rules and account for different gpu needs

Got it. Never ran GPU workload in EKS before. Do you have any experience and things to watch out for?