barebones, can a step in a pipeline refer to a previous step by name and get it?

So General would have created a General instead of Args?

From code perspective what changes?

My question is - I have this in a notebook now. How can i make it such that any update to the upstream database triggers this data transformation step

Nope, that doesn’t seem to be it. Will debug a bit more.

Model says PACKAGE, that means it’s fine right?

Will debug a bit more and see what’s up

this task - https://github.com/allegroai/clearml/blob/5a9155b2039413280f13dfded1121470c4c4323d/examples/pipeline/step2_data_processing.py

What’s the point of saying General?

AgitatedDove14 - where does automation.controller.PipelineController fit in?

For now that's a quick thing, but for actual use I will need a proper model (pkl) and the .py

Yes, I have no experience with triton does it do lazy loading? Was wondering how it can handle 10s, 100s of models. If we load balance across a set of these engine containers with say 100 models and all of these models get traffic but distribution is not even, each of those engine container will load all those 100 models?

AgitatedDove14 you had mentioned some pipeline related changes coming, are they in?

Anything that is shown in git status as untracked? So ignore .gitignored. and maybe a oaram or config to say include untracked. Anyway, it's only a nice to have feature.

Example:

name: ml-project template: nbdev pipelines_runner: gitlab pipelines: pipeline-1: steps: - name: "publish-datasets" task_script: "mlproject/publish_datasets.py" - name: "training" task_script: "mlproject/training.py" parents: ["publish-datasets"] - name: "test" task_script: "mlproject/test.py" parents: ["training"]Have cli which goes through each of the tasks and creates them

Progress with boto3 added, but fails:

It would be good if new agent package is published

As of now my understanding is agent and worker are just interchangeable words

Yeah got it. Was mainly wondering if k8s glue was meant for this as well or not

AgitatedDove14 - looks like the serving is doing the savemodel stuff?

https://github.com/allegroai/clearml-serving/blob/main/clearml_serving/serving_service.py#L554

I can contribute as well as needed

I used .update_weights(path) with path being the model dir containing the model.py annd the config.pbtxt. Should I use update_weights_package ?

It did pick it from the task?

AgitatedDove14 - on a similar note, using this is it possible to add to requirements of task with task_overrides?

I don’t use the UI. Which param do you mean?

It’s a bit difficult to figure out the exact key. Will try this to see if it works

Having a pipeline controller and running actually seems to work as long as i have them as separate notebooks

Not able to understand what’s really happening in the links

Hey TimelyPenguin76 - i am just using the helm chart and haven’t done any setup on top of that. the agentservices is running as is from the helm chart

Yeah please if you can share some general active ones to discuss both algos and engineering side

Also btw, is this supposed to be screenshot from community verison? https://github.com/manojlds/clearml-serving/blob/main/docs/webapp_screenshots.gif