From code perspective what changes?

So General would have created a General instead of Args?

I am doing something like this with a yaml based pipelines DSL

Example:

name: ml-project template: nbdev pipelines_runner: gitlab pipelines: pipeline-1: steps: - name: "publish-datasets" task_script: "mlproject/publish_datasets.py" - name: "training" task_script: "mlproject/training.py" parents: ["publish-datasets"] - name: "test" task_script: "mlproject/test.py" parents: ["training"]Have cli which goes through each of the tasks and creates them

AS in clone the task from the UI and run it

I guess the question is - I want to use services queue for running services, and I want to do it on k8s

AgitatedDove14 - thoughts on this? I remember that it was Draft before, but maybe because it was in a notebook vs now I am running a script?

Or in the code? - https://github.com/allegroai/clearml/blob/master/clearml/task.py

What happens if I do blah/dataset_url ?

AgitatedDove14 - these instructions are out of date? https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_kubernetes_helm.html

It did pick it from the task?

The repo seems to be

I have a wrapper SDK over clearml that includes default conf and others are loaded from secret manager / env vars as needed

Thanks AlertBlackbird30

AgitatedDove14 - any doc yet for scheduler? Is it essentially for just time based scheduling?

I then install this wrapper SDK in my containers, notebook instances etc

Ok couldn’t see it in the docs - https://clear.ml/docs/latest/docs/references/sdk/task

sure, will do AlertBlackbird30

🙏

Ok it’s not implemented right, that’s what I was asking

in the repo whereas the docs are https://allegroai.github.io/clearml-server-helm/

AgitatedDove14 sounds almost what might be needed, will give it a shot. Thanks, as always 🙂

Ok it’s here? https://github.com/allegroai/clearml/blob/master/clearml/backend_interface/task/task.py#L1352

There’s also the cli to create tasks- https://github.com/allegroai/clearml/blob/master/docs/clearml-task.md

All right got it, will try it out. Thanks for the quick response.

Thanks!

AlertBlackbird30 - got it running. Few comments:

Nodeport is set by default despite being parameter in values.yml. For example:` webserver:
extraEnvs: []

service:
type: NodePort
port: 80 `2. Ingress was using 8080 for webserver but service was 80
3. Had to change path in ingress to “/*” instead of “/” to get it working for me

The helm chart installs a agentservice, how is that related if at all?

AgitatedDove14 - is my understanding right that we have to call pipe.wait() ?

Only one. Will replicate it in detail and see what’s actually up