🤞

(as i see the services worker is only in the services-queue, and not my default queue (where my other servers/workers are)

So basically the service-mode is just a flag passed to the agent, and the services queue is the name of the queue it will pull from.

If i want a normal worker also

You can just add another section to the docker-compose, or run it manually after you spin the docker-compose.

LazyFox65 wdyt ?

does clearml expect them to be actuall installed to add them as installed packages for a task?

It should add itself to the list (assuming you will end up calling Task.init in your code)

Yep it is the scale 🙂 and yes it should appear once you upgrade

Yeah.. that should have worked ...
What's the exact error you are getting ?

what's the docker version?

I see, so there’s no way to launch a variant of my last run (with say some config/code tweaks) via CLI, and have it re-use the cached venv?

Try:
clearml-task ... --requirements requirements.txtYou can also clone / override args with
clearml-task --base-task-id <ID-of-original-task-post-agent> --args ...See full doc: https://clear.ml/docs/latest/docs/apps/clearml_task/

1633204289496 clearml-services DEBUG docker: invalid reference format.

This is the strange message, like the execution command is not valid...

one can containerise the whole pipeline and run it pretty much anywhere.

Does that mean the entire pipeline will be running on the instance spinning the container ?
From here: this is what I understand:
https://kedro.readthedocs.io/en/stable/10_deployment/06_kubeflow.html

My thinking was I can use one command and run all steps locally while still registering all "nodes/functions/inputs/outputs etc" with clearml such that I could also then later go into the interface and clone an...

HurtWoodpecker30 currently in the open source only AWS is supported, I know the SaaS pro version supports it (I'm assuming enterprise as well).
You can however manually spin an instance on GCP and launch an agent on the instance (like you would on any machine)

Actually, no. This is ti spin the clearml-server on GCP, not the agent

So you are saying it ignored everything after the bucket's "/" ?

What was the url you ended up with ?

It should have been:
output_uri="s3://company-clearml/artifacts/bethan/sales_journeys/artifacts/examples/load_artifacts.f0f4d1cd5eb54795b11508dd1e739145/artifacts/filename.csv.gz/filename.csv.gz

Hmm, let me check something

Okay, let me check soemthing

So the thing is clearml automatically detects the last iteration of the previous run, my assumption you also add it hence the double shift.
SourOx12 could that be it?

Okay let me check....

BTW:

======> WARNING! Git diff to large to store (1327kb), skipping uncommitted changes <======

This means all your git changes are stored as an artifact, which is consistent with the "wait for upload" message.

I think for it to work you have to have ssh running on the host machine (the socket client itself), no?

Hi FiercePenguin76

should return all datasets from all projects?

Correct 🙂

Hmm you either need to run with SUDO or make sure the running user has docker run permissions

Hmmm are you saying the Dataset Tasks do not have the "dataset" system_tag as well as the type ?

RC3 is out with a few fixes, please feel free to test 🙂

DeliciousBluewhale87 out of curiosity , what do you mean by "deployment functionality" ? is it model serving ?

JitteryCoyote63 Is this an Ignite feature ? what is the expectation ? (I guess the ClearML Logger just inherits from the base ignite logger)

JitteryCoyote63 nice hack 😄
how come it is not automatically logged as console output ?

Can you see it on the console ?

Hmm how do you launch the autoscaler, code?

terminal