sure, will do AlertBlackbird30

Thanks AgitatedDove14 - is the datasets feature pretty new?

AgitatedDove14 - just saw about start_remotely - https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#start_remotely

This means the services agent will take care of taking it to completion right?

Or WARNING should be param not found, but using General/param etc

I have convention of root task for experiments and a sub-project Pipelines for running “sandboxed” pipeline task. So I do some magic like this

It does have click==8.0.1

Good question 🙂

this is what I am seeing in the logs:

` No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
No tasks in queue 9154efd8a1314550b1c7882981720861
No tasks in Queues, sleeping for 5.0 seconds
K8S Glue pods monitor: Failed parsing kubectl output...

But if haven’t done a task yet

AgitatedDove14 - these instructions are out of date? https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_kubernetes_helm.html

Yes. It's a pickle file that I have added via OutputModel

OK i found what’s happening:

I had an additional Task.init() - just the blank one, to get the project name. Adding the disable to that as well fixed the issue

I am on 1.0.3

I am doing something like this with a yaml based pipelines DSL

and then do Task.current_task() it is None as expected

Any updates on trigger and schedule docs 🙂

I always have my notebooks in git repo but suddenly it's not running them correctly.

And other question is clearml-serving ready for serious use?

AlertBlackbird30 - i don’t understand why it can’t be a focus though. Probably missing some context.

Yeah will use that

“notebook as a service” right

Task.add_requirements would fit the bill yeah, thanks

Initially thought use_current_task in Dataset.create does it, but that seems to set the DataprocessingTask itself to the current task

Only one. Will replicate it in detail and see what’s actually up

Something like the Gitlab Kubernetes Executor could work well.

This Docker Mode - https://allegro.ai/clearml/docs/docs/use_cases/clearml_agent_use_case_examples.html#docker-mode

Got it thanks TimelyPenguin76 .

AgitatedDove14 - this was an interesting one. I think I have found the issue, but verifying the fix as of now.

One of the devs was using shutil.copy2 to copy parts of dataset to a temporary directory in a with block - something like:

with TemporaryDirectory(dir=temp_dir) as certificates_directory: for file in test_paths: shutil.copy2(f"{dataset_local}/{file}", f"{certificates_directory}/file")
My suspicion is since copy2 copies with full data and symlin...

` if project_name is None and Task.current_task() is not None:
project_name = Task.current_task().get_project_name()

if project_name is None and not Task.running_locally():
task = Task.init()
project_name = task.get_project_name() `

Would be good to have frequentish releases if possible 🙂

AgitatedDove14 - worked with mutable copy! So was definitely related to the symlinks in some form