Hi @<1523701868901961728:profile|ReassuredTiger98>

Anyone here with any idea why my service tasks get aborted when going to sleep?

I think I understand the issue, clearml==1.4.0 try running with the latest clearml (1.10.x)
It will keep pinging the backend "Im alive" so the backend does not think this process is dead (which I suspect what happened, and after 2 hours the backend basically set the Task to aborted because it "thought" it was killed)

👍

BTW, it looks like a lot of users really like the idea of runnig pipeline steps as subprocesses (which frankly I really cannot understand as Python Process is such an amazing tool to do just that),
anyhow We will have PipelineDecorator.debug_pipeline() which will run the pipeline steps as functions, and PipelineDecorator.execute_locally() which will run the Pipeline steps as subprocess
wdyt?

Seems like credentials error
Do you have everything setup correctly in your ~/clearml.conf ?

Hi GiganticTurtle0
ClearML will only list the directly imported packaged (not their requirements), meaning in your case it will only list "tf_funcs" (which you imported).
But I do not think there is a package named "tf_funcs" right ?

I mean, can you install it with something like ?
pip install git+Basically the agent will install main repository, and any git submodules. But it cannot install multiple repositories, as the directory structure might be too much.
wdyt?

Hi PanickyMoth78 an RC with a fix is out, let me know if it works (notice you can now set the max_workers from CLI or Dataset functions) pip install clearml==1.8.1rc1

Hmm, so what I'm thinking is "extending" the capabilities of the "configuration" section (as it seems this is the right context). Allowing to upload a bunch of files (with the same mechanism as artifacts), as zip files, in the configuration "editable" section have the URL storing the zip, together with the target folder. wdyt?

because a pipeline is composed of multiple tasks, different tasks in the pipeline could run on different machines.

Yes!

. Or more specifically, they could run on different queues, and as you said, in your other response, we could have a Q for smaller CPU-based instances, and another queue larger GPU-based instances.

Exactly !

I like the idea of having a queue dedicated to CPU-based instances that has multiple agents running on it simultaneously. Like maybe four agents.

Th...

@<1562610699555835904:profile|VirtuousHedgehong97>

source_url="s3:...", 

This means your data is already on S3 bucket, it will not "upload" it it will just register it.
If you want to upload files, then they should be local and then when you call upload you can specify the target S3 bucket, and the data will be stored in a unique folder in the bucket
Does that make sense ?

MuddyCrab47 could you post the full sample code you are using?

ComfortableShark77 it seems the clearml-serving is trying to Upload data to a different server (not download the model)
I'm assuming this has to do with the CLEARML_FILES_HOST, and missing credentials. It has nothing to do with downloading the model (that as you posted, will be from the s3 bucket).
Does that make sense ?

Hi @<1707565838988480512:profile|MeltedLizard16>
Maybe I'm missing something but gust add to your YOLO code :

from clearml import Dataset

my_files_folder = Dataset.get("dataset_id_here").get_local_copy()

what am I missing?

Hmm let me check something

DefeatedCrab47 If I remember correctly v1+ has their arguments coming from argparse .
Are you using this feature ? 2. How do you set the TB HParam ? Currently Trains does not support TB HParams, the reason is the set of HParams needs to match a single experiment. Is that your case?

LazyTurkey38 , ohh I think you are correct 😞
it should be:
# patch the Task and actually send it for execution if Task.running_locally(): # this will verify all auto repo detection and python is done. task.close() # so that we can edit the task task.reset() # update the repo task.update_task(task_data={'script': {'branch': 'new_branch', 'repository': 'new_repo'}}) # now to actually enqueue the Task Task.enqueue(task, queue_name='default')wdyt?

Martin, thank you very much for your time and dedication, I really appreciate it

My pleasure 🙂

Yes, I have latest 1.0.5 version now and it gives same result in UI as previous version that I used

Hmm are you saying the auto hydra connection doesn't work ? is it the folder structure ?
When is the Task.init is called ?
See example here:
https://github.com/allegroai/clearml/blob/master/examples/frameworks/hydra/hydra_example.py

If you want to rename it (any pipeline), click on the "Full details" in the "Run Info" (right hand side panel), then in the full detail of the Pipeline Task you will be able to rename the pipeline execution
(Is renaming useful? should we add a right click to rename ?)

Thanks SmallDeer34 , I think you are correct, the 'output' model is returned properly, but "input" are returned as model name not model object.
Let me check something

I added the following to the

clearml.conf

file

the conf file that is on the worker machine ?

, I can see the shape is

[136, 64, 80, 80]

. Is that correct?

Yes that's correct. In case of the name, just try input__0
Notice you also need to convert it to torchscript

And you have the exact same folder structure / content, and server A/B give a different set of experiments ?
(is serverB empty, meaning no experiments at all?)

Hi ShinyWhale52
Every execution of the pipeline (by definition) will create a new job based on the pipeline steps
This is the reason you see all the steps twice (the default assumption is you wish to re-run the step, as this is part of the processing workflow (e.g. training a model)

the model has been overwritten. I guess this is due to this instruction:

This is because you are storing it locally to the same path, it just reflects the fact you just overwrote your model.
To create a...

Hmmm that is a good use case to have (maybe we should have --stop get an argument ?)
Meanwhile you can do
$ clearml-agent daemon --gpus 0 --queue default $ clearml-agent daemon --gpus 1 --queue default then to stop only the second one: $ clearml-agent daemon --gpus 1 --queue default --stopwdyt?

This would work to load the local modules, but I’m also using poetry and the

pyproject.toml

is in the subdirectory, so the agent won’t install any dependency if I don’t set the

work_dir

hmmm true, in terms of requirements, you can list them in the decorator (see packages argument)

ElegantCoyote26 could you upgrade the docker-compose ?

Hi @<1628927672681762816:profile|GreasyKitten62>
Notice that in the github actions example this psecific Task is executed on the GitHub backend, the Task it creates is executed on the clearml-agent.
So basically:
Action -> Git worker -> task_stats_to_comment.py -> Task Pushed to Queue -> Clearml-Agent -> Task execution is here

Does that make sense ?

Sounds good to me. DepressedChimpanzee34 any chance you can add a github feature request, so we do not forget to add it?

Hi MinuteWalrus85
This is great question, and super important when training models. This is why we designed a whole system to manage datasets (including storage querying, balancing data, and caching). Unfortunately this is only available in the paid tier of Allegro... You are welcome to https://allegro.ai/enterprise/ the sales guys.
🙂

Great, you can test directly from the master 🙂
pip3 install -U git+