I can reproduce 😞 give me a moment to verify

Hi @<1523701079223570432:profile|ReassuredOwl55> let me try ti add some color here:
Basically we have to parts (1) pipeline logic, i.e. the code that drives the DAG, (2) pipeline components, e.g. model verification
The pipeline logic (1) i.e. the code that creates the dag, the tasks and enqueues them, will be running in the git actions context. i.e. this is the automation code. The pipeline components themselves (2) e.g. model verification training etc. are running using the clearml agents...

TightElk12 I think this message belongs to a diff thread ;)

Hi @<1523701181375844352:profile|ExasperatedCrocodile76>
the docker containers should get the host IP, not the internal docker IP. what am I missing ?

This is something that we do need if we are going to keep using ClearML Pipelines, and we need it to be reliable and maintainable, so I don’t know whether it would be wise to cobble together a lower-level solution that has to be updated each time ClearML changes its serialisation code

Sorry if I was not clear, I do not mean for you ti do unstable low-level access, I meant that pipelines are Designed to be editable externally, they always deserialize themselves.
The only part that is mi...

@<1523710674990010368:profile|GreasyPenguin14> make sure it to uses https not ssh:
edit ~/clearml.conf

force_git_ssh_protocol: false

and that you have both git_user & git_pass set in your clearml.conf

got to love the ascii art 😍

FYI: pipeline callbacks are already part of v1.0 🙂

Hi DeliciousBluewhale87
clearml-agent 0.17.2 was just release with the fix, let me know if it works

because step can be constructed with multiple

sub-components

but not all of them might be added to the UI graph

Just to make sure I fully understand when we decorate with @sub_node we want that to also appear in the UI graph (and have it's own Task / metrics etc)
correct?

Hi BattyLion34
The windows issue seems like it is coming from missing QT installed on the Host machine
Check the pyqt5 version in your "Installed packages"
see here:
https://superuser.com/questions/1433913/qtpy-pythonqterror-no-qt-bindings-could-be-found

Regrading the linux, it seems your are missing the object_detection package, where do you usually install it from ?

I think there is a bug on the UI that causes series with "." to only use the first part of the series name for the color selection. This means "epsilon 0" and "epsilon 0.1" will always get the same color, and this will explain why it works on other graphs

Sure, ReassuredTiger98 just add them after the docker image in the "Base Docker image" section under the execution Tab. The same applies for setting it from code.
example:
nvcr.io/nvidia/tensorflow:20.11-tf2-py3 -v /mnt/data:/mnt/dataYou can also always force extra docker run arguments by changing the clearml.conf on the agent itself:
https://github.com/allegroai/clearml-agent/blob/822984301889327ae1a703ffdc56470ad006a951/docs/clearml.conf#L121

The .ssh is mounted, but the owner is my local user,

sudo -H clearml-agent ...to allow sudo to access home

Hi ReassuredOwl55

How would I find Tasks that have the same code with different inputs/parameters?

Assuming you have the git repo
you can do:
Task.query_tasks(..., task_filter={'_all_'=dict(fields=['script.repository'], pattern='github.com/user/repo'))wdyt?

Is it the same with the latest RC 1.10.0rc?

🙏

Can you try to set this in your clearml.conf:

agent.pip_download_cache.enabled: false

this should disable the local caching, of your wheel, I suspect there is some issue with the local cache file in windows...

Does adding external files not upload them ti the dataset output_uri?

@<1523704667563888640:profile|CooperativeOtter46> If you are adding the links with add_external_files these files are Not re-uploaded

This one? https://clear.ml/docs/latest/docs/references/sdk/logger#report_single_value

OmegaConf is the configuration, the overrides are in the Hyperparameters "Hydra" section
None

Hi @<1631826770770530304:profile|GracefulHamster67>
if you want your current task:

task = Task.current_task()

if you need the pipeline Task from the pipeline component

pipeline = Task.get_task(Task.current_task().parent)

where are you trying to get the pipelines from? I'm not sure I understand the use case?

im not running in docker mode though

hmmm that might be the first issue. it cannot skip venv creation, it can however use a pre-existing venv (but it will change it every time it installs a missing package)
so setting CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=1 in non docker mode has no affect

SpicyCrab51 you can change the task to complete, it is just a state change nothing will actually change other than the status. Task.get_task(pass_dataset_id_here).mark_complete()

You could change infrastructure or hosting, and now your data is associated with the wrong URL

Yeah that makes sense, so have it on a specific dns name? (this is usually the case with k8s deployments)

feature request: tell me what gets passed along each edge of the pipeline graph

Nice! please feel free to add to GH issue 🙂
Hmm yes that is odd, let me see if I can reproduce

And is it talking to the correct server ?

WackyRabbit7 How do I reproduce it ?

That makes total sense. The question was about the Mac users and OS environment in the configuration file and having that os environment set in code (this is my assumption as it seems that at import time it does not exist). What am I missing here?

I think you are correct the env variable is not resolved in "time". It might be it's resolved at import not at Task.init