This part is odd:
SCRIPT PATH: tmp.7dSvBcyI7mHow did you end with this random filename? how are you running this code?

Hi @<1657918706052763648:profile|SillyRobin38>

I'm curious to know if it's possible to prevent uploading a duplicate endpoint.
...and we attempt to upload it again without any changes to the command content,

Basically you overwrite it, and yes, possible 🙂

any other aspect, could the system prevent the duplicate upload?

so basically check the hash and say, no need to upload?

Shout-out to Emilio for quickly stumbling on this rare bug and letting us know. If you have a feeling your process is stuck on exit, just upgrade to 1.0.1 😉

@<1523701868901961728:profile|ReassuredTiger98> thank you so much for testing it!

🤞

git config --system credential.helper 'store --file /root/.git-credentials'

Maybe we should use this hack for cloning with user/token in general ...

NICE!

Follow-up; any ideas how to avoid PEP 517 with the auto scaler?

Takes a

long

time to build the wheels

enable venv caching ?
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L116

UnevenDolphin73
fatal: could not read Username for ' ': terminal prompts disabled .. fatal: clone of ' ' into submodule path '/root/.clearml/vcs-cache/xxx.60db3666b11ac2df511a851e269817ef/xxx/xxx' failedIt seems it tries to clone a submodule and fails due to to missing keys for the submodule.
https://stackoverflow.com/questions/7714326/git-submodule-url-not-including-username
wdyt?

You put it there 🙂 so the assumption you know what you are looking for, or use glob? wdyt?

But it should work out of the box ...

Yes it should ....

The user and personal access token are used as is and it propagates down to submodules, since those are simply another git repository.

Can you manually successfully run:
git clone --recursive https://user:token@github.com/company/repo_with_submodules

Hi JealousParrot68

do tasks that are created through create_function_task run the entry_script again instead of just the pure function

Basically they will run the code until the "create_function_task" call, but never after. We are working on adding a decorator to a function, making it a "standalone" script, is this what you actually need ?

HugeArcticwolf77 actually it is more than that, you can embed the graphs now in the markdown, when you hove over any plot/graph/image you now have a new button that copies the embed test, so that you can directly copy it into your markdown editor (internal or external)
More documentation and screenshots are coming after the holiday, mean time you can check:
https://clear.ml/docs/latest/docs/webapp/webapp_reports
https://clear.ml/docs/latest/assets/images/webapp_report-695dddd2ec8064938bf8...

Hi PanickyMoth78

I had several pipeline components getting it and uploading files to is concurrently.

Should not be a problem

I've attached it's log file which only mentions skipping one file (a warning)

So what exactly is the error you are getting?

from the screenshot it looks like a "gs://" access issue?
could that be it?

at least you did not change permission of your K8s etcd folder 😄

No worries 🙂

Yes that is exactly what I will make sure we change :)

Could you also provide the full log?

And the agent section on this machine is:
api_server:  web_server:  files_server: 
Is that correct?

yes, or (because I deployed clearml using helm in kubernetes) from the same machine, but multiple pods (tasks).

Oh now I see, long story short, no 😞 the correct way of doing that is every node/pod creates it's own dataset,
then when you are done, you create a new version with the X datasets that you created as parents, the newly created version is just "meta" it basically tells the system how to combine the previously generated datasets (i.e. no data is actually re-uploa...

Oh!
I see this is using the colab as remote agent (i.e. to launch jobs on it),

[ColabKernelApp] CRITICAL | Bad config encountered during initialization: The 'kernel_class' trait of <main.ColabKernelApp object at 0x7fa41b29e5c0> instance must be a type, but 'google.colab._kernel.Kernel' could not be imported

Can you send the full log?

Hi @<1523702307240284160:profile|TeenyBeetle18>

and url of the model refers to local file, no to the remote storage.

Do you mean that in the Model tab when you look into the model details the URL points to a local location (e.g. file:///mnt/something/model) ?
And your goal is to get a copy of that model (file) from your code, is that correct ?

but the logger info is missing.

What do you mean? Can I reproduce it ?
BTW: The code sample you shared is very similar to how you create pipelines in ClearML, no?
(also could you expand on how you create the Kedro node ? from te face o fit it looks like another function in the repo, but I have a feeling I'm missing something)

Thanks EnviousStarfish54
Let me check if I can reproduce it

Why does my task execution freeze after pip installation (running agent in foreground mode)?

Hi AdventurousButterfly15
Are you running in agent docker mode or venv mode ?
What do you mean freeze? do you see anything on the Taks console log in the UI? what's the host OS ?

Hi PompousParrot44
What do you have in the Execution/"script path" ?

Checkout the trains-agent repo https://github.com/allegroai/trains-agent
It is fairly straight forward.

Are Kwargs supported in functions decorated as a pipeline component?

They are, but I think the main issue is the casting, without prior knowledge, everything will be a tring

before exposing our IP to the world, I suggest going over security advisory in the docs: None
as a general note, do not expose your server, the open source version is not designed for it, just put it inside your VPN and it will be fine