PompousBeetle71 BTW: if you remove the type=str from the argparse, it will do what you want, None will stay None (instead of ''), all other values will be of type str as this is always the default 🙂

Ok..so I should generally avoid connecting complex objects? I guess I would create a 'mini dictionary' with a subset of params, and connectvthis instead.

In theory it should always work, but this specific one fails on a very pythonic paradigm (see below)

from copy import copy
an_object = copy(object)

A good rule of thumb is to connect any object/dict that you want to track or change later

Just verifying the Pod does get allocated 2 gpus, correct ?
What do you have under the "script path" in the Task?

Hi MelancholyChicken65
I'm not sure you an control it, the ui deduces the URL based on the address you are browsing to: so if you go yo http://app.clearml.example.com you will get the correct ones, but you have to put them on the right subdomains:
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config#subdomain-configuration

I guess i need to do something like the following after the task was created:
...

Yes!

Why use the "post" callback and not the "pre" callback?

The post get's back the Model object. The pre allows you to decide if you actually want to log in the first place (come to think about it, maybe you want that as well 🙂 )

SuperficialGrasshopper36 regrading the codeartifact
I think the easiest will be to have a bash script authenticating the codeartifact with the aws command at the beginning of each docker spin. This can be done by adding it to:
https://github.com/allegroai/clearml-agent/blob/81edd2860fbc09e2a179985d8315ffaba851dcd7/docs/clearml.conf#L136
For example:
extra_docker_shell_script: ["apt-get install -y aws_cli_or_something", "aws cli authenticate me command"]wdyt?

Sure, run:
clearml-agent initIt is a CLI wizard to configure the initial configuration file.

No

make sense ?

. Does

Task.connect

send each element of the dictionary as a separate api request? Has anyone else encountered this issue?

Hi SuperiorPanda77
the task.connect ends up as a single call with all the data being sent on a single request.
That said, maybe the connect dict is not the best solution for thousand key dictionary ...
Maybe artifact, or connect_configuration are better suited ?
wdyt?

Hmm so I guess the actual code adds it into the reporting itself ...
How about we call:
task.set_initial_iteration(0)

AstonishingWorm64
You can turn on the venv cache , it will just handle it's own full env caching 🙂
See here:
https://github.com/allegroai/clearml-agent/blob/4f7407084d1900a79d455570c573e60f40208742/docs/clearml.conf#L100

👍

clearml - WARNING - Could not retrieve remote configuration named 'hyperparams'

What's the clearml-server version you are working with ?

In both logs I see (even in the single GPU log, it seems you "see" two GPUs, is that correct?)
GPU 0,1 Tesla V100-SXM2-32GB (arch=7.0)

Last question, this is using relatively old clearml version (0.17.5), can you test with the latest version (1.1.1)?

UnevenDolphin73 since at the end plotly is doing the presentation, I think you can provide the extra layout here:
https://github.com/allegroai/clearml/blob/226a6826216a9cabaf9c7877dcfe645c6ae801d1/clearml/logger.py#L293

NonchalantDeer14
I think the issue is the way it spins the subprocess is not with fork but with Popen, so clearml is not "loaded" into the subprocess hence no logging.
The easiest fix is to call Task.current_task() inside the actual code (somewhere when it starts), it should trigger clearml.

Hi SmugLizard25 I was able to test and it seems that style is being ignored by the FE 😞
I passed to FE guys to make sure it is fixed in the next version.
Notice this is just for tables, anything else works as expected (i.e. styling any other type of plot)

It should have worked....
Can you run the examples from the repo and see if they work?

and the agent default runtime mode is docker correct?

Actually the default is venv mode, to run in docker mode add --docker to the command line

So I could install all my system dependencies in my own docker image?

Correct, inside the docker it will inherit all the preinstalled packages, But it will also install any missing ones (based on the Task requirements. i.e. "installed packages" section)

Also what is the purpose of the

aws

block in the clearml.c...

SmarmyDolphin68 if you can reproduce the behavior in a standalone script , it will really accelerate fixing this issue

DisturbedWorm66 it does, I think there is an example here:
https://github.com/allegroai/nvidia-clearml-integration/tree/main/tlt

SpotlessFish46 unless all the code is under "uncommitted changes" section, what you have is a link to the git repo + commit id

named as 

venv_update

 (I believe it's still in beta). Do you think enabling this parameter significantly helps to build environments faster?

This is deprecated... it was a test to use the a package that can update pip venvs, but it was never stable, we will remove it in the next version

Yes, I guess. Since pipelines are designed to be executed remotely it may be pointless to enable an 

output_uri

 parameter in the 

PipelineDecorator.componen...

Hi @<1575656665519230976:profile|SkinnyBat30>
Streamlit apps are backend run (i.e. the python code drives the actual web app)
This means running your Tasks code and exposing the web app (i.e. http) streamlit.
This is fully supported with ClearML, but unfortunately only in the paid tiers 😞
You can however run your Task with an agent, make sure the agent's machine is accessible and report the full IP+URL as a hyper-parameter or property, and then use that to access your streaml...

So the main difference is kedro pipelines are function based steps (I might be overly simplifying, so please take it with a grain of salt), while in ClearML pipeline is Job, i.e. it needs its own environment and is longer than a few seconds (as opposed to a single function)

SmallBluewhale13
And the Task.init registers 0.17.2 , even though it prints (while running the same code from the same venv) 0.17.2 ?

GreasyPenguin14 could you test with the matplotlib lib example ? (I cannot reproduce it and it seems like something to do with pycharm and matplotlib backend)
https://github.com/allegroai/clearml/blob/master/examples/frameworks/matplotlib/matplotlib_example.py

still it is a chatgpt interface correct ?

Actually, no. And we will change the wording on the website so it is more intuitive to understand.
The idea is you actually train your own model (not chatgpt/openai) and use that model internally, which means everything is done inside your organisation, from data through training and ending with deployment. Does that make sense ?

seems to run properly now

Are you saying the problem disappeared ?

Does it wok if you remove the Task.init call?