... the one for the last epoch and not the best one for that experiment,

well

Now we realized there is an option tu use

"min_global"

on the sign, is this what we need?

Yes 🙂 (or max_global)

you can also set the 

agent.package_manager.extra_index_url

 , but since this is dynamic,...

You are correct, sine this is dynamic there is no need to set the " extra_index_url " configuration in clearml.conf, the additional bash script will configure pip directly. Make sense ?

Hi CleanPigeon16
You need to pass the private repository docker credentials to the aws instance, I would use the custom bash script option of the aws autoscaler to create the docker credentials file.

Hi VexedElephant56
Yes it is:
Define CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=1
(if running in doecker mode add -e CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1 as container args)
https://clear.ml/docs/latest/docs/clearml_agent/clearml_agent_env_var

How do I best utilize clearml in this scenario such that any coworker of mine is able to reproduce my work with the same pipeline?

Basically this sounds to me like proper software developemnt design (i.e. the class vs stages).
In order to make sure Anyone can reproduce it, you mean anyone can rerun the "pipeline" ? If this is the case just add Task.init (maybe use a specific Task type) and the agents will make sure this is Fully reproducible.
If you mean the data itself is stored, the...

ReassuredTiger98

How can I make clearml-agent use pre-installed version from the nvidia/pytorch

If the Same version is required, the agent will not try to reinstall it (the new venv the agent is creating inside the container, inherits from the preinstalled system packages)

Comes with PyTorch Version 1.12 based on a commit

. I tried

torch >= 1.11

,

torch == 1.12

If in your installed packages you have torch==1.12
the agent should not tr...

Hi @<1590514584836378624:profile|AmiableSeaturtle81>
I think you should use add_external_files , instead of add_files (which is for local files)
None

Yeah I can write a script to transfer it over, I was just wondering if there was a built in feature.

unfortunately no 😞
Maybe if you have a script we can put it somewhere?

JitteryCoyote63 How is it so far ?

but then an error message in the web-app pops up

Fetch parents failed

and the Scheduler task disappears

And the Task is still running? What's he clearml python version and webui version ?

Hi @<1643060801088524288:profile|HarebrainedOstrich43>
I think I understand what's going on, in order for the pipeline logic to be "aware" of the pipeline component, it needs to be declared in the pipeline logic script file (or scope if you will).
Try to import from src.testagentcomponent import step_one also in the global pipeline script (not just inside the function)

When using the UI with regex to search for experiments, due to the greedy nature of the search, it consistently pops up the "ERROR Fetch Experiments failed" window when starting to use groups in regex (that is, parentheses of any kind).

hmm that is a good point (i.e. only on enter it would actually search)

Could it be updated so that if an invalid regex pattern is given, it simply highlights the search bar in red (or similar) rather than stop us while writing the search pattern?

...

Basically lock the Task (so you cannot reset it or change it). Usually it also marks "ready to use" etc. It also will publish the models the Task created.

Hi @<1523704207914307584:profile|ObedientToad56>

hat would be the right way to extend this with let's say a custom engine that is currently not supported ?

as you said 'custom' 🙂
None
This is actually a custom engine, (see (3) in the readme, and the preprocessing.py implementing it) I think we should actually add a specific example to custom so this is more visible. Any thoughts on what would...

LazyFox65 seems like a docker issue.
Can you manually run the docker ?

Can't say I have noticed that, is this a delay on the send ? Which for some reason is correlated with the epochs ? What was the case with 0.17.5?

Hi @<1556450111259676672:profile|PlainSeaurchin97>
You mean instead of the parallel coordinates ?
None

NastyOtter17
Usually the first report will happen after 30 seconds, could that be the difference ?

I'm guessing some network issue, though I can't figure why it cannot connect and curl seems to work

Wait so how come the "clearml" package is not installed ? can you provide the full log?

LOL I hear you 🙂

I have the same offset (that appear after each fail on my scalars).

Hmm, I actually would think this is the "correct" behavior, but I see your point:
Any chance you can open a GH issue ?

Hi @<1571308003204796416:profile|HollowPeacock58>

parameters = task.connect(config, name='config_params')

It seems that your DotDict does not support the python copy operator?
i.e.

from copy import copy
copy(DotDict())

fails ?

tried it and restarted the agent, but not working properly

What do you mean not working? can you provide logs ?

Hi @<1566596960691949568:profile|UpsetWalrus59>
you should call it before initializing the Task

Task.ignore_requirements("pywin32")
task = Task.init(...)

Hi VexedCat68

txt file or pkl file?

If this is a string , it just stored it (not as a file, this is considered a "link")
https://github.com/allegroai/clearml/blob/12fa7c92aaf8770d770c8ed05094e924b9099c16/clearml/binding/artifacts.py#L521

Yep, but a funny hack nonetheless.
No idea why they have it there...

You are doing great 🙂 don't worry about it

Decorators are good 🙂
Something along the lines of
` @PipelineDecorator.pipeline(...)
def pipeline(skip_a=False):
if not skip_a:
a = step_a()
else:
# somehow get a previous A?
# let's call it cached A
a = "replace with real'

step_b(a)
... `Is this the gist?
If it is, this looks like, "how can I control whether A is cached or not", is that correct?

With k8s glue going, want to finally look at clearml-session and how people are using it.

If used with k8s glue, you will have to run the glue with --ports-mode, then the clearml session will know how to connect to container itself, since at runtime the container will register the gateway + port for the learml-session client to connect to