Quick update, I found the issue, working on a fix 🙂

I think you cannot change it for a running process, do you want me to check for you if this can be done ?

Hmm interesting...
of course you can do:
dataset._task.connect(...)But maybe it should be public?!

How are you using that (I mean in the context of a Dataset)?

I should mention this is run within a TF v1 session context

This should not be connected.

everything gets stored as intended (to clearML dashboard)

So in jupyter it works? But from command line it does not ? what's the difference ?

I think I found something,
https://github.com/allegroai/clearml/blob/e3547cd89770c6d73f92d9a05696018957c3fd62/clearml/storage/helper.py#L1442
What's the boto version you have installed?

some dependencies will sometimes require different pip versions.

none 🙂 maybe setuptools, but not pip version
(pip is just a utility to install packages, it will not be a dependency of one)

LOL, if this is important we probably could add some support (meaning you will be able to specify it in the "installed packages" section, per Task).
If you find an actual scenario where it is needed, I'll make sure we support it 🙂

Hi DepressedChimpanzee34
This is not a query call, this is a reporting call. see docs below
https://clear.ml/docs/latest/docs/references/api/workers#post-workersstatus_report
It is used by the worker to report its own status.
I think this is what you are looking for:
https://clear.ml/docs/latest/docs/references/api/workers#post-workersget_stats

we can add non-clearml code as a step in the pipeline controller.

Yes 🙂 , btw you can kind of already do that, with pre/post function callbacks (notice they are running from the same scope as the actual pipeline controller).
What exactly did you have in mind to put there ?

Lately I've heard of groups that do slices of datasets for distributed training, or who "stream" data.

Hmm so maybe a "glob" alike parameter for get_local_copy(select_filter='subfolder/*') ?

Since you are running in venv mode, adding the OS environment before the clearml-agent, will basically make sure it will propagate to the process itself.
ReassuredTiger98 make sense ?

So I think this is a good example of pipelines and data:
Basically Task A generates data stored using the cleamrl-data (See Dataset class). The output of that is an ID of the Dataset. Then Task B uses that ID to retrieve the Dataset created by Task A.
documentation
https://github.com/allegroai/clearml/blob/master/docs/datasets.md

Example:
Step A creating Dataset:
https://github.com/alguchg/clearml-demo/blob/main/process_dataset.py
Step B training model using the Dataset created in ...

You mean the job with the exact same arguments ?
do you have other arguments you are passing ?
Are you using Optuna / HBOB ?

So far, i modified the code to set DOCKER_ROOT_CONF_FILE to what i want !!!
Interesting, do you think a PR is a good next step ? how one would configure it?

Yes

WickedGoat98
Put the agent.docker_preprocess_bash_script in the root of the file (i.e. you can just add the entire thing at the top of the trains.conf)

Might it be possible that I can place a trains.conf in the mapped local folder containing the filesystem and mongodb data etc e.g.

I'm assuming you are referring to the trains-=agent services, if this is the case, sure you can,
Edit your docker-compose.yml, under line https://github.com/allegroai/trains-server/blob/b93591ec3226...

If this is the case, then we do not change the maptplotlib backend
Also

I've attempted converting the 

mpl

 image to 

PIL

 and use 

report_image

  to push the image, to no avail.

What are you getting? error / exception ?

Could I just build it and log these parameters using

task.set_parameters()

so that I call

task.get_parameters()

later?

instead of manually calling set/get, you call task.connect(some_dict_or_object) , it does both:
When running manually (i.e. without an agent) it logs the keys/values on the Task,
when running with an agents, it takes the values from the backend (Task) and sets them on the dict/object
Make sense ?

Found the issue, fix in the next RC (soon to be out)

This is something that we do need if we are going to keep using ClearML Pipelines, and we need it to be reliable and maintainable, so I don’t know whether it would be wise to cobble together a lower-level solution that has to be updated each time ClearML changes its serialisation code

Sorry if I was not clear, I do not mean for you ti do unstable low-level access, I meant that pipelines are Designed to be editable externally, they always deserialize themselves.
The only part that is mi...

Hi @<1523701079223570432:profile|ReassuredOwl55> let me try ti add some color here:
Basically we have to parts (1) pipeline logic, i.e. the code that drives the DAG, (2) pipeline components, e.g. model verification
The pipeline logic (1) i.e. the code that creates the dag, the tasks and enqueues them, will be running in the git actions context. i.e. this is the automation code. The pipeline components themselves (2) e.g. model verification training etc. are running using the clearml agents...

Are you doing from keras import ... or from tensorflow.keras import ?

Cool, how will you be using it?

Let me check what we can do 😉

I mean what is the actual link?
File:// is a path to a file.
If your machine cannot access that path you get an error.
For example:
file:///home/user/file.bin
translates to /home/user/file.bin
If you do not have the file /home/user/file.bin on your machine you get an error.
GrievingTurkey78 make sense ?
Note that by default trains / clearml will not upload your weights file anywhere , only if you set "output_uri" to a specific location it will do that .

The dokcer itself does not have the host configured.

the Task scheduler itself is a Task. What we did is we added a new parameter section on the Task (the task.connect call), so that we can later clone and modify it and use the new value in runtime
(Task.connect will put the data from the Task/UI back into the dict when the agent is running the Scheduler)
Does that make sense?

CurvedHedgehog15 there is not need for :
task.connect_configuration( configuration=normalize_and_flat_config(hparams), name="Hyperparameters", )Hydra is automatically logged for you, no?!

the other repos i have are constantly worked on and changing too

Not only it will be cloned automatically, the git diff of the sub-modules are stored as well 🙂

BTW: what's the clearml-server version ?