post_optional_packages: ["google-cloud-storage", ]Will install it last (i.e. after all the other packages) but only if you have it in the "Installed packages" list

I can definitely feel you!
(I think the implementation is not trivial, metrics data size is collected and stored as commutative value on the account, going over per Task is actually quite taxing for the backend, maybe it should be an async request ? like get me a list of the X largest Tasks? How would the UI present it? As fyi, keeping some sort of book keeping per task is not trivial either, hence the main issue)

PlainSquid19 yes the link is available on in the actual paid product 😞
I don't think they have the documentation open yet...
My recommendation is to fill the contact us form, you'll get a free online tour as well 😉

What's the log you are seeing?

Yep it should :)
I assume you add the previous iteration somewhere else, and this is the cause for the issue?

You mean why you have two processes ?

DilapidatedDucks58 trains-agent adds the artifactory URL as --extra-index-url , are you sure you are getting the correct torch version in the container? because the torch html is not an artifactory html, it is a list of links, I just want to make sure you are getting the correct version, because otherwise it can default to the CPU version, which we don't want 🙂 anyhow you can use the direct link in the "installed packages and just put there " https://download.pytorch.org/whl/nightly/cu101...

can configuration objects refer to one-another internally in ClearML?

Interesting, please explain?

BTW: the new documentation should contain a full search over the docstring

None
The thing is the server will not return a 502 error, only a 500 error,
None
Could it be the k8s ingress maybe ?

ohh, not really 😞 this is really low level editing the DB.
You might be able to forcefully edit the links (i.e. artifacts) on the Dataset (task)
Check if this works

from clearml.backend_api.session.client import APIClient
c = APIClient()
t = c.tasks.get_by_id("DATASET_UUID_HERE")
# you might need to loop over the artifacts
t.data.execution.artifacts[0].uri = "

"
c.tasks.edit(task=t.id, execution=t.data.execution, force=True)

Just to clarify, where do I run the second command?

Anywhere just open a python console and import the offline task:
from trains import TaskTask.import_offline_session('./my_task_aaa.zip')

Related, how to I specify in my code the cache_dir where the zip is saved?

This is the Trains cache folder, you can set it in the trains.conf file:
https://github.com/allegroai/trains/blob/10ec4d56fb4a1f933128b35d68c727189310aae8/docs/trains.conf#L24

Hi @<1624941407783358464:profile|GrievingTiger47>
I think you should try to contact the sales guys here: None

Alright I have a followup question then: I used the param --user-folder “~/projects/my-project”, but any change I do is not reflected in this folder. I guess I am in the docker space, but this folder is not linked to my the folder on the machine. Is it possible to do so?

Yes you must make sure the docker can mount a persistent folder for you to work on.
Let me check what's the easiest way to do that

How do I tell from the ClearML UI which datasets version am I using?

Hi SubstantialElk6 , what exactly do you mean by "ClearML UI which datasets am I using" ? Do you mean is there an auto magic adding the dataset ID when you call Data.get() in your code ? (because if you are I specifically remember discussing adding this feature a few days ago, which you just bumped the priority of 😉 )

UnevenOstrich23

but interesting that auto-reload config does not working as I expected.

Unfortunately the trains-agent does not support auto reloading the config file yet. If you think this will be a great feature, please feel free to open a GitHub feature request issue 🙂

Could you send the "installed packages" section of the Task that was created in the notebook ?

What I'd really want is the same behaviour in the console (one smooth progress bar) and one line per epoch in the logs; high hopes, right?

I think they send some "odd" character instead of CR, otherwise I cannot explain the difference.
Can you point to a toy example demonstrating the same issue ?

Also I just tried the pytorch-lightning 

RichProgressBar

 (not yet released) instead of the default (which is unfortunately based on tqdm) and it works great.

Yey!

Hmm that is a good question, are you mounting the clearml.conf somehow ?

Hi MagnificentSeaurchin79
Unfortunately there is currently no way to reorder the plots, but you have a valid point. I suggest a GitHub UX issue ?
Regrading the debug samples, the difference is that the confutation matrix report is actually metadata, you can get these numbers by the API or the download, but the debug samples are static images ...

BTW: you can try to produce an interactive side by side confusion matrix with plotly, and use report_plotly_figure

What are you getting with:

curl http://<ip>:8008/auth.login

from what I gather there is a lightly documented concept

Yes ... 😞 the reason for it is that actually one could do:
` @PipelineDecorator.pipeline(...)
def pipeline(i):
....

if name == 'main':
pipeline(0)
pipeline(1)
pipeline(2) `Basically rerunning the pipeline 3 times
This support was added as some users found a use case for it, but I think this would be a rare one

CourageousLizard33 column order / specific selection is stored per user. If you press the share button you will have a link with all the definitions embedded on it.
Column resizing and order is in the next version release :)

Queues can have multiple workers, and that implies multiple instances of a task can run concurrently.

@<1533619716533260288:profile|SmallPigeon24> as long as these are the Exact same instances you can have them runing simultaneously (think multi node training), that said each one should "know" not to report over the others, because of course it will overwrite the reports.

Back to your point on multiple agents:
You cannot have two Tasks in the same queue, that means that a single agen...

Hi JuicyOtter4
Seems like you cannot access your sever ?

PompousParrot44
It should still create a new venv, but inherit the packages from the system-wide (or specific venv) installed packages. Meaning it will not reinstalled packages you already installed, but it will ive you the option of just replacing a specific package (or install a new one) without reinstalling the entire venv

Hi PunyGoose16 ,
I think the website is probably the easiest 🙂
https://clear.ml/contact-us/
I think they get back to quite quickly

Anyway, in the docs, there is a function called task.register_artifact()

Yes, this is rather deprecated... The idea is that it will monitor an obejct and auto sync it (i.e. serialize and upload).
That said, it is just so much easier to do task.upload_artifact and you can always update/overrwrite if you are passing the same name, that I cannot see the actual use case. Does that make sense? What are you using it for ?

WickedGoat98
Put the agent.docker_preprocess_bash_script in the root of the file (i.e. you can just add the entire thing at the top of the trains.conf)

Might it be possible that I can place a trains.conf in the mapped local folder containing the filesystem and mongodb data etc e.g.

I'm assuming you are referring to the trains-=agent services, if this is the case, sure you can,
Edit your docker-compose.yml, under line https://github.com/allegroai/trains-server/blob/b93591ec3226...

Hi @<1569496075083976704:profile|SweetShells3>
Try to do:

import torch.distributed as dist

if dist.get_rank()==0:
  task = Task.init(...)

This will make sure only the "master" process is logged

or

if int(os.environ.get('RANK'))==0:
  task = Task.init(...)