And an example of the missing comparison:
the two experiments 2. plot on the first one 3. plot on the second 4. comparison plot only shows other plots (only the confusion matrices)

0.16.1-320

With matplotlib I only get the suptitle

made with

thanks that works ! perfect

and ctrl-f (of the browser) doesn’t work as lines below not loaded (even when you scroll it will remove the other lines not visible, so you can’t ctrl-f them)

made a PR to help a bit loading console logs None
logs can be huge but are loaded 7kB at a time currently

100+ parameters is quite a lot indeed but very quickly achieved when using frameworks like detectron2, where you configure the model in the configuration (+dataloader, datasets, evaluators, augmentation, optimizer, lr_scheduling). anyway the search is broken as soon as one line you search is not currently visible, so already with 20+ ...

thanks!

Ok thanks I will check first for permission issues

Also it would be awesome if the front-end integrated a small reverse-proxy to have everything on 1 address, I don't know if this is somewhere on the roadmap ? Or what are advantages of having 3 separate addresses ?

Oh ok I thought it would be relative to the server, how do i run this migration ?

for plots also

I welcome the day clearml saves relative urls by default ^^ it is supported by browsers (i.e. fetching /someurl is relative to the current hostname) so maybe only the clearml client would need to be updated right ? to push images with a relative url instead of the clearml server url.

We tried with a docker-compose on a GCE VM + load balancers, and then in kube, we get the same error: clearml-init returns Error: could not verify credentials: key=241... secret=NhC...

I suppose the images are in db.task but I can't find them

we still don't what was happening with the VM + docker compose + load balancers

Is there a way to store relative urls in clearml ? We can't connect to our server with a public address, it only works with the internal dns from GCE

And the comparison for the confusion matrices without the name of the experiments

Hello AgitatedDove14 it does not throw an exception, but in the ui the link is broken so the image does not show

hello, yes it’s like typos, I want to compare some experiments that were created by different versions of a script for instance, and the metrics names changed so I can’t compare it on clearml UI

we managed to upgrade it but the volume claim thing changed somehow, it created new disks, i will backup from the old disks and upload to the new ones to migrate but the backup procedure is not detailed for kubernetes, do you have info for this?
should i only do mongodb?

so if anybody needs this someday (migrating your hostname which is saved inside your experiments (debug images and plots with images)) you need this https://github.com/allegroai/clearml-server/issues/83
but it's slow , you can restrict the query to the items that are actually updated, with:
` # on index events-training_debug_image-yourid

OLDHOST/ should be something like or

NEWHOST/ same

"script": {
"source": "ctx._source.url = ctx._source.url.replace('OLDHOST/', 'NEWHO...

Yes

Thanks ! I think .execute_remotely() is exactly what I need

managed a workaround thanks to the API doc, if someone encouters the same bug:
tasks = [] page = 0 while True: page_tasks = Task._query_tasks(project_name=project, system_tags=[] if archived else ['-archived'], page=page, page_size=500) tasks += page_tasks page += 1 if len(page_tasks) < 500: break

thanks, is there a way to do it with the SDK ? maybe override the page size ?

which is kind of logical

The script is inside a git repo (and it's the one I launch, I would get an importerror if it was something else missing)

not exactly, I want to launch the script (create a new experiment, not clone an existing one in the UI), how can I do it ?

Does clearml-agent install the repo with pip install -e . if it should be ? (i.e. my local repo is installed with pip install -e . where I launch my script which calls Task.init and .execute_remotely() ).