Thank you 😅 Very much

Upon removing the phase loop the epoch was detected automatically again.

Ahh excellent! Thanks AgitatedDove14 🙂

And another detail, upon running the same code in a notebook session, everything gets stored as intended (to clearML dashboard)

I guess I could do a backup of the DB and flush the data, but what I’m looking for is more of a “Select X experiments -> Send to blob storage” to free up space.

I am able to register that the image exists, however, the push towards the clearML server just does not happen

AgitatedDove14 , after some more pronging, the error seem to stem from the clearML server. The upload from client side does not seem to occur, or the server is not registering the uploads.

I've attempted to restart the server and pull the latest image, same error.

I see, no i do not get anything. I either get pop-ups when trying to capture the mpl figure, and otherwise nothing happens when using report_image .

I should mention this is run within a TF v1 session context

Hi AgitatedDove14 , I am also unable to archive the individual experiments in it.

It's not a major issue, but it would be nice to remove as some users may get confused.

No, but I am able to delete other folder with experiements

Can I force the login window?

A lot of bad requests and connection refused

That’s embarassing

And now it works

Many thanks! Got to say; You’ve made and are making a killer of a product!

Yes, the login details to the Trains UI

Oh god

Hi Martin, thanks for the swift response.

Yes, the artifacts, as backing up the full database would not resolve the question of capacity. Unless I’m missing something

Hang on, It was the static - No auto update AMI:
ami-0d6f44a1a7145a9f8

Oh okay, so you’re saying the UI will always use plotly to display the data? I guess I got confused since the color choices in https://allegro.ai/docs/examples/frameworks/pytorch/pytorch_tensorboard/ made it look like the TensorBoard graphs.

Not sure if you've tried this, but would click work? Otherwise the simplest thing I know is possible in Click, is to send the Dict of arguments further down the pipeline for the Allegro callback listener.

I'm afraid I just had an improper indentation somewhere, no harms done. Thanks for the reply.

Hi AgitatedDove14 , thanks for your reply.

I basically follow the example to use the SummaryWriter to add plots to the dashboard, however, the plots added are Plotly plots and not the tensorboard equivalent.

Hi SuccessfulKoala55 , I believe I was only given one option in my region (EU Stockholm) which was the 0.16.1 version with the AMI location:

aws-marketplace/allegroai-trains-server-0.16.1-320-273-c5c210e4-5094-4eb9-a613-a32c0378de31-ami-06f5e9f4dfa499dca.4

I used the Trains AMI, and I am not sure whether it was the auto-updated or static one

I just changed the login details prior the restarting of the server, could this be at fault? I changed the trains-user to a different one

Still having the same issue I'm afraid

Likely related to: https://github.com/allegroai/clearml/issues/321 - clearML waits indefinitely on logs from TF v1