Reputation
Badges 1
50 × Eureka!Hi AgitatedDove14 , thanks for your reply.
I basically follow the example to use the SummaryWriter to add plots to the dashboard, however, the plots added are Plotly plots and not the tensorboard equivalent.
Oh okay, so you’re saying the UI will always use plotly to display the data? I guess I got confused since the color choices in https://allegro.ai/docs/examples/frameworks/pytorch/pytorch_tensorboard/ made it look like the TensorBoard graphs.
Many thanks! Got to say; You’ve made and are making a killer of a product!
Hi AgitatedDove14 , I am also unable to archive the individual experiments in it.
It's not a major issue, but it would be nice to remove as some users may get confused.
However in this case the option is grayed out
Yes, exactly, from a previously executed run. Essentially, I write a grid of images which are supposed to learn a generative task, i’d like to download all images and generate a gif from the collection of images.
No, but I am able to delete other folder with experiements
Hi AgitatedDove14
Turns out my double loop caused some issues.
for e in range(num_epochs): for phase in ['train','valid']: for batch in dataloader:
Upon removing the phase
loop the epoch was detected automatically again.
I’d like to create an animated gif
Not sure if you've tried this, but would click work? Otherwise the simplest thing I know is possible in Click, is to send the Dict of arguments further down the pipeline for the Allegro callback listener.
I'll see if I can get time to do a reproducible example.
The issue only arises upon sending Images. (Both numpy, mpl and PIL)
Still having the same issue I'm afraid
It seems there is some async behavior going on. After ending a run, this prompt just hangs for a long time:
2021-04-18 22:55:06,467 - clearml.Task - INFO - Waiting to finish uploads
And there's no sign of updates on the dashboard.
And another detail, upon running the same code in a notebook session, everything gets stored as intended (to clearML dashboard)
I see, no i do not get anything. I either get pop-ups when trying to capture the mpl
figure, and otherwise nothing happens when using report_image
.
I should mention this is run within a TF v1 session context
Likely related to: https://github.com/allegroai/clearml/issues/321 - clearML waits indefinitely on logs from TF v1
Scalars plot fine, images do not find their way to the dashboard
I'm manually executing on a local machine
AgitatedDove14 , after some more pronging, the error seem to stem from the clearML server. The upload from client side does not seem to occur, or the server is not registering the uploads.
I've attempted to restart the server and pull the latest image, same error.
Got it to work now. Suspect something to do with a hydra issue, as both clearML hosted nor self-hosted option presented the same issue. Will update as I get a chance.
Yes, I am aware of this. None of the values are being reported, neither the scalar, images in debug-samples, nor in the plots tab.
I am able to register that the image exists, however, the push towards the clearML server just does not happen
I might add, this error only started to occur upon upgrading from trains to clearmL
I'm running in the command line with Hydra and TFv1
Thanks for the rapid reply
Hi AgitatedDove14 - I used the Tensorboard writer.scalars
function. Haven't tried the Plotly natively, but I guess its the same, since I imagine you're just doing a passthrough.