Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Am Trying To Upload A Plot To An Existing Task Using The

Hi, I am trying to upload a plot to an existing task using the old_task = Task.get_task() function and then calling old_task.get_logger().report_matplotlib_figure() but I noticed that it didn't work unless I had another task = Task.init() in the code. Is this normal behaviour (It's given in the documentation here https://allegro.ai/docs/task.html#trains.task.Task.get_task so I'm assuming it is)? If yes, can we just reference an old task and upload to that without creating another experiment/task? Thanks

  
  
Posted 3 years ago
Votes Newest

Answers 30


SmarmyDolphin68
BTW: there is no automatic reporting when you have task = Task.get_task(task_id='your_task_id')
It's only active when you have one "main" task.
You can also check the continue_last_task argument in Task.init , it might be a good fit for your scenario
https://allegro.ai/docs/task.html#trains.task.Task.init

  
  
Posted 3 years ago

Understood, I'll look into it!

My use case is that, let's say I'm training with a file called train.py in which I have Task.init() , now after the training is finished, I generate some more graphs with a file called graphs.py and want to attach/upload to this training task which has finished. That's when I realised Task.get_task() is not working as intended, but it is when I have a Task.init() before it.

  
  
Posted 3 years ago

No, the sample code I sent above works as intended, uploads to 'Plots'. But the main code that I've written, which is almost exactly similar to the sample code behaves differently.

  
  
Posted 3 years ago

BTW: I tested the code you previously attached, and it showed the plot in the "Plots" section
(Tested with latest trains from GitHub)

  
  
Posted 3 years ago

Could you amend the original snippet (or verify that it also produces plots in debug samples) ?
(Basically I need something that I can run 🙂 )

  
  
Posted 3 years ago

trains 0.16.3
trains-server 3.6

  
  
Posted 3 years ago

Are you doing 

plt.imshow

Nope

And yes, I set the report_image=False

  
  
Posted 3 years ago

AgitatedDove14 , thanks a lot! I'll get back with a script in a day or two.

  
  
Posted 3 years ago

SmarmyDolphin68 okay what's happening is the process exists before the actual data is being sent (report_matplotlib_figure is an async call, and data is sent in the background)
Basically you should just wait for all the events to be flushed
task.flush(wait_for_uploads=True)That said, quickly testing it it seems it does not wait properly (again I think this is due to the fact we do not have a main Task here, I'll continue debugging)
In the meantime you can just do
sleep(3.0)And it will solve the issue 🙂

  
  
Posted 3 years ago

What's the trains version / trains-server version ?

  
  
Posted 3 years ago

What's the matplotlib version ? and python version?

  
  
Posted 3 years ago

Okay, my bad, the code snippet I sent is correctly uploading it to 'Plots' but in the actual script I use this:
` def plot_graphs(input_df,output_df,task):

# Plotting all the graphs
for metric in df_cols:


    # Assigning X and Y axes data
    in_x = input_df["frameNum"]
    in_y = input_df[metric]
    out_x = output_df["frameNum"]
    out_y = output_df[metric]

    # Creating a new figure
    plt.figure()

    plt.xlabel('Frame Number')
    plt.ylabel('{} Score'.format(metric))

    # Plotting a simple line graph
    plt.plot(in_x,in_y)
    plt.plot(out_x,out_y)

    # Adding legend
    plt.legend(["Input {}".format(metric),"Output {}".format(metric)])

    # Reporting the plot through Trains Logger API
    task.get_logger().report_matplotlib_figure('Input vs Output {}'.format(metric),'Test',iteration=3,figure=plt,report_image=False)
    # task.logger.flush()
    time.sleep(3.0) `This is the code I'm using, not much different from the snippet. I've used plt.figure() to create a new plot since it's in the for loop, could that be causing this?
  
  
Posted 3 years ago

SmarmyDolphin68

Debug Samples tab and not the Plots,

Are you doing plt.imshow ?
Also make sure you have report_image=False when calling the report_matplotlib_figure
(if it is true it will upload it as an image to "debug samples")

  
  
Posted 3 years ago

Oh! With the sleep() function? Let me try it again

  
  
Posted 3 years ago

Thanks you !
I'll look into it

  
  
Posted 3 years ago

trains-server 3.6

you mean 0.16?

  
  
Posted 3 years ago

SmarmyDolphin68 , All looks okay to me...
Could you verify you still get the plot on debug samples as image with the latest trains RC
pip install trains==0.16.4rc0

  
  
Posted 3 years ago

0.16.1-320

you mean 0.16?

Ah my bad, I picked up the version from docker-compose file :D

  
  
Posted 3 years ago

SmarmyDolphin68 if you can reproduce the behavior in a standalone script , it will really accelerate fixing this issue

  
  
Posted 3 years ago

I tried it, and it is uploading it to Debug Samples ( Task.get_task() ) with task.get_logger().report_matplotlib_figure() , but with a Task.init() , it's uploading it to Plots.

Checking with the RC package now

  
  
Posted 3 years ago

, I generate some more graphs with a file called 

graphs.py

  and want to attach/upload to this training task

Make total sense to use Task.get_task, I just want to make sure that you are aware of all the options, so you pick the correct one for you :)

  
  
Posted 3 years ago

SmarmyDolphin68 is it the same code as in the snippet ?

  
  
Posted 3 years ago

SmarmyDolphin68 What's the matplotlib version ? and python version?

  
  
Posted 3 years ago

It's going to Debug Samples with the RC too

  
  
Posted 3 years ago

Go to https://demoapp.trains.allegro.ai/profile
You should see something like 0.16.2-123

  
  
Posted 3 years ago

Indeed, sleep() did the trick but it's going into the Debug Samples tab and not the Plots, any reason why? Earlier (with Task.init() followed by Task.get_task() ) the same plt plots got reported to 'Plots'.

  
  
Posted 3 years ago

Weird that this code is also uploading to the 'Plots'. I replicated the same thing as my main script, but main script is still uploading to Debug Samples.

SmarmyDolphin68 are you saying the same code behaves differently ?

  
  
Posted 3 years ago

Let me check

  
  
Posted 3 years ago

` from trains import Task
import matplotlib.pyplot as plt
import numpy as np
import time

task = Task.get_task(task_id='task_id')

for i in range(0,10):

x_1 = np.random.rand(50)
y_1 = np.random.rand(50)

x_2 = np.random.rand(50)
y_2 = np.random.rand(50)
plt.figure()
plt.scatter(x_1, y_1, alpha=0.5)
plt.scatter(x_2, y_2, alpha=0.5)
# Plot will be reported automatically
# plt.show()
task.get_logger().report_matplotlib_figure(title="My Plot Title", series="My Plot Series", iteration=i, figure=plt)
time.sleep(6.0) `Weird that this code is also uploading to the 'Plots'. I replicated the same thing as my main script, but main script is still uploading to Debug Samples.

Python - 3.8.5
Matplotlib - 3.3.3
(Using same env to run test script and main script)

  
  
Posted 3 years ago

AgitatedDove14 , here's the code snippet you requested

  
  
Posted 3 years ago