[feature request] Record a plot to a image

[balbok0]

[feature request] Record a plot to a image

[kushalkolar]

Hi! Thanks for posting an issue! 😄 That's definitely the types of use case that inspired fastplotlib in the beginning!

Do you bacially want to save the data of an image graphic to a PNG? Can you post a screenshot or example of what you're looking for?

[balbok0]

Hi, here is a screenshot of the plot I'm conceptually looking for:

I would prefer to save it as png, but other popular image formats should work too (jpeg etc.).

Here is a relevant code:

import fastplotlib as fplt
import numpy as np

grid_plot = fplt.GridPlot((2, 3))

# Image
img = (np.random.random((720, 1280, 3)) * 255).astype(dtype=np.int8)
grid_plot[0, 0].add_image(data=img)

# Predicted Image with features over-layed on top
pred_img = (np.random.random((200, 200)) * 255).astype(dtype=np.int8)
grid_plot[0, 1].add_image(data=pred_img)
grid_plot[0, 1].set_title("Pred Image with features over-layed on top")
grid_plot[0, 1].add_line(data=np.arange(10, 60).reshape(-1, 2), colors="yellow")
grid_plot[0, 1].add_line(data=np.random.rand(1, 2) * 200, colors="red")

# PDF, but there is no histogram.
# Implemented as a heatmap, with color serving the scaling purpose
probs = np.linalg.norm(np.random.rand(2, 1), axis=1, keepdims=True)
grid_plot[0, 2].add_heatmap(data=probs, vmin=0, vmax=1.0)
grid_plot[0, 2].set_title("Cost Histogram")


# Predicted line with more features
grid_plot[1, 0].set_title("Pred Line with bounds")
x = np.arange(1, 200)
y = np.sqrt(x)
dy = 3 * np.log(y)
grid_plot[1, 0].add_line(np.array([x, y]).T, colors="blue")
grid_plot[1, 0].add_line(np.array([x, np.maximum(0, y - dy)]).T, colors="blue", alpha=0.5)
grid_plot[1, 0].add_line(np.array([x, y + dy]).T, colors="blue", alpha=0.5)

# Show angles. In matplotlib this was also implemented as histogram/bar chart.
grid_plot[1, 1].add_heatmap(data=np.random.rand(2, 1) * 90, vmin=-45, vmax=45)
grid_plot[1, 1].set_title("Angles")

grid_plot[1, 2].add_heatmap(data=np.random.randn(9, 1), vmin=-25, vmax=25)
grid_plot[1, 2].set_title("More distributions")

# Here I would like to
# grid_plot.render() # <- Optionally, or some other call
# grid_plot.save_to_file(file_path)  # file_path should accept str and pathlib.Path types preferably

grid_plot.show()

# Small note: It's weird that there is discrepancy between arguments here: (width, height)
# and add_image function data shape (height, width)
grid_plot.canvas.set_logical_size(1280, 720)

for subplot in grid_plot:
    subplot.auto_scale()

fplt.run()

Overall, there are a few more features that are needed for me to use this library (unrelated to this issue, but most of them are listed on this roadmap: #55), but just wanted to call this one out, since I haven't noticed it being called out anywhere.

[kushalkolar]

Thanks for the screenshot and example! If you're in jupyter and you want to save the entire canvas as a png you can use plot.canvas.get_frame(). It returns an RGBA array of the jupyter WGPU canvas.

import imageio.v3 as iio

iio.imwrite("/path/to/img.png", plot.canvas.get_frame())

However it looks like you're using glfw so for now you'll need to create and offscreen canvas and renderer which can save individual subplots to pngs.

from wgpu.gui.offscreen import WgpuCanvas
import pygfx

import imageio.v3 as iio

# create offscreen canvas and renderer
os_canvas = WgpuCanvas(size=(1000, 1000))
sub_renderer = pygfx.WgpuRenderer(os_canvas)

# iterate through subplots
for subplot in grid_plot:
    # draw the scene onto the offscreen canvas
    os_canvas.request_draw(lambda: sub_renderer.render(subplot.scene, subplot.camera))

    pos = subplot.position

    # write to png
    iio.imwrite(f"./{pos[0]}_{pos[1]}.png", np.asarray(os_canvas.draw()))

I'm not sure how to do it with the entire canvas and pygfx plans to refactor the viewport and renderer system soon so I will wait until then to implement this properly.

pygfx does have an svg renderer but it's not complete, see #238 . I haven't reported it upstream yet since there are many other low-level priorities.

grid_plot.render() # <- Optionally, or some other call

Just note, render(), start_render() etc. will be made private soon, show() takes care of setting up the rendering and this is what users should use.

We currently don't plan to prioritize plots such as histograms and bar plots since they usually don't require high performance rendering like image, lines, or scatters. Do you know of a use case for high performance histograms?

Small note: It's weird that there is discrepancy between arguments here: (width, height)
and add_image function data shape (height, width)
grid_plot.canvas.set_logical_size(1280, 720)

Yup as far as I'm aware in most libraries images are [n_row, n_cols] by convention, whereas canvas sizes are (width, height).

there are a few more features that are needed for me to use this library (unrelated to this issue, but most of them are listed on this roadmap: #55

If you can let us know which features are required it can help us prioritize the most wanted features from users! 😄 I'm also happy to help out if you want to do a PR. Fast prototyping with ML pipelines and algorithm development is our main internal usecase for fastplotlib.