bluesky/bluesky-mpl

ARCHIVED. See instead http://blueskyproject.io/bluesky-widgets

Bluesky — An Experiment Specification & Orchestration Engine

Source	https://github.com/bluesky/bluesky
PyPI	pip install bluesky
Documentation	https://bluesky.github.io/bluesky
Releases	https://github.com/bluesky/bluesky/releases

Bluesky is a library for experiment control and collection of scientific data and metadata. It emphasizes the following virtues:

• Live, Streaming Data: Available for inline visualization and processing.
• Rich Metadata: Captured and organized to facilitate reproducibility and searchability.
• Experiment Generality: Seamlessly reuse a procedure on completely different hardware.
• Interruption Recovery: Experiments are "rewindable," recovering cleanly from interruptions.
• Automated Suspend/Resume: Experiments can be run unattended, automatically suspending and resuming if needed.
• Pluggable I/O: Export data (live) into any desired format or database.
• Customizability: Integrate custom experimental procedures and commands, and get the I/O and interruption features for free.
• Integration with Scientific Python: Interface naturally with numpy and Python scientific stack.

Bluesky Documentation.

The Bluesky Project enables experimental science at the lab-bench or facility scale. It is a collection of Python libraries that are co-developed but independently useful and may be adopted a la carte.

Bluesky Project Documentation.

See https://bluesky.github.io/bluesky for more detailed documentation.

bluesky-mpl's Issues

LiveScatter support `seq_num` as key

Sometimes it would be nice to plot things against sequence number in a scatter plot

Expected Behavior

LiveScatter('seq_num', 'temperature', 'var') works

Current Behavior

The x = doc['data'][self.x] code precludes this since seq_num is not part of the data.

Possible Solution

1. Have a cutout for popular non-data keys (seq_num, time, etc.)
2. Require a full data address ('data', 'key_name') which would also support ('seq_num', )

Context

This would be helpful for plotting analyzed Quantities of Interest.

Proposed API for Best-Effort Callback successor

The name

"Best-Effort Callback" is not very descriptive from a user point of view. Since this is, in practice, about creating and managing matplotlib figures based on documents, let's just call it Viewer.

Structure

The Viewer has multiple "outer tabs". Each outer tab can be used to (over-)plot one or more runs. The plots reside in one or more "inner tabs", contained by an outer tab.

Proposed API

Viewer is a callable that can be used directly as a callback for consume streaming live data

RE.subscribe(viewer)

or saved data.

for name, doc in run.canonical(fill='yes'):
    viewer(name, doc)

A method that understands the API of run is available as a terse way to display saved data.

viewer.add_run(run)

By default all of the above target the currently-active outer tab. It is also possible to address a specific outer tab, to explicitly control what is plotted over what.

viewer.add_tab(name)  # must be unique or ValueError is raised

A tab can be accessed by name

viewer.tab[name].add_run(run)

or by position.

viewer.tab[0].add_run(run)

A tab may be renamed.

viewer.tab[name].rename(new_name)  # must be unique

Make the colorbar limits interactively adjustable.

'dark mode' for blueksy browser

I am suggesting adding a 'dark mode' to the bluesky-browser, as they tend to be easier on the eyes. The plan is that this will be hot-swappable with 'light mode' via a drop-down menu item.

An example screenshot (of a different QT GUI) is included below to give an indication of what it will look like.

One use case outlined

Quoting an email I don't want want to lose track of:

I am converging on an outline of how I would like to interact with scattering scans through the bluesky browser. It might be that what I'm suggesting is beyond the intended scope of a bluesky-browser, but it is what we need for a data-browser. It might be that all I am asking for is a lot of BEC customization. It seems like it would be generally useful for most beamlines.

1.) All scans in the list should be able to display some arbitrary / definable metadata (sample name / configuration) detector(list) and motor(list) and number of datapoints collected to allow easy browsing (in addition to the scan id)

2.) When one scan is selected, in addition to what is currently shown, it would be excellent to have a second listbox of all of the datapoints (labeled by the active motor positions), and a third listbox showing the other primary stream components. These should both be multiple select mode.

2a.) For the primary stream components list box, clicking on one of them should graph it out (a drop down list for which component to use as the x-axis should be available as well) multiple selections should plot them on interdependent Y axes vs the same X-axis. 1D datasets which are in the stream should be displayed in a waterfall graph by default. 2D or higher components should not be shown in this list

2b.) For the datapoint list box, selecting one should display the higher dimensional images for that datapoint. This is the view where I will want to add some interactive definition of masks, beam centers, and integrations - data reductions which are probably beyond bluesky browser. The reduced data (to 0 or 1D data) that I define here should then be added to the listbox in the primary stream list, so that they can be plotted for all datapoints. It is important to allow some labeling of datapoints which should not be displayed in the primary component graphs.

So the user could quickly survey the data, pick the ranges of datapoints which are interesting, the data reduction method, and then quickly see this reduction applied to the whole scan. Once these settings are determined for one scan, it would be excellent to easily apply that data reduction to other scans and live data as well.

Image Viewer

@EliotGann was kind enough to share some sleek image-viewing code of his that we should incorporate in bluesky-browser when we are ready. I'm creating an issue with the link here so we keep track of it: https://github.com/NSLS-II-SST/profile_collection/blob/master/startup/81-plotting.py

@tacaswell has suggested that some of this (the zoom-in-and-see-intensity-annotations part) should be pushed upstream into matplotlib itself.

Add config file search path

Current bluesky-browser looks for config only in the current working directory where it is launched. It should search in standard locations. Previously we have written search-path logic ourselves (see databroker) but we should consider using https://github.com/ActiveState/appdirs.

Log vs Linear Norms for LiveImage

It would be nice to have a toggle between log and linear norm for LiveImage.

Expected Behavior

Press button and plot changes scale

Current Behavior

Scale must be set at instantiation

Context

This kind of functionality is available from other software (eg fit2d)

Enable user to change what is plotted interactively.

The axes inferred from the hints should be merely a default.

Show the most recent N traces on over-plotted figures.

Best effort callback should clear plots after a certain amount of draws.
for ex:
BestEffortCallback(maxplots=10)
should clear the figure after 10 plots.

The issue we're experiencing at IOS is that after a certain number of plots the best effort callback crashes.

ValueError: bottom cannot be >= top

Error happens at this line (then goes on to matplotlib)
https://github.com/NSLS-II/bluesky/blob/master/bluesky/callbacks/best_effort.py#L348

(fig.tight_layout())

when the issue occurs, the plots also seem to be way oversized and y axis inverted.

BEC: do not plot a channel against itself.

Discovered at IOS with the ramp plan, one of the live plots in the ramp_plan is showing the ramped value against the ramped value.

Likely someplace around https://github.com/NSLS-II/bluesky/blob/ad11d1582a3b9f091d3eb69bac23007e29f0ae3b/bluesky/callbacks/best_effort.py#L240

Waterfall plots

See bluesky/bluesky#930

Monitors & custom math

Thanks for the demo which is developing nicely! I have a couple of questions/suggestions

1. Plotting signal divided by monitor versus scanned motor or time is a very common requirement. It's not obvious that this is easily done at the moment. I guess this is in the pipeline?

2. What type of basic plot manipulations are envisaged?

In the Image class, give the colorbar and image axes "names"

I refer to whatever "name" shows up when you click the matplotlib canvas toolbar settings button and asks you to choose an axis. It currently says "anonymous" and the Python object id.

@tacaswell -- Any hints?

Image stack viewer

Add an image stack viewer. Features should include:

• scrubber for moving through time
• UI (checkbox?) for setting the scrubber to automatically follow the most recent frame, as new frames may be added during streaming from live acquisition

This can build on the work in https://github.com/bluesky/bluesky-browser/blob/master/bluesky_browser/viewer/image.py