nemocrys/multilog

Measurement data recording and visualization using various devices.

multilog's Issues

Improve Daq6510 configuration

Make channels individually configurable (npc, lsync, ...), currently these settings are global.

change file format of IR images

One csv file takes 1.5 mb per image, a compressed file only needs 0.2 mb. Switching to another data format here would save a lot of disk space.

Rogowski factor

Rogowski_300A factor is x50 (not x100)

Exit button sometimes not working correctly

Sometimes, probably after running for a long time, the exit button is not working correctly. Additional logging could be helpful to track this problem.

Add hide checkbox

Add a checkbox to hide the lines of selected sensors, as it was implemented in multilog v1. This is helpful if there are many sensors in one plot.

Add testing

There are many reasons for testing. I think using pytest is a good idea.

Multilog does not work with Optris camera on Windows

The camera xml file is hardcoded in optris_ip640.py included several fixed paths. These are not correct for windows and should be adjustable, e.g., in the main config file.

More controls to save images and video from cameras

Currently capturing of camera images is automatically started and cannot be changed later. The following features would be very useful without restarting multilog (for each camera separately):

• Adjustment of capture parameters, e.g. framerate, exposure time, infrared temperature range
• Buttons to Stop / Continue image storage
• Interrupt continuous storage of images and store a short video with predefined length and other settings

Replace the SignalMetaclass by class decorator?

See https://realpython.com/python-metaclasses/ and comment in man.py - not sure if that's possible, but it'd increase readability of the code.

Daq6510 refactoring

Re-implement the device-class to make the code more readable and maintainable, e.g. using a separate object for each channel. The current implementation based on multilog1 using various lists is hard to understand / maintain.

Warn if cooling water flow is zero

Check if the cooling water flow is greater than zero when multilog measurement is started. If not, open a warning in a popup window.

Add proper documentation

Docstrings are already present, use them to generate a readthedocs website!

Code and data in separate subfolders

Currently all data is stored within the script folder (clone of Github repository). An option would be useful to set a working directory (e.g., one level up from script folder) for configuration files, measurement data and logs.

Eurotherm: log setpoint

Currently only temperature measurement and operating point are logged. It'd be nice to also log the setpoint "wsp" and the selection op / wsp.

OptrisIP640: add labels showing configuration

Add labels with Emissivity, ... information to the view.

IR Camaera shows values of ~1e14

This could also a device-related problem. Additional logging would be helpful here.

Real-time cooling rates?

Calculate and show cooling rate (W) in various loops in real time under IFM tab?

Requirement for .git repository

Currently program can be started, but pressing "Start" to begin data saving produces an error if the base folder is not a clone of the git repository. I.e., downloading the code from github is not sufficient and git software is needed to update the code on every PC running it. An option to skip this or enter the missing information (program version?) manually would be useful here.

Main config file not clearly written

The config_template.yml is not really clear about which lines exactly should be commented out to deactive a device. Due to the large number of settings and permanent comments at many places, it would be easier to add an enable switch for each device. These could be also used to avoid unnecessary imports of packages, e.g., Optris or Basler.

Fresh install not working

Fresh copy on a new Windows notebook shows an unclear error when clicking on Start, see attachment. It works if the call to write_nomad_file() is commented out in main.py and if Process-Condition-Logger is disabled in the config.

config.txt

Output files not working for specific device setup

The procedures write_nomad_file() and write_metadata() lead to an unspecific FileNotFoundError: [WinError 2] for a minimal device setup on Windows (attached).
config.txt

Process condition

With the plan to transition to digital lab-books in mind, I suggest having the process condition tab as the landing page/tab, so we remember to fill in the info. I also suggest having no default values (or maybe NaN) to avoid confusion in case the fields are not filled in.

Add comment section for all sensors / devices

This can be used to describe the position of the sensor and other notes that used to be in the Lab book.

Send commands for process control

Currently process conditions can be manually entered and only logged in the corresponding tab. An option could be added to send these changes to the devices such as Eurotherm:

• Checkbox next to the input field to activate sending of the changed value
• Extension of the RS232 interface to send the value and verify the response

Generally a separate program could be useful for process control (vs. logging), but this small extension seems reasonable also for the logging part as long as we have this tab here.

Race condition in updating measurement data

There is a race condition for updating of measurement data (but the output is still valid). I think the problem was that the time-list may have a different length than the sampling-list when the view is updated.

Log-file snippet:
2022-09-15 12:49:57,408 WARNING multilog.devices - IFM-flowmeter save_measurement: time difference between event and saving of 1.092403 seconds for samplint timestep 2022-09-15 12:49:56.315+02:00 - 2653.875
2022-09-15 12:50:09,401 WARNING multilog.devices - IFM-flowmeter save_measurement: time difference between event and saving of 1.085731 seconds for samplint timestep 2022-09-15 12:50:08.315+02:00 - 2665.875
2022-09-15 12:50:20,377 WARNING multilog.devices - IFM-flowmeter save_measurement: time difference between event and saving of 2.062282 seconds for samplint timestep 2022-09-15 12:50:18.314+02:00 - 2675.874
2022-09-15 12:50:20,382 ERROR multilog.main - Error in updating view of IFM-flowmeter
Traceback (most recent call last):
  File "/home/nemocrys/Documents/experiments/meas-multilog/multilog/main.py", line 175, in update_view
    self.tabs[device].set_measurement_data(self.rel_time, device_sampling[device])
  File "/home/nemocrys/Documents/experiments/meas-multilog/multilog/view.py", line 506, in set_measurement_data
    self.temperature_widget.set_data(sensor, rel_time, meas_data["Temperature"][sensor])
  File "/home/nemocrys/Documents/experiments/meas-multilog/multilog/view.py", line 388, in set_data
    self.lines[sensor].setData(x, y, connect=np.logical_and(con, np.roll(con, -1)))
  File "/usr/local/lib/python3.8/dist-packages/pyqtgraph/graphicsItems/PlotDataItem.py", line 595, in setData
    self.updateItems( styleUpdate = self.property('styleWasChanged') )
  File "/usr/local/lib/python3.8/dist-packages/pyqtgraph/graphicsItems/PlotDataItem.py", line 644, in updateItems
    self.curve.setData(x=x, y=y, **curveArgs)
  File "/usr/local/lib/python3.8/dist-packages/pyqtgraph/graphicsItems/PlotCurveItem.py", line 365, in setData
    self.updateData(*args, **kargs)
  File "/usr/local/lib/python3.8/dist-packages/pyqtgraph/graphicsItems/PlotCurveItem.py", line 419, in updateData
    raise Exception("X and Y arrays must be the same shape--got %s and %s." % (self.xData.shape, self.yData.shape))
Exception: X and Y arrays must be the same shape--got (1340,) and (1339,).
2022-09-15 12:50:21,453 WARNING multilog.devices - IFM-flowmeter save_measurement: time difference between event and saving of 1.138052 seconds for samplint timestep 2022-09-15 12:50:20.315+02:00 - 2677.875
2022-09-15 12:50:23,385 WARNING multilog.devices - IFM-flowmeter save_measurement: time difference between event and saving of 1.069698 seconds for samplint timestep 2022-09-15 12:50:22.315+02:00 - 2679.875
2022-09-15 12:50:27,376 WARNING multilog.devices - IFM-flowmeter save_measurement: time difference between event and saving of 1.061414 seconds for samplint timestep 2022-09-15 12:50:26.314+02:00 - 2683.874
2022-09-15 12:50:35,380 WARNING multilog.devices - IFM-flowmeter save_measurement: time difference between event and saving of 1.065411 seconds for samplint timestep 2022-09-15 12:50:34.315+02:00 - 2691.874

Sampling time was 2s.

Add datetime axis to the plots

As was implemented in multilog v1, see empty function calc_x2_ticks() in view.py.

Tile view for tabs

Clicking between the many tabs during a busy experiment is quite distracting. A tile view, where several tabs (especially the process data input!) can be shown simultaneously, would help here.

Save screenshots on exit

The graphical interface contains quite advanced plots of measurement data and an overview of configuration information. It would be useful to store screenshots of all tabs on Exit.

Update nomad files to new structure

See description in FAIRmat-NFDI/AreaA-data_modeling_and_schemas#3 and templates in https://github.com/FAIRmat-NFDI/AreaA-data_modeling_and_schemas/tree/main/melt_czochralski_Dadzis

config_template.yml missing

This was not included when multilog2 was published.

Changes required to multilog for new nomad plugin

• file extensions *.multilog.csv
• move metadata that is now in the old archive.yaml files to a more straightforward format
• write a script to reprocess old measurement data and move it to the new format