silab-bonn/irrad_control

Control and data aquisition / visualization software for the irradiation site at HISKP cyclotron

irrad_control's Issues

Let ZMQ close/terminate sockets/contexts on it's own

Manually terminating a ZMQ context can lead to stuck code due to sockets having a non-empty in/output queue when the context termination i called (see here). ZMQ calls all needed close/term methods for messages, sockets and the context correctly on its own, when the programm exits. Therefore, to reduce the probability of stuck code, the manual closing should stop

Bug fixes for #27

• Crash due to monitor tab/converter if no ADC is selected, see #30 and a299b68
• Setub tab of server is not removed when checkbox is unchecked in setup_tab, see #31 and 0cc8b7e
• Converter crashes on clean-up due to missing 'positions' field in XY-stage configuration, see #33 and 48c9ccd
• Command tab crashes on adding new positions due to missing 'positions' field in XY-stage configuration, see 0683f2b
• Error due to identically labeled channels, see #29, fixed in 4780f98
• Starting a scan should also resume the data recording if it was previously paused, see e8037db
• Disable the pause button in the DAQ dock when a scan is started and it is enabled, see e8037db
• Possible regression in fluence histogram when showing the mean fluence values, dealt with in 86c3e6c

Add scan object to handle scanning of DUTs

Instead of putting the scan code into the XY-Stage, it is more generic to have a DUTScan-object which takes a reference to the 2D motorstage as well as information on the DUT geometry, beam properties, etc. and conducts the scan. This way the scan is independent of the XY-Stage implementation

Crash if no known servers are found

irrad_control crashes after start if none of the previously known server IPs responds due to not handling an empty list as return value see here

Ditch Travis for Github Actions for CI

Travis does not work anymore since June 2021. Move to Github actions

Add watchdog to be aware of dying server processes

Add watchdog thread in order to inform about unexpected dying of processes and or entire servers by using the functionalities already implemented in process manager

Stop killing server process like an uncivilized buffoon

https://github.com/SiLab-Bonn/irrad_control/blob/master/irrad_control/utils/server_manager.py#L91

Enable storing and loading of setup tab content

Github pages

The webpage looks good so far but there is still stuff to be done.

• Remove the text which is still remaining from here which also causes the double heading
• Remove one of the setup pics
• Add 1 plot showing the beam profile measurement with BLM
• Edit the table: we only need specifics regarding the proton irradiation: 14 MeV (~12.5 MeV on DUT) beam current 20 nA - 2 uA, typical beam width 4/6 mm x/y, temperature at DUT ~ -20 to -40 °C, also add max. dimensions of the DUT e.g. 19cmx11cm, max DUT thickness (Si) 300 um
• Put the table after the setup
• Beam diagnostics should come at the very end
• Fluence estimation should be Fluence determination.
• Explain that we measure the Ip with 20 to 200 Hz within each row of a scan and get DeltaY and v from the XY Stage.
• Add the newest plot for Hardness factor and make own heading "Proton hardness factor". Say we used "thin (200 um) LFoundry test structures"
• Generally, the figures should all be centered
• Please re-read the text for spelling mistakes ;)
• Make pictures clickable
• Optimize the body width

Add feature to restart server

The RaspberryPi server controlling the setup is currently operating close to the irradiation setup, inside a lead / poly brick bunker. Due to the high radiation environment during extraction, the server experiences SEE which causes a kernel crash. The server needs to be power cycled to operate again. Currently this means that irrad_control on the DAQ PC must be restarted as well which means a new output file is created. It would be easier to reconnect via paramiko and send the previous server config (including zmq ports) and restart the server.

Enable storing the plots as e.g. PNG in a screenshot-like manner

Add method to store the IrradPlotWidgets plot into a common graphics file format e.g. as PNG. This method can be connected to a button in the PlotWrapperWidget which allows to easily store the current plot

Producer for replaying previously recorded data

Enable reading previsously recorded irradiation data and replay it

Auto-switching of readout scales

With #48 , the custom readout board will be supported, which allows to switch readout scales on-the-fly. Using this, it's easy to switch the scales automatically if e.g. 95% (switch to larger scale) / 15% (switch to smaller scale)of the current scale is reached.

Setub tab of server is not removed when checkbox is unchecked in setup_tab

Interpreter stores same data multiple times

If interpreter receives data from more than 1 server, data can be stored multiple times which is not intended (see here)

Improvements for #27

• Storing stage positions in command tab, see #9 and f4a9ef5
• Automated Calculation of Time-Average and Standard-Deviation within Timeframe, see #24 and 814d288
• Workflow Enhancement: Record Button, see #25 and 25e43ca
• Optimize launch order and callback of sub-threads and -processes, see #26 and 8b22ad9
• Improve processes for interpreter/server, see #28 and d121a05
• XY-Stage behaviour: stage should have method to pass a zmq.Context() instance to, in order to publish data, see 211007a
• XY stage should be aware of the movements and write them down in a file, see 211007a
• If no special answer is needed, the server / stage should answer to each command with a generic answer containing various infos, see movement_tracker in 211007a
• Separate Worker class into ThreadWorker and QtWorker in order to use it to get traceback in threads of server

Unittests

Add some tests for data interpreter etc.

Improve processes for interpreter/server

• Enable ZMQ to select free ports instead of hardcoding ports

Use bind_to_random_port to let pyzmq handle the allocation of free ports. Introduces more stability to to the fact that hard-coded port numbers depend on the individual system and may be free or not on other systems. This also relates to

• Write PID-file in config to keep track processes and the ports they use

Bugs

• TypeError caused here by missing comma in tuple: e90c5ac
• Converison of sum voltage to current is missing a factor of 4 (see here): 3a7fe17
• irrad_control can be closed while xy-stage hasn't send its last reply. This causes a crash of the server and therefore should not be possible: 45bcf61
• Correctly receiving log messages from all processes: c4d32d6
• Prevent pyqtgraph from crashing by setting InfiniteLine coordinates far outside viewbox when position determination is done without beam: 9e4123e
• Don't return None when ZeroDivisionError but 0: 9e4123e
• Setup crashes when sampling rate of 2.5 SPS is selected in setup: c45a619
• Call the interpreter subprocess with the same python executable that runs the main; causes .desktop file to not work right now: 75672b1

Optional writing data to ASCII-file

Enable optional writing data to txt/CSV-file in addition to HDF5 format

Clean-up of `rad_monitor` branch

@christian235 please clean up the redundant files in the rad_monitor branch so the folder structure looks like this:

• rad_monitor:
  -> __init__.py
  -> iseg_hv.py
  -> arduino_freq_counter.py

  • ArduinoFreqCount:
    -> ArduinoFreqCounter.ino

    • FreqCounter:
      -> The files of the FreqCounter library (utils, examples, ...)

Also, make sure to test and comment ;)

Enable pausing and resuming data writing to file

Add functionality to pause the data recording without stopping the interpreter

Interpreter overwrites xy_stage_stats config file skeleton

A copy of this skeleton (created here) should be used instead see here

Automated Calculation of Time-Average and Standard-Deviation within Timeframe

An automated calculation of the time-average and standard deviation of all selected channels within the displayed timeframe would be nice.

Add shortcut to dump output data in setup tab

Often the output data of irrad_control is unwanted since it is only launched to adjust the beam or move the setup. Therefore a shortcut, in the form of a button, should be added so the output files are not stored permanently.

Refactor of Arduino sketches

When using Arduino sketches, avoid using any Serial API which uses a timeout to wait for data from the 64 byte serial buffer. Rather use readBytesUntil and define an end byte (e.g. \n) to escape the waiting. General refactor of sketches

Add offset compensation for individual readout scales

When changing the readout scale, the signal offset (or noise) also varies. Therefore, the offsets should be determined for each scale in order to subtract the correct offset when switching readout scales

Interpreter crash if output file contains whitespaces

Interpreter crashes / doesn't start if output file path contains whitespaces

Optimize launch order and callback of sub-threads and -processes

Currently the sub processes and threads are launched on init of the main window. There should be a launch order with respect to the dependencies among the process with callbacks which launch each subsequent process to avoid errors e.g.

• Launch server -> successful server launch -> launch interpreter -> successful interpreter launch -> launch listener/handler threads

Error due to identically labeled channels

irrad_control does not seem to be able to close itself "elegantly" when you name two (temperatur) channels with identical lables.

It this case, two temperature sensors on a connected Arduino were accidentally named "t1" which resulted in the following bash output:

Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/irradiation/miniconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/irradiation/miniconda2/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/irradiation/git/irrad_control/irrad_control/utils/daq_proc.py", line 356, in recv_cmd
self.handle_cmd(**cmd_dict)
File "/home/irradiation/git/irrad_control/irrad_control/converter.py", line 480, in handle_cmd
self._start_interpreter(data)
File "/home/irradiation/git/irrad_control/irrad_control/converter.py", line 467, in _start_interpreter
self._setup_daq()
File "/home/irradiation/git/irrad_control/irrad_control/converter.py", line 178, in _setup_daq
self.temp_data[server] = np.zeros(shape=1, dtype=temp_dtype)
ValueError: field 't1' occurs more than once

Converter crashes on clean-up due to missing 'positions' field in XY-stage configuration

Plotting performance

Currently, all plots are updated whenever data comes in. For fast datastreams this is very heavy-duty load on GPU/CPU. Therefore, plotting should have fixed intervals with which updates happen e.g. between 10 Hz and 60 Hz.

Scan not working correctly in development

Storing stage positions in command tab

Enabling storing stage positions of devices (e.g. screen) per session

Workflow Enhancement: Record Button

To increase the efficiency during measurements it would be nice to have a small "Rec"-toggle-button somewhere on the GUI of the raw data visualisation (maybe on the beam parameter visualization too) to toggle between the data recording and non recording status.

Enable setting travel range of XY-stage in control tab

Add data_logger.py enabling direct command line usage of the ADC

Adding data_logger.py script, enabling to use the ADC directly from the command line for data logging

Improvements of irrad_control

This lists the major improvements of irrad_control on the development branch which will replace master on release of version 1.0.1. The entire package undergoes a major restructuring which is not explicitly referenced here

• Better logging with separation of infos, warnings and erros, etc: 763d425
• R/O scales of R/O electronics for each channel individually to enable use of to different electronics: 1562259
• Separate interpreter process from Qt GUI main thread by using subprocess: 9e4123e
• Enable naming individual servers according to their task or location: d0d520f
• Much improved server setup; don't copy all the files but rather install from repo. also allow to configure server: a7a4354
• Add temperature measurement using Arduino Nano and NTC resistors: 0706134
• Improve RawDataPlot to be able to switch between Volt and Ampere as units: 046979f

Crash due to monitor tab/converter if no ADC is selected

Due to this line

Add TID calculation

Add TID calcultion for delivered fluence in data as well as plotting. TID is just a function of fluence and stopping power of respective proton energy in Si

Add button to cancel offsett of ADC in DaqInfo widget

Move this button from control tab to DaqInfoWidget due to now having multiple server support

Fix plotting

Plotting of temperatures is missing in master and development

Store motorstage data

For some setup it is useful to know the stage position corresponding to a given timestamp. Since the stage publishes data on every movement, it can be easily stored and therefore this should be implemented.

Support of multiple server RPi's

Multiple RPi servers for more ADC channels and/or more remotely controlled hardware. This is already partially supported in many widgets etc but needs to be implemented in irrad_control's main functions. This comes with an entire restructuring of the software package

Documentation

Write documentation using Github pages and docs folder (see here)

Basil driver

It would be awesome to have a basil driver, to allow synchronization between chips measurements with the fluence. Some simple interface, like start/pause/stop/get_fluence.

Add bash-script for installation of software

Add a bash-script which installs the software on the host PC side e.g. executes all necessary commands to

1. Install irrad_control into the local Python environment via the setup.py file
2. To do the necessary shh-key generation and copying to the respective server

This is bash-only stuff and should be nice to get into the project. Maybe this script is also useful to look at Please make a feature branch for this. Thx

XY-Stage improvements

• Only scan if beam is above certain current
• Make config file to keep track of mileage of x and y axis (Needed for maintenance)

Add calibration for different particles in setup

Add the beam current calibrations for the Al SEM for the different particles / energies to the config and setup

Add offline analysis and plot generation

Enable offline plotting / analyzing of the output files generated during irradiation