silab-bonn/basil

A data acquisition framework in Python and Verilog.

Basil

Basil is a modular data acquisition system and system testing framework in Python.

It also provides generic FPGA firmware modules for different hardware platforms and drivers for wide range of lab appliances.

Features

Firmware:

• very simple single master bus definition
• multiple basic modules (SPI, SEQ, GPIO, I2C, JTAG)
• multiple interfaces (UART, USB2, USB3, Ethernet)

Software:

• layer structure following hardware
• generation based on yaml file
• register abstract layer (RAL)
• simulator interface allows software test against simulated RTL (thanks to cocotb)

Installation

Clone the repository to get a copy of the source code (for developers):

git clone https://github.com/SiLab-Bonn/basil.git
cd basil
pip install -e .

or install via PyPI into the Python's site-packages folder (for non-developers):

pip install basil_daq==version

where version is a version number (e.g., 3.0.1)

or install from git, when no PyPI package is available (for non-developers):

pip install git+https://github.com/SiLab-Bonn/basil.git@branch

where branch is a branch name or tag (e.g., master or v3.0.1).

Check .travis.yml for installation/testing hints.

Support

Please use GitHub's issue tracker for bug reports/feature requests/questions.

For CERN users: Feel free to subscribe to the basil mailing list

Documentation

Documentation can be found under: https://basil.rtfd.org

Example Projects:

• pyBAR - Bonn ATLAS Readout in Python
• MCA - Multi Channel Analyzer
• fe65_p2 - DAQ for FE65P2 prototype

License

If not stated otherwise.

Host Software:

The host software is distributed under the BSD 3-Clause ("BSD New" or "BSD Simplified") License.

FPGA Firmware:

The FPGA code is distributed under the GNU Lesser General Public License, version 3.0 (LGPLv3).

basil's Issues

Virtual DDR FIFO (AXI)

Implement a virtual FIFO based for DDR memory. Proabably best using AXI interface so will work with any memory type/controller.

initilalization of StdRegister with bitorder

the "bitorder" will not be applied when basil initialize values from YAML file.

Downloading firmware to MIO3 and MMC3

The downloading feature is still missing in pySiLibUSB and SiUsb3 TL (https://github.com/SiLab-Bonn/basil/blob/development/host/basil/TL/SiUsb3.py#L46).

Change firmware folder structure

The firmware folder should contain subfolders for different firmwares (FPGA modules, Arduino, etc.).

Use unit prefix only instead of full unit

instead set_voltage(self, channel, value, unit='mV') use set_voltage(self, channel, value, unit='m').

Makes things easier when comes to abstraction. We can also do some unit conversion based on a new class that we can put inside utils. The conversion is then based on a dict like that one:
_prefix = {'y': 1e-24, # yocto
'z': 1e-21, # zepto
'a': 1e-18, # atto
'f': 1e-15, # femto
'p': 1e-12, # pico
'n': 1e-9, # nano
'u': 1e-6, # micro
'm': 1e-3, # mili
'c': 1e-2, # centi
'd': 1e-1, # deci
'k': 1e3, # kilo
'M': 1e6, # mega
'G': 1e9, # giga
'T': 1e12, # tera
'P': 1e15, # peta
'E': 1e18, # exa
'Z': 1e21, # zetta
'Y': 1e24, # yotta
}
also check http://stackoverflow.com/questions/10969759/python-library-to-convert-between-si-unit-prefixes.

TDC: wrong TDC_OUT

https://github.com/SiLab-Bonn/basil/blob/development/firmware/modules/tdc_s3/tdc_s3_core.v#L129

It rathre should be:

assign TDC_OUT = CONF_EN_INVERT_TDC ? &TDC : |TDC;

as now it is problem to close the timing and actually doing something else then description.
Any particular reason?

tdc_s3: inferred latches

Here:

basil/device/modules/tdc_s3/tdc_s3_core.v

Lines 472 to 481 in 215594a

	always@(*) begin
	if (CNT_TRIG==0 && NEW_TRIG==1 && NEW_TDC==1)
	TRIG_CNT = LENGTH_TRIG - LENGTH_TDC;
	else if (CNT_TRIG==0 && NEW_TRIG==1 && NEW_TDC==0)
	TRIG_CNT = LENGTH_TRIG;
	else if (CNT_TRIG==1 && NEW_TDC)
	TRIG_CNT = CLKDV*4 - LENGTH_TDC;
	else if (CNT_TRIG==1)
	TRIG_CNT = CLKDV*4;
	end

There is else missing. This will be latch which is bad.

Setting power gives error

@laborleben
'''
self.dut['ADAPTER_CARD'].set_voltage('VDDD1', 1.2)
File "/basil/basil/HL/FEI4AdapterCard.py", line 161, in set_voltage
self._set_dac_value(value=value, **self._ch_map[channel]['DACV'])
File "/basil/basil/HL/FEI4AdapterCard.py", line 85, in _set_dac_value
self._intf.write(self._base_addr + self.MAX_520_ADD, array('B', pack('BB', channel, value)))
error: ubyte format requires 0 <= number <= 255
'''

SiUart: driver and api cleanup

TL.SiUart needs changes

SiUart can be derived from TL.Serial.

It is also not clear to me, why SiUart uses board_id and avoid_download conf parameter.

remove host folder to make packaging/releasing simpler

do you agree?

Initialization of ISRC for GPAC

current of ISRCs should be set to 0 during the initialization.

Use Serial TL for ISEG driver

TCP timeout

With the latest version of basil (development) I got last testbeam sometimes this kind of error:

Traceback (most recent call last):
 File "meta_tune_threshold_simple.py", line 193, in <module>
   mtt.start(**local_configuration)
 File "meta_tune_threshold_simple.py", line 46, in start
   self.scan(**kwargs)
 File "meta_tune_threshold_simple.py", line 139, in scan
   th1, scurve, mean, tdac1 = correct(1, tdac)
 File "meta_tune_threshold_simple.py", line 89, in correct
   scan.start(**kwargs)
 File "/home/silab/git/bdaq53/bdaq53/scan_base.py", line 473, in start
   self.chip.init()
 File "/home/silab/git/bdaq53/bdaq53/rd53a.py", line 335, in init
   super(RD53A, self).init()
 File "/home/silab/git/basil/basil/dut.py", line 119, in init
   catch_exception_on_init(item)
 File "/home/silab/git/basil/basil/dut.py", line 113, in catch_exception_on_init
   mod.init()
 File "/home/silab/git/basil/basil/TL/SiTcp.py", line 175, in init
   self._sock_tcp.connect((self._init['ip'], self._init['tcp_port']))
 File "/home/silab/miniconda2/lib/python2.7/socket.py", line 228, in meth
   return getattr(self._sock,name)(*args)
socket.timeout: timed out

Before I never observed this and after switching one commit behind this, the error was never again observed during testbeam. Is it maybe related to this commit 99f77a3?

I tested this change before merging once with pymosa and no issue was observed.

TDC module with inverted trigger and tigger delay does not work

Only a positive edge trigger for the trigger delay determination is working (= delay between edges of TDC and trigger) . If a negative edge trigger is used it fails. The setting EN_INVERT_TRIGGER signal is not recognized by the TDC module. The result looks the same if it's 0 / 1.

Not working

TLU cfg:
EN_INVERT_TRIGGER = 1
TDC cfg:
ENABLE = True
EN_TRIGGER_DIST = True

Working

TLU cfg:
EN_INVERT_TRIGGER = 0
TDC cfg:
ENABLE = True
EN_TRIGGER_DIST = True

Auto wait (is_ready)

Would be nice to make it possible that start() command that in nature are non-blocking became blocking (with timeout) through configuration.

gpio driver auto configuration

Should read back width and direction on init from module. Rise exception when needed.

Clock domain crossing issues in m26_rx

Reading the timestamp uses a clock from an external source where the timestamp itself is synchronous to an internal clock.

See:

basil/firmware/modules/m26_rx/m26_rx_core.v

Line 149 in 07b7cf3

reg [31:0] TIMESTAMP_save;

Wrong configuration does not lead to a code crash

One can have a typo in the configuration file and basil will not complain.

For example:

TLU:
    TRIGGER_NODE        : 0

just works.

That bug was most likely introduced in this comit a28d617.

Tdc: do not use TRIG_IN by default

Use parameter and generate to exclude IDDR on TRIG_IN. Will limit functionality but this can lead to implementation errors if this input is not connected(not needed).

Read calibration data for GPAC

Read calibration data from EEPROM on GPAC init.

Read module VERSION

Here:
https://github.com/SiLab-Bonn/basil/blob/master/host/basil/HL/RegisterHardwareLayer.py#L53

Read VERSION only once and only if it exist. Also check for debug!

SiTCP.py bug

Please fix the line 130 in basil/TL/SiTCP.py to:
ret += self._read_single(new_addr, size + self.RBCP_MAX_SIZE - next_size)
Otherwise, the last packet is not read correctly.

Driver for si5324

C++ version: https://github.com/m-labs/artiq-chip-reg/blob/master/si5324drv.c

Test for TDC

Documentation for TLU module

Add get_status() function

A function that returns status of all modules/registers etc. useful for logging.
Probably with level information argument like get_status(level)

Read during write to hardware module register

If size and offset modulo 8 == 0 no need to read?
https://github.com/SiLab-Bonn/basil/blob/master/host/basil/HL/RegisterHardwareLayer.py#L95

Clean up TL base classes

Every layer needs a very simple base class.

Add negative indexing support to BitLogic

Something like this should be supported:

bv[-1] = 0
bv[-2:-1] = 1

Logging for bronkhorst_elflow instead of pring

@thirono Could you change print to logger here:
https://github.com/SiLab-Bonn/basil/blob/development/basil/HL/bronkhorst_elflow.py

Example: https://github.com/SiLab-Bonn/basil/blob/development/basil/HL/NTCRegister.py

+ format the source

TLU working with non fei4 modules

CMD_READY should not need to be toggled:

Suggestions:

• configuration bit

or better

• change this to trigger acknowledge signal that cmd_seq will issue (pulse at the end) for others can be tide to 1

Remove User Layer

Move existing UL drivers to HL and allow multiple mapping within HL.
Probably need to use something like this: http://stackoverflow.com/questions/5121931/in-python-how-can-you-load-yaml-mappings-as-ordereddicts so that initialization order is controlled.

SiTCP: underlying object is terminated before readout thread is terminated.

The thread is sometimes terminated after the underlying SiTCP object is terminated resulting in incomplete errors messages:

Exception in thread TcpReadoutThread (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/home/jens/miniconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner

or

Exception in thread TcpReadoutThread (most likely raised during interpreter shutdown):
Traceback (most recent call last):

or

Exception in thread TcpReadoutThread (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/home/jens/miniconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  File "/home/jens/miniconda2/lib/python2.7/threading.py", line 754, in run

The reason is an AttributeError because the attribute doesn't exist anymore.
The implementation of connect() and disconnect() can solve that issue.

Support for other oscilloscopes (e.g. Tektronix MSO4034)

Retrieving waveforms from various oscilloscope may vary from device to device.
Therefore a better support for retrieving the raw data is necessary.
For example, the driver for the MSO4104b works also for the MSO4034 but retrieving raw data is different.
For the MSO4034 the following settings are necessary:
resource.query_binary_values("DATA:SOURCE CH1;:WAVFrm?", datatype='b', is_big_endian=True)

Basil on PyPI

Since basil is not shipping scripts you need to change on a frequent basis I would recommend that we put the tags on PyPI? Its quite simple and it would ease the installation process. Unfortunately the name is already taken.

usb3 and SiLibUsb

Finally should have usb3 in main branch

There are few possibilities.
One suggestion can be to meke new TL called SiUsb3 (at the end this can look better then some ugly if). If not one need to add error message in SiUsb if wrong version of SiLibUsb is used.

Nevertheless one need to change SiLibUsb api or count count argument for FastBlockWrite and FastBlockRead or just always use WriteExternal and ReadExternal only.

What you think?

StdRegister offset

As pointed by @hansk68 offset should point to LSB and not to MSB position.
Changing this will break a lot of compatibilities.

Options:

• break compatibility
• remove offset and change this something else like lsb_pos used here: https://www.vmmcentral.org/uvm_vmm_ik/files3/reg/uvm_reg_field-svh.html
• add new filed and deprecate offset

Comments welcome.

basil.TL.TransferLayer.Serial.query() doesn't handle receive buffer with old data properly

Sometimes the receive buffer has data from commands that were previously sent. When issuing a query, query() sends the query command and reads the data from the previous command, because _readline() just checks if read termination occurs at the end of the string (while reading character by character). _readline() does not check if more data is remaining in the receive buffer and if the termination strings exists more than once in the received data.

Test of SCPI fails

ERROR: test_write (test_SimScpi.TestSimScpi)

Traceback (most recent call last):
File "/home/travis/build/SiLab-Bonn/basil/tests/test_SimScpi.py", line 43, in test_write
self.device['Pulser'].set_on()
File "/home/travis/build/SiLab-Bonn/basil/basil/HL/scpi.py", line 65, in method
raise ValueError('Invalid SCPI command %s for device %s' % (name, self.name))
ValueError: Invalid SCPI command set_on for device Pulser

The error in the test never occurred because set_on command was never executed because "if .. elif .." statement had no final "else". I added "else" recently, so the error is visible now.

What was the intention of "set_on" / "get_on"? These commands doesn't exist in our device drivers. There is only "on" / "off". We might just implement "set_on" / "get_on" / "set_off" / "get_off" if that makes sense.

SEQ_GEN changes output when changing the memory content when in idle

Is this behavior expected? In my opinion the output should only change when START is triggered, and not when changing the data or LENGTH register.

Annoying warning from basil.TL.Serial

Since a few weeks ago, i always get a warning from basil.TL.Serial for Keithley 2410 and Keithley 2400 (only tested those two):

[basil.TL.Serial] - WARNING Found 1 bytes in the input buffer of interface Serial2 which will be flushed

Readout still works fine, so the warning is very misleading and annoying.

Q: Tune tlu

Could this https://github.com/SiLab-Bonn/pymosa/blob/development/pymosa/tune_tlu.py be part of basil? Mauby basil/utils?

@laborleben @YannickDieter @DavidLP

Autoupload to PyPI

Shall i add autoupload to PyPI for successfull release builds?

CONF_DONE not working in seq_gen_core.v

Reading the state of the SEQ_GEN does not return the expected result. Simulation shows that the CONF_DONE is going high even when the SEQ_GEN is still sending out data.

bug in log print

https://github.com/SiLab-Bonn/basil/blob/development/basil/TL/SiTcp.py#L108

%d --> %s

TLU trigger counter

The trigger counter from the TLU module cannot be set with self.dut['TLU']['TRIGGER_COUNTER'] = 0 or the read value via self.dut['TLU']['TRIGGER_COUNTER'] is wrong.

Clean up HL

Just some ideas for the HL folder for discussion:

1. I would like to clean up the HL base class. It should not have a base adress. This is something only needed for the RegisterHardwareLayer (right?).
2. The interface (self._intf) is an instance of a transfer layer module. So why not to name it self._TL_intf to make this obvious? Or at least we can add a comment that the interface is a TL instance.
3. Some modules do not use any TL from basil. These modules should be either changed or taken out of basil. They seem to be "stand alone" modules (agilent33250a, iseg_shq) and do require additional modules that are neither shipped with basil nor std. python packages. That makes them pretty "displaced".
4. The naming is ugly. Upper/lower/CapWord convention all mixed up. I would prefer to follow https://www.python.org/dev/peps/pep-0008/#package-and-module-names
5. In total we have description parts (e.g. RegisterHardwareLayer, TransferLayer) and implementations of these (e.g. pulse_gen) in the same folder. I would like to have these seperated, but the question is if we gain something from this. HL looks already crowded now, and the very basic classes are just between their implementations.
6. The scpi device descriptions are different from the other files (yaml file not python script). Maybe better to put it somewhere else? A subfolder? Not shippinh with basil?

Configuration yaml not working

When a device is initialized like this:

 transfer_layer:
  - name  : device name
    type  : device type
    additional_parameter : Bar

Then the additional parameter(s) are not taken into account. This bug (or feature, why?) was introduced in this commit: 6f9091a

Module level logging

Effective logging needs tunable filtering. Using the root logger prevents tunable filtering. Thus I propose to use module level logging.