nthallen/monarch

Monarch Data Acquisition System

Monarch

Monarch is a portable data acquisition and control system developed at the Anderson, Keith and Keutsch Groups in The Harvard John A. Paulson School of Engineering and Applied Sciences.

monarch's Issues

OUI and CMake conflict

To get CMake to run OUI, we'll need to add a PATH facility to OUI. CMake runs the tools in the binary directory, which is fine unless the source file tries to reference another file in the source directory.

Resolve msg_hdr_default

Figure out what to do about
const char *msg_hdr_default;
In lgr.oui, tm_client.oui, etc.

oui: need <stdio.h> to support print_usage() in output file

print_usage uses printf, so we need to make sure we include <stdio.h>

When should DAS_IO automatically manipulate flags?

Socket tries to do this in a lot of cases, but not all. Should it be totally automatic?

• It is almost always reasonable to set Fl_Read
• Fl_Write should be set whenever the output buffer is non-empty
• Fl_Except should be set manually by processes that do something with it.
• Fl_Timeout should be set whenever TO.Set()

cmdgen.skel features hack fix

I am currently writing the ncurses.h feature discovery values into cmdgen.skel so it gets into generated code. The problem with that is that when cross-compiling, we are using the native cmdgen compiler, and hence see the features of the native environment rather than the features of the target. Those features should go into cmdalgo.h instead, so they are evaluated for the target system. cmdalgo.h is included from cmdgen.skel, so it will still get loaded.

DAS_IO::Socket connection failure messages should be suppressed

After the first message, if retries are supported, no more complaints should be issued until either the limit is reached or a successful connection occurs.

A successful connection might not ordinarily be announced, but if a connection failure is reported, then the connection must be announced.

Authenticator should update iname to match client info

I believe the iname is persisting as 'clientcon', but it should identify the client.

memo: shutting down server before clients quit

memo -l2 closes its listening sockets after 2 clients connect so subsequent clients cannot.

cis_initialize() and cis_terminate(): Are these ever used?

These are defined in cmdgen.skel, which makes them unusable. Perhaps these would be better handled as virtual methods in a cmd_server class.

cmdgen implementation

Classes defined in cmdalgo.h
Instances generated in cmdgen/interfaces.c
Class implementation tmpplib/src/cis.cc

• cmdif_rd: interfaces served by the command server
• cmdif_wr: interfaces served by another server to which the command server will connect
• cmdif_dgdata: Like cmdif_wr, but specifically for TM data (SWData)

Add addtional not_*() functions

Look in Zeno, any other drivers out there.

Reconfigure directory structure

• Consider collapsing empty directories
• Top level organization:
  • libs
    • dasio
    • nl
    • tm
  • tools
    • cmdgen
    • oui
    • tmc
    • tmcalgo
    • table
  • utils
    • memo
    • lgr
    • rdr
    • tmbfr
  • drivers -- currently an independent cmake project
    • subbus

memo: adapt to DAS_IO::Server updates

• Eliminate memo_server class in favor of DAS_IO::Server
• Make necessary changes to get it to compile
• Use Server::set_passive_exit_threshold() to get memo to terminate.

ci_* now cmd_client should change to cmd_writer?

To disambiguate from a cmd_reader. Both are clients of the command server. cmd_writer opens cmd/server. cmd_reader will specify the subservice, e.g. cmd/Modbus. These could easily be the same Client, so maybe cmd_client would work. cmd_writer doesn't do anything until something in the rest of the program wants to send a command. cmd_reader doesn't do anything until input comes in. Readers probably need to be subclassed to do the right thing with the input. Writer would just assume an "OK" or some error response.

Revisit Server Termination

WAS: Work out Signal handling for Collection, etc.

The main loop should handle SIGINT by default to exit the event loop cleanly.
Will need to handle something like SIGUSR1 via timer_create() in order to manage telemetry. Is the signal necessarily handled by one of the Interfaces? I should look closely at collection. It might be multi-threaded like TMbfr.

Server refactor

This incorporates #22. I want to

• Define Serverside_Client class which knows about Server
  • Add client to Server's list of clients
  • Authenticator should be a Serverside_Client then, right?
  • Does that mean every server must use Server class?
• Change socket_clone_t to return Serverside_Client*
• Provide appropriate hooks in Server::Start to do an orderly shutdown
  • list of all clients (including Authenticators)
  • close all clients (is there a case where this is undesirable?)
    • memo should wait for all clients to close their connections
  • or wait for all clients to close (but what are the preconditions?)
  • I'd like to avoid needing to subclass Server, so let's put any particulars into the Serverside_Client or make the options general enough to apply to all servers.

Rewrite cmd_server using new Server module

Update distribute for remote installation via scp

Need to use scp instead of cp. Need to:

• Figure out the appropriate options
• and everything else

Also need to customize for path to sudo. This is fine for the native builds (which sudo), but I'm not sure how to address it for the cross compilation. Oh wait, distribute does not need to be cross-compiled.

Add _callback suffix to callback functions

This is simply to clarify the API. Callbacks affected include:

• DAS_IO::Interface
  • ProcessData()
  • read_error()
  • protocol_input()
  • protocol_timeout()
  • protocol_except()
  • tm_sync()
  • closed()
• DAS_IO::Socket
  • connected()
  • connect_failed()
• DAS_IO::Client
  • app_input()
  • app_connected()

Note that most or all of these should really be protected, but many have been declared public.

Server: Change client connection message

Currently the interface name is always 'clientcon', but by the time this message appears, we know more about the client, so it makes sense to use the new interface name.

Refactor DAS_IO::Server

Rename DAS_IO::Server to DAS_IO::Server_socket

Create a new DAS_IO::Server class containing

• DAS_IO::SubServices object
• DAS_IO::Loop object
• DAS_IO::Server_socket *Unix, *TCP;

memodev: missing file: oui.oui

oui/src/oui.oui is missing from the memodev branch.

OUI needs to replace QNX usage

Instead of generating #ifdef USAGE, generate oui_print_help(), and then modify oui_init_options() to always recognize -h or --help and/or display help whenever invalid options are entered.

Log Interface statistics on shutdown

This was a feature of Selectee.

bfr/tm_client/unix_name: run directory handling

Currently, tm_client is missing some functionality previously implemented in DC.cc, namely the creation of a file in the run directory that identifies that the program with a particular PID has established a connection with bfr.
[ ] Implement that feature in tm_client
[ ] While we're at it, combine the definitions of the run dir locations into a dasio/rundir.h and eliminate hard-coded paths in unix_name and bfr/rundir.h.

Note, there are two run directories:

• /var/run/linkeng is the base for unix_name sockets
• /var/run/linkeng/run is the base for tm_client PID files

Socket::close() should not delete unix_name

That should happen on destruction.

Server bufsize handling

Currently, when creating a Server, you must specify the input bufsize. This value propagates to the Server_socket, the Authenticator and ultimately to the new server/client Socket. This was supposed to make things a little easier, but:

• The Server_socket needs no input buffer at all
• The Authenticator needs a size sufficient for negotiation and nothing more
• The ultimate application socket's requirements may not actually be known until after negotiation
  • This is true for DG/data, where the size is determined by the datum size.

I suggest we eliminate the bufsize spec for servers, fix the value for Authenticator, and determine the server/client Socket's bufsize in the cloning function.

msg.oui: Add app identifier option.

This is necessary to allow overriding the application name in AppID in order to differentiate log messages.

Migrate compilers from autotools to CMake

cmdgen and oui are now in-tree, but configured and compiled separately.

Need to think through oui implementation

Specifically, how will AppID initialization work?

If the application is responsible for instantiating the object, then the application-level choice will take effect. An AppID oui option would still allow command line override. This kind of eliminates the need to provide the carefully structured hdr override system we used before, where various packages could provide a suitable default. A Cmd_writer program might use 'Clt', and algo, 'TMA'.

I think the AppID approach, which includes more specific information, is probably an improvement, since early messages will have the right name.

Quit: Allow optional

Under certain circumstances, subbusd needs to start before the command server. In such cases, failure to connect to the cmd/Quit service should be OK.

On the other hand, if subbusd is the only case where this is an issue, perhaps subbusd should take a commandline option to avoid connection to cmd/Quit.

Tools that don't need to be cross-compiled

appgen
appgen_module
distribute
genui
genui_chk

msg() needs to be reimplemented

The current version always outputs to stderr.

nctable and nct_getch()

In arp-das, this used a select() loop to read from stdin and the screens of other display programs. I would like to port this to use a standard DAS_IO::Loop. Obviously stdin can be setup as a simple Interface. Then I could setup a server to accept input from display programs. The display programs can read from their own screens and forward input to the keyboard server. This is potentially slower than directly opening the remote terminals to read, but it is more flexible. Display programs could be shutdown and restarted gracefully.

CMake: Setup cross-compile toolchain file for AM335x

Reference is here.

Might be nice to figure out how to translate the SDK environment setup script into a toolchain file.

subbusd_CAN: Report timeout/recovery once

We need a log entry to know if a CAN board is not present, but not more than one.

test_socket appears to be fragile

test_socket needs to be improved to resolve timing issues causing test failures.

msg: New option -a conflicts with nctable.oui

This will conflict on all keyboard client and display programs.

Templates: Create matching client templates

These can double as simple testbenches

cmdgen is writing tabs in its output

Probably need to purge the source code of tabs as well

Google Test Inclusion CMake breaks under Cygwin

The CMake configuration recommended by Google Test does not compile under Cygwin. They assert std=c++11, but that hides several functions used by Google Test. I have manually modified the CMakeLists.txt to enable extensions, but that can only be done after it fails the initial build. It would be nice to have an approach that would work out of the box.

On the Google Test github, there was mention that the same effect can be obtained by defining

#define _POSIX_SOURCE 1
#define _XOPEN_SOURCE 500

not_alt* functions need curation, documentation

not_alt() actually consumes matches, which is just wrong. None of these are documented.

Header reorganization

Move all of the DAS_IO stuff into header files under dasio/

• dasio/interface.h
• dasio/loop.h
• dasio/socket.h
• ...

Also avoid over-nesting class names. Put client/server directly under DAS_IO, even though they are derived from Socket.

Modbus is a special case. Modbus RTU is derived from Serial, but Modbus TCP is derived from Socket. So could go with:

• DAS_IO::Serial::Modbus
• DAS_IO::Socket::Modbus

or I could instead do:

• DAS_IO::Modbus::RTU
• DAS_IO::Modbus::TCP

Implicit declaration warning for strcasecmp on cross-compile of oui

Links OK, so what am I missing?

Replace modbus_req::ascii_escape()

Modbus requests, being binary, should be displayed either as a hex dump or with the PDU structure specifically spelled out. In either case, might need some way to handle multiple lines of output

DAS_IO::Server refactor

SubServices should be a member of the Server class. No point in passing it in. Instantiate the Server first, then add subservices.

Also add add_subservice() and rm_subservice() methods.

This affects implementation of Cmd_server

memo: On client connection, interface name should be updated

This is probably an issue in the socketclone_t function, which should pull the client name out of the Authenticator.

CMake: Move to proper out-of-tree build strategy

• le-das
  • git
    • .git/
    • libs/
    • ...
  • build-master/
  • build-cross-master/
  • dox/
    • index.html -- to all generated HTML
    • libs/dasio
    • libs/nl
    • libs/tm
    • tools/cmdgen
    • tags/
    • ...

Include test to prevent in-tree build

memo: -o creates file, but messages go to stdout

For some reason, the -o foo.log option does not cause output to go into the file.

DAS_IO::Modbus::RTU constructor flags unnecessary

I believe the flags must always be O_RDWR|O_NONBLOCK or the equivalent. Non-block is provided by the library anyway, but it wouldn't be Modbus RTU if it weren't read/write.