cafana/srproxy

SRProxy

(Please also see the general CAFAna README for more information.)

SRProxy is a toolkit for fast reads of StandardRecord objects from ROOT files. It can read two kinds of files:

• "Structured" (or traditional) CAFs, in which there is one StandardRecord object per entry
• "Flat" CAFs, in which a StandardRecord object is 'flattened' during serialization into basic ROOT types, and the structure is maintained in the branch names only.

Such CAFs are written by "CAF-maker" software maintained by the experiments that use CAFs as their analysis files.

When used, SRProxy provides automatic compilation-time deduction of which branches within the StandardRecord object need to be enabled when reading from the file. Any unused branches are disabled. For complicated StandardRecord objects, this can result in speedups of several orders of magnitude.

Usage

SRProxy needs to be templated over a concrete StandardRecord type that contains the relevant fields for the user's needs. In-practice examples include the implementations by SBN and DUNE.

It would be nice to have a technical digest of how to do this here, but in the meantime, please contact the CAFAna librarian and we can discuss your use case.

srproxy's Issues

python dependency (`v3_9_2` to `v3_9_13` since larsoft `v09_74_00`)

larsoft v09_74_00 now depends on art v3_12_00 rather than v3_09_04, and it now depends on later version of root:

larsoft v09_73_00
- art v3_09_04
- root v6_22_08d
-- python v3_9_2

larsoft v09_74_00
- art v3_12_00
- root v6_26_06a
-- python v3_9_13

srproxy v00.40 depends on python v3_9_2, so we need a new release with python v3_9_13

add support for python 3.9.13

With the move to art 3.12 DUNE need a new release of duneanaobj which includes some changes from Kyle K. and a new ROOT. That ROOT depends on python 3.9.13 rather than 3.9.2.
SRProxy currently doesn't support that dependence so need to add support for python 3.9.13

add support for python 3.9.15

via Miquel in SBN: with the upcoming upgrade of art in LArSoft, comes the upgrade to root v6_28_10a and python v3_9_15. SRProxy will need an update too. We are trying to get ahead as SBN is in a tight deadline for production but are willing to get art3.14.3.

basically, we need to grab a new version of pygccxml, specifically pygccxml v2_2_1b built against python v3_9_15.

I'll add py3.9.15 support and update pygccxml -- then proceed to build, and release new version of srproxy

can't work with `StandardRecord` objects with scoped enums

SRProxy can't work with classes that contain scoped enums (i.e. elements from enum class declarations). Fundamentally this is because it internally depends on implicitly converting them into ints, which is not possible with scoped enums (which may not even be based on int type!).

I spent a while trying to figure out how to support them. I eventually wound up at a dead-end because of the Restorer class that works with altered standard records (used in systematic shifts). See https://github.com/cafana/SRProxy/blob/master/BasicTypesProxy.h#L340-L383.

The idea here is that a single Restorer object needs to be able to hold separate collections of all the different RestorerT types.  This is a neat way of having one collection per type.  Unfortunately, with scoped enums in play, the list of possible types is now variable (since each scoped enum is its own type and can't be converted implicitly to int).

I thought for awhile about this problem (needing a "dynamic" collection of things that are known at compile time but not declaration time), and I couldn't think of any very neat solutions.  The best I could come up with was using an unordered_map where the key is the typeinfo (or some hash of it) of the type RestorerT is templated over, and the value is a pointer-to-vector-of base-class-of-RestorerT (so I'd have to introduce a base class of RestorerT).  I worry though that because Restorer::Add() is a hot code path, all these map lookups are likely to be far slower than a call to a function where its location is known at compile time.

I'm leaving my explorations in branch feature/scoped_enums, in case anybody ever wants to come back to them, but for now, I think I'll be just converting any enum classes I see back to unscoped enums 😢

Verify that SRProxy works correctly with C arrays

Pull request #6 fixes an issue with indexing of vector branches.
As a check left behind, we may need to confirm that the issue was not present in C array branches, or that it is solved with that fix.

Adding declarations to std namespace is UB

SRProxy/BasicTypesProxy.h

Line 432 in c99450b

namespace std

Experiment-specific code should be removed

SRProxy/BasicTypesProxy.cxx

Line 437 in c99450b

if(fName == "rec.me.trkkalman" ) return "rec.me.nkalman";

Compilation with Clang 7.0 fails

I have received a report that the compilation of this repository with Clang (7.0) fails.
I need to confirm and get more information.