After installing Go, the program can be run in either of these 2 ways:
$ go run main.go input.txt
$ cat input.txt | go run main.go
The central recurring theme (and guiding principle) is a focus on a production-ready architecture for future extensibility -- putting structures into place that make it easy for (and encourages) future developers to Do The Right Thing (tm) without needing to make any sweeping changes as long as the overall shape of the problem doesn't fundamentally change.
That can be seen in the strict interface boundaries laid out between packages (and the isolation of package-internal data domains), the type definitions that allow for minimally-disruptive changes to the modeling of central concepts, and the data structures and algorithms chosen that shouldn't require any modifications until we hit large scale.
There are detailed "Maintainer Notes" in many files that talk about these things in more specific detail.
Aside from modeling the system on components that follow very naturally from breaking down the problem statement into core responsibilities, the architecture has leaned towards being amenable to running in a streaming+serverless environment as far as possible -- the goal is that it should be relatively simple to slap this software onto an AWS environment composed of (say) Kinesis (for streaming infrastructure), Lambda (for serverless compute), and DynamoDB (for storage).
The flow of the system (and thus the interaction of the packages) is described below in terms of the elevator pitches of their main components.
Reads from an event source (in the case of our specific scenario, an input file) and generates a stream of input.EventEnvelope
objects.
Receives a stream of input.EventEnvelope
objects and a handle to an eventstore.EventStore
, and emits a stream of error
s that may
result from processing those events.
Parses the input.EventEnvelope
objects just enough to be able to deduce the eventhandler.EventType
, based off of which it delegates to the
concrete eventhandler.Interface
implementation registered with eventhandler.GlobalRegistry()
.
Provides independent sub-modules that implement eventhandler.Interface
for handling all the events supported by the system, and that store
event information of value to downstream systems in the eventstore.EventStore
that is passed in to them.
The eventhandler/
README talks more about the reasoning for the directory structure in there, as well as the steps to add support for new events.
Serves as storage for retain-worthy event information.
Provides eventstore.VisitorInterface
for inspection of all that retained information.
Provides output.ReportGenerator
that implements eventstore.VisitorInterface
and generates a report in the desired output format.
A collection of shared utilities for mathematical operations that are hard to get right.
In keeping with the streaming-centric architecture of the system, there's 3 concurrent threads of execution that are naturally modeled as stages of a streaming pipeline:
- Reading the input from the event source to generate events (
input.StartReading()
). - Processing the events (
eventprocessor.EventProcessor.Process()
). - Handling the errors from processing the events (this happens in main.go on the main thread).
There's near-100% unit test coverage for every package in the system, and the tests utilize multiple techniques (as appropriate for the component under test):
-
Table-Driven Tests -- this is the recommended Golang style and is used for all but one component's tests (because the structure of each test case was different enough that trying to generalize it would make the hypothetical single unified test case too unreadable).
-
Fakes + Behavioral Testing -- see eventprocessor_test.go.
-
Declarative Testing -- see eventstore_test.go.