ryanskraba/beam-enchiridion

A small manual or handbook for Apache Beam

The Beam Enchiridion

Enchiridion: A small manual or handbook. It's a bit like a tech cook book, but a bigger, fancier, SEO-optimizabler word.

This project describes how to do many common tasks using Beam.

Topics

I want to...	See...
write a simple ParDo	ThrowDart, MeasureDistanceDoFn (test)
write a composite PTransform	MonteCarloPiTransform (test)
write a source
write a sdf
side input
state
metrics
streaming
my own windowing function

Modules

module	description
core	Examples for the beam core Java SDK.
core-scala	Examples for the beam core Java SDK in Scala (but not scio).
io-files	Examples for reading from and writing to files.

Running with a locally-built SNAPSHOT

# Build beam artifacts and publish to /tmp/snapshots/
cd beam
./gradlew -Ppublishing -PdistMgmtSnapshotsUrl=/tmp/snapshots/ publishToMavenLocal

# Build this project against the snapshots.
cd beam-enchiridion
mvn -Dbeam.version=2.x.0-SNAPSHOT -Plocal-snapshot clean install