thi-ng/umbrella

⛱ Broadly scoped ecosystem & mono-repository of 192 TypeScript projects (and 157 examples) for general purpose, functional, data driven development

• About
• Getting Started
  • Project templates
  • #HowToThing
  • Blog posts
  • Videos
• Examples & Showcase
  • awesome.thi.ng
  • Generative art projects
• Community, contributing, getting help
  • Wiki
• Projects
  • Latest updates
• Building
• Acknowledgements
• License
• Contributors

About

"A collection of functional programming libraries that can be composed together. Unlike a framework, thi.ng is a suite of instruments and you (the user) must be the composer of. Geared towards versatility, not any specific type of music." — @loganpowell via Twitter

Please visit thi.ng for additional information & topic based search of packages relevant to your use cases...

This project is NOT a framework and all packages can be used in isolation.

This mono-repository is home to 192 individual TypeScript libraries/packages and 156 example projects illustrating their usage, currently totalling ~208k SLOC and ~4000 source files...

Unlike most other large mono-repos this one is not for a single project, but a broad collection of jointly developed, yet independent libraries, tools and general purpose building blocks for the following non-exhaustive list of topics (see package overview below):

• Functional programming (ES6 iterators/generators, composition, memoization, transducers, multi-methods)
• Reactive programming, stream / transducer based dataflow graphs / pipelines / DOM
• Fiber process tree abstraction for ES6 generators (co-routines / cooperative multitasking)
• Data structures & data transformations for wide range of use cases (maps, sets, heaps, queues, graphs etc.)
• WebAssembly bridge APIs & data structure bindings code generators for multiple target languages
• PEG-style functional parser combinators w/ (optional) custom grammar definition language
• Customizable HTML & Markdown parsers
• 2D geometry generation, shape primitives, math, manipulation, intersections, conversions & visualizations
• Canvas abstractions, pixel buffers & SVG serialization/conversion
• Comprehensive function collection (900+) for nD-vectors and matrices (dense & sparse)
• ECS implementations with optional support for strided memory layouts
• Semi-declarative WebGL 1/2 abstraction layer
• DSL for shader functions defined in TypeScript and cross-compilation to GLSL, JS, VEX etc.
• Value-based equivalence (vs. native object identity) and associative data structures (sets, maps)
• DSP building blocks: oscillators, noise generators, filters, 1D FFT/IFFT, muxers, rate converters
• Immutable data handling, state containers, transacted state updates, Undo-Redo history
• Reactive UI component toolkits (DOM-based, canvas-based, immediate-mode, multiple approaches...)
• Multi-format, multi-channel pixel buffers (int & float based), conversions, dithering, Porter-Duff alpha-blending operators
• Color space/format conversions, matrix based color manipulation, gradient generators, color palettes, dominant color extraction
• Date-time abstraction, relative dates, iterators, formatters, math
• WebWorker workflow abstractions
• Forth-style pointfree DSL for functional composition and DSL development/extension
• S-expression parser & runtime (interpreter) infrastructure for custom DSL creation
• WASM-based SIMD batch-processing of vector data
• Pen-plotter (AxiDraw) toolchain & geometry conversions
• Interpolations, math helpers, automatic differentiation (Dual numbers)
• etc.

Once more, this project is NOT a framework. There's no turn-key, one-size-fits-all approach and instead the overall design philosophy encourages a mix & match philosophy for key aspects of application design (inside & outside the browser). Customization points are provided wherever useful and usually only expect certain interfaces/type signatures rather than hard-coded concrete implementations.

All packages:

• are versioned independently
• have auto-generated online documentation at docs.thi.ng
• built via esbuild and tested via bun.sh
• released via thi.ng/monopub publishing toolchain
• distributed as ESM modules (ES2022 syntax) with export maps, TypeScript typings & change logs
• highly modular with often only a single function / class (incl. closely related functions) per file to help w/ selective imports and tree shaking
• provide re-exports of all their publics for full library imports
• have either none or only @thi.ng internal runtime dependencies (w/ very few exceptions! All dependencies are listed in each package readme)
• declare public interfaces, enums & types in an api.ts file
• licensed under Apache Software License 2.0

Getting started

A common misconception is to think of this repo as single project. It is not! The sheer number and varied nature & purpose of these packages makes it impossible to provide traditional "Getting started" tutorials. There would have to be dozens of them... To compensate, this repo provides 150+ example projects, detailed package readmes (at the very least for all the larger and/or more important ones), as well as hundreds of small usage examples/snippets in doc strings.

Documentation and changelogs for all packages: docs.thi.ng

If you're unsure about something, please reach out! Any constructive feedback is highly appreciated!

Project templates

You might find one of the following template repos an useful starting point:

• tpl-umbrella-basic: Bare-bones template repo for browser-based projects
• tpl-umbrella-fxhash: Project template repo for generative art projects on the fx(hash) platform
• tpl-umbrella-zig: Minimal browser project template for hybrid TypeScript & Zig (WebAssembly) apps

#HowToThing

Ongoing since August 2023: #HowToThing is a series of short posts on Mastodon, demonstrating a range of different techniques, patterns and use cases from across the thi.ng/umbrella ecosystem. These are not necessarily intro examples, but each one comes with heavily commented code (and often with visual outputs/results).

• 001: FPS counter with moving average
• 002: Sorting an array by potentially CPU-costly sort criteria
• 003: Multiple key states and extracting commands via FSM
• 004: Creating text-based plots to debug & visualize sequential data
• 005: Barnsley fern IFS fractal
• 006: Clustering arbitrary n-dimensional data using K-means
• 007: Converting Google Maps bookmarks to KML
• 008: CSV parsing into structured data & multi-plot SVG dataviz
• 009: 2D canvas drawing & threadLast() dataflow operator
• 010: Basic web app UI/DOM via Zig/WASM and thi.ng/wasm-api
• 011: 2D cellular automata as WebGL2 multi-pass shader pipeline
• 012: Pure CSS image transition/reveal with thi.ng/hiccup-css
• 013: Lisp interpreter using thi.ng/sexpr & thi.ng/defmulti
• 014: A simple browser REPL for a Lispy mini language
• 015: Generative audio synth offline renderer and WAV file export
• 016: Building a reactive Mastodon UI, parsing & transforming API data
• 017: (Re)Constructing the thi.ng logo as 2D signed-distance field
• 018: Topological sorting, creating & visualizing a task dependency graph
• 019: Building a responsive & reactive stacked column layout
• 020: Generating randomized 4-point 2D multicolor gradient maps/images
• 021: Iterative animated convex polygon subdivision & heat map viz
• 022: Quasi-random voronoi lattice generator
• 023: Tag-based Jaccard similarity ranking/filtering using bitfields
• 024: 2.5D hidden line visualization of digital elevation files (DEM)
• 025: Fitting, transforming & plotting 10k data points per frame using SIMD
• 026: Shader meta-programming to generate animated function plots
• 027: Flocking sim with neighborhood queries to visualize proximity
• 028: Randomized, space-filling, nested 2D grid layout generator
• 029: Forth-like DSL & livecoding playground for 2D geometry generation
• 030: Procedural stochastic text generation via custom DSL & parse grammar

Blog posts

• "Of umbrellas, transducers, reactive streams & mushrooms" (ongoing series):
  • Part 1 - Project & series overview
  • Part 2 - HOFs, Transducers, Reducers
  • Part 3 - Convolution, 1D/2D Cellular automata
  • Part 4 - Disjoint Sets, Graph analysis, Signed Distance Fields
• How to UI in 2018
• The Jacob's Ladder of coding

Videos

(FIXME: Recently I closed my Google account and still have to migrate the below videos, currently defunkt)

• Building a web editor for creating/testing parse grammars
• Building a shader graph editor (WebGL, shader AST transpiler, UI)
• Crash course: TypeScript mapped types in action
• thi.ng/umbrella livestream #3

Examples & Showcase

There's a steadily growing number (~155) of standalone examples of different complexities (often combining functionality from several packages) in the examples directory.

Example screenshots	(small selection)

awesome.thi.ng

Due to other priorities still very much in its infancy & planning stage, but please help to document your own usage of these packages by contributing project information to the awesome.thi.ng repo, which will be used to build a showcase site... Thank you!

Generative art projects

Several generative art projects by Karsten Schmidt on fx(hash) were created exclusively with libraries from this collection.

De/Frag series
Quasiflock	C-SCAPE	ASCII-SCAPE
Bubblemania	Danza (unreleased)	S-TRACE (unreleased)

Community, contributing, getting help

Join the discussions here on Github, get in touch via Mastodon or use the issue tracker. If you'd like to contribute in other ways, please first read this document.

In general, we welcome contributions of all kinds (docs, examples, bug fixes, feature requests, financial contributions etc.). You can find a detailed overview for contributors here: CONTRIBUTING.md.

Note: The default branch for this repo is develop and all PRs should be based on this branch. This too means, the README files on this branch might refer to yet-unreleased features or packages. Please use the main branch for viewing the most recently released version(s)!.

Wiki

To date, the wiki has only been updated sporadically, but please be sure to check it out for project-wide glossary, information, cookbooks, useful snippets etc.

Projects

Latest updates

As of: 2024-04-23

Status	Package	Version	Changelog
	@thi.ng/file-io		changelog
	@thi.ng/wasm-api		changelog
	@thi.ng/wasm-api-bindgen		changelog
	@thi.ng/wasm-api-dom		changelog

Fundamentals

Project	Version	Changelog	Description
@thi.ng/args		changelog	Declarative & functional CLI arg parsing & coercions
@thi.ng/api		changelog	Common types, decorators, mixins
@thi.ng/bench		changelog	Basic benchmarking helpers
@thi.ng/checks		changelog	Type & value checks
@thi.ng/compare		changelog	Comparators
@thi.ng/compose		changelog	Functional composition helpers
@thi.ng/date		changelog	Date/time iterators, formatters, rounding
@thi.ng/defmulti		changelog	Dynamic multiple dispatch
@thi.ng/distance		changelog	n-D distance metrics & K-nearest neighborhoods
@thi.ng/equiv		changelog	Deep value equivalence checking
@thi.ng/errors		changelog	Custom error types
@thi.ng/expose		changelog	Conditional global variable exposition
@thi.ng/fibers		changelog	Process hierarchies & ops for cooperative multitasking
@thi.ng/hex		changelog	Hex value formatters for U4-64 words
@thi.ng/logger		changelog	Basis infrastructure for arbitrary logging
@thi.ng/memoize		changelog	Function memoization w/ customizable caching
@thi.ng/oquery		changelog	Pattern based query engine for JS objects
@thi.ng/parse		changelog	Parser combinators & AST generator/transformer
@thi.ng/paths		changelog	Immutable nested object accessors
@thi.ng/strings		changelog	Higher-order string formatting utils
@thi.ng/system		changelog	Minimal life cycle container for stateful app components
@thi.ng/testament		changelog	Minimal test runner
@thi.ng/transclude		changelog	Template engine for text document generation
@thi.ng/units		changelog	Extensible SI unit conversions

Maths

Project	Version	Changelog	Description
@thi.ng/dual-algebra		changelog	Dual number algebra / automatic differentiation
@thi.ng/dsp		changelog	DSP utils, composable signal gens/processors
@thi.ng/fuzzy		changelog	Fuzzy logic primitives & rule inference engine
@thi.ng/intervals		changelog	Open/closed intervals, queries, set ops
@thi.ng/math		changelog	Assorted common math functions & utilities
@thi.ng/matrices		changelog	Matrix operations
@thi.ng/sparse		changelog	Sparse matrix & vector impls
@thi.ng/timestep		changelog	Fixed timestep simulation updates with state interpolation
@thi.ng/vectors		changelog	Fixed & arbitrary-length vector ops

Randomness

Project	Version	Changelog	Description
@thi.ng/colored-noise		changelog	1D colored noise generators
@thi.ng/ksuid		changelog	K-sortable unique identifiers, binary & base-N encoded
@thi.ng/lowdisc		changelog	n-D Low discrepancy sequence generators
@thi.ng/random		changelog	Seedable PRNG implementations, distributions & utilities
@thi.ng/random-fxhash		changelog	PRNG impl & utilities for fxhash projects

File / file format / hardware support

Project	Version	Changelog	Description
@thi.ng/axidraw		changelog	Minimal, declarative AxiDraw plotter controller
@thi.ng/bencode		changelog	Bencode binary format encoding
@thi.ng/csv		changelog	Customizable CSV parser/object mapper
@thi.ng/dot		changelog	Graphviz DOM & export
@thi.ng/dsp-io-wav		changelog	WAV file format exporter
@thi.ng/file-io		changelog	Assorted file I/O utils for NodeJS
@thi.ng/geom-io-obj		changelog	Wavefront OBJ model parser
@thi.ng/hiccup-css		changelog	CSS from nested JS data structures
@thi.ng/hiccup-html		changelog	Type-checked HTML5 element wrappers for hiccup
@thi.ng/hiccup-html-parse		changelog	HTML parsing & transformation to hiccup format
@thi.ng/hiccup-markdown		changelog	Hiccup-to-Markdown serialization
@thi.ng/hiccup-svg		changelog	hiccup based SVG vocab
@thi.ng/iges		changelog	IGES format geometry serialization
@thi.ng/markdown-table		changelog	Markdown table generator / formatter
@thi.ng/mime		changelog	File extension to MIME type mappings
@thi.ng/msgpack		changelog	Msgpack serialization/deserialization
@thi.ng/pixel-io-geotiff		changelog	GeoTIFF reader support for thi.ng/pixel
@thi.ng/pixel-io-netpbm		changelog	1/8/16/24bit NetPBM image format reader/writer
@thi.ng/pixel-io-pfm		changelog	Portable FloatMap image format reader/writer
@thi.ng/prefixes		changelog	Linked Data, RDF & xmlns prefixes/URLs
@thi.ng/sax		changelog	SAX-like XML parser / transducer
@thi.ng/tangle		changelog	Literate programming utilities

Iterator, stream & sequence processing

Project	Version	Changelog	Description
@thi.ng/csp		changelog	Channel based async ops
@thi.ng/fsm		changelog	FSM / parser primitives
@thi.ng/grid-iterators		changelog	2D grid iterator strategies
@thi.ng/iterators		changelog	ES6 generators / iterators
@thi.ng/seq		changelog	Lisp/Clojure-style sequence abstraction
@thi.ng/transducers		changelog	Composable data transformations
@thi.ng/transducers-async		changelog	Async transducers, reducers & iterators
@thi.ng/transducers-binary		changelog	Binary data related transducers
@thi.ng/transducers-fsm		changelog	Finite state transducer
@thi.ng/transducers-hdom		changelog	Transducer based hdom UI updates
@thi.ng/transducers-patch		changelog	Patch-based, array & object editing
@thi.ng/transducers-stats		changelog	Technical / statistical analysis

Reactive programming

Project	Version	Changelog	Description
@thi.ng/rstream		changelog	Push-based, reactive event stream primitves
@thi.ng/rstream-csp		changelog	Adapter bridge CSP -> rstream
@thi.ng/rstream-dot		changelog	Graphviz visualization of rstream topologies
@thi.ng/rstream-gestures		changelog	Mouse & touch event stream abstraction
@thi.ng/rstream-graph		changelog	Declarative dataflow graph construction
@thi.ng/rstream-log		changelog	Hierarchical structured data logging
@thi.ng/rstream-log-file		changelog	Log-file output handler
@thi.ng/rstream-query		changelog	Triple store & query engine

Algorithms & data structures

Project	Version	Changelog	Description
@thi.ng/adjacency		changelog	Adjacency matrices & graph algorithms
@thi.ng/arrays		changelog	Array utilities
@thi.ng/associative		changelog	Alt Set & Map implementations
@thi.ng/atom		changelog	Immutable value wrappers, views, history
@thi.ng/bitfield		changelog	1D/2D bit field implementations
@thi.ng/buffers		changelog	Generic read/write buffers w/ diff. behaviors
@thi.ng/cache		changelog	In-memory caches / strategies
@thi.ng/cellular		changelog	Highly configurable 1D Cellular automata
@thi.ng/dcons		changelog	Doubly-linked list
@thi.ng/diff		changelog	Array & object diffing
@thi.ng/dgraph		changelog	Dependency graph
@thi.ng/ecs		changelog	Entity-Component System
@thi.ng/egf		changelog	Extensible Graph Format
@thi.ng/gp		changelog	Genetic programming helpers / AST gen
@thi.ng/heaps		changelog	Binary & d-ary heap impls
@thi.ng/idgen		changelog	Versioned ID generation / free-list
@thi.ng/k-means		changelog	K-means clustering of n-D data
@thi.ng/ramp		changelog	Parametric, interpolated lookup tables
@thi.ng/quad-edge		changelog	Quad-edge, dual-graph data structure
@thi.ng/resolve-map		changelog	DAG computations & value resolution
@thi.ng/vclock		changelog	Vector clock functions / comparators
@thi.ng/zipper		changelog	Immutable tree editing / navigation

Frontend / UI

Project	Version	Changelog	Description
@thi.ng/blurhash		changelog	Fast image blurhash encoder/decoder
@thi.ng/canvas		changelog	Canvas creation & HDPI support helpers
@thi.ng/dl-asset		changelog	Asset/canvas/file download helpers
@thi.ng/emoji		changelog	Bi-directional emoji lookup tables (names/chars)
@thi.ng/hdiff		changelog	String diffing w/ hiccup output (includes CLI)
@thi.ng/hdom		changelog	Hiccup based VDOM & diffing
@thi.ng/hdom-canvas		changelog	hdom adapter for hiccup-canvas
@thi.ng/hdom-components		changelog	hdom based UI components
@thi.ng/hdom-mock		changelog	hdom mock implementation (testing / prototyping)
@thi.ng/hiccup		changelog	S-expression based HTML/XML serialization
@thi.ng/hiccup-canvas		changelog	hiccup interpreter for canvas api
@thi.ng/hiccup-carbon-icons		changelog	IBM Carbon icons in hiccup format
@thi.ng/imgui		changelog	Immediate mode GUI
@thi.ng/interceptors		changelog	Composable event handlers & processor
@thi.ng/meta-css		changelog	CSS framework codegen, transpiler, bundler
@thi.ng/rdom		changelog	Reactive, diff-less, async UI components
@thi.ng/rdom-canvas		changelog	rdom component wrapper for thi.ng/hiccup-canvas
@thi.ng/rdom-components		changelog	Unstyled, customizable component collection
@thi.ng/rdom-forms		changelog	Datadriven HTML form generation
@thi.ng/router		changelog	Customizable browser & non-browser router
@thi.ng/text-canvas		changelog	Text-mode canvas, drawing, tables, charts
@thi.ng/text-format		changelog	Color text formatting w/ ANSI & HTML presets

Geometry, image & visualization

Project	Version	Changelog	Description
@thi.ng/boids		changelog	Configurable n-dimensional boids simulation
@thi.ng/color		changelog	Color conversions, gradients
@thi.ng/color-palettes		changelog	Collection of image-based color palettes
@thi.ng/dgraph-dot		changelog	Dependency graph -> Graphviz
@thi.ng/distance-transform		changelog	Image based distance field generation
@thi.ng/fuzzy-viz		changelog	Visualization, instrumentation for @thi.ng/fuzzy
@thi.ng/geom		changelog	2D only geometry types & ops
@thi.ng/geom-accel		changelog	Spatial indexing data structures
@thi.ng/geom-api		changelog	Shared types & interfaces
@thi.ng/geom-axidraw		changelog	Shape conversions for AxiDraw pen plotter
@thi.ng/geom-arc		changelog	2D elliptic arc utils
@thi.ng/geom-clip-line		changelog	2D line clipping
@thi.ng/geom-clip-poly		changelog	2D convex polygon clipping
@thi.ng/geom-closest-point		changelog	Closest point helpers
@thi.ng/geom-fuzz		changelog	Fuzzy 2D shape drawing / filling
@thi.ng/geom-hull		changelog	2D convex hull (Graham scan)
@thi.ng/geom-isec		changelog	Point & shape intersection tests
@thi.ng/geom-isoline		changelog	2D contour line extraction
@thi.ng/geom-poly-utils		changelog	2D polygon helpers
@thi.ng/geom-resample		changelog	nD polyline / curve resampling
@thi.ng/geom-sdf		changelog	2D SDF creation, conversions, operators, utilities
@thi.ng/geom-splines		changelog	nD cubic / quadratic spline ops
@thi.ng/geom-subdiv-curve		changelog	nD iterative subdivision curves
@thi.ng/geom-tessellate		changelog	nD convex polygon tessellators
@thi.ng/geom-trace-bitmap		changelog	bitmap image to vector conversion
@thi.ng/geom-voronoi		changelog	2D iterative delaunay/voronoi
@thi.ng/imago		changelog	Declarative & extensible image processing pipelines
@thi.ng/lsys		changelog	Extensible L-System architecture
@thi.ng/pixel		changelog	Multi-format pixel buffers
@thi.ng/pixel-dither		changelog	Image dithering w/ various algorithm presets
@thi.ng/poisson		changelog	nD Poisson disk sampling
@thi.ng/porter-duff		changelog	Alpha blending / compositing ops
@thi.ng/rasterize		changelog	Shape drawing, filling & rasterization
@thi.ng/scenegraph		changelog	Extensible 2D/3D scenegraph
@thi.ng/simd		changelog	WebAssembly SIMD vector batch processing
@thi.ng/viz		changelog	Declarative & functional data visualization toolkit

WebGL / GPGPU

Project	Version	Changelog	Description
@thi.ng/shader-ast		changelog	AST DSL for x-platform shader code
@thi.ng/shader-ast-glsl		changelog	GLSL code generator
@thi.ng/shader-ast-js		changelog	JS code generator
@thi.ng/shader-ast-optimize		changelog	AST code optimization strategies
@thi.ng/shader-ast-stdlib		changelog	100+ useful AST shader functions
@thi.ng/webgl		changelog	WebGL 1/2 / GPGPU facilities
@thi.ng/webgl-msdf		changelog	MSDF font rendering
@thi.ng/webgl-shadertoy		changelog	Shadertoy-like WebGL setup

Low-level, binary, memory management, interop

Project	Version	Changelog	Description
@thi.ng/base-n		changelog	Arbitrary base-n encoding/decoding with presets
@thi.ng/binary		changelog	Assorted binary / bitwise ops, utilities
@thi.ng/bitstream		changelog	Bitwise input / output streams
@thi.ng/dlogic		changelog	Digital logic ops / constructs
@thi.ng/leb128		changelog	WASM based LEB128 varint encoder / decoder
@thi.ng/malloc		changelog	Raw & typed array memory pool & allocator
@thi.ng/morton		changelog	Z-order-curve / Morton coding
@thi.ng/range-coder		changelog	Binary data Range encoder / decoder
@thi.ng/rle-pack		changelog	Run-length encoding data compression
@thi.ng/soa		changelog	Memory mapped data structures & serialization
@thi.ng/unionstruct		changelog	Wrapper for C-like structs / unions
@thi.ng/vector-pools		changelog	Data structures for memory mapped vectors
@thi.ng/wasm-api		changelog	Modular, extensible JS/WebAssembly bridge API
@thi.ng/wasm-api-bindgen		changelog	Polyglot bindings code generator for C/Zig/TS
@thi.ng/wasm-api-canvas		changelog	WASM-side Canvas2D API bindings
@thi.ng/wasm-api-dom		changelog	WASM-side DOM manipulation
@thi.ng/wasm-api-schedule		changelog	WASM-side delayed code execution/scheduling

DSLs

Project	Version	Changelog	Description
@thi.ng/pointfree		changelog	Stack-based DSL & functional composition
@thi.ng/pointfree-lang		changelog	Forth-like syntax layer for @thi.ng/pointfree
@thi.ng/sexpr		changelog	S-Expression parser & runtime infrastructure

Building

git clone https://github.com/thi-ng/umbrella.git
cd umbrella

yarn install
yarn build

Once the entire mono-repo has been fully built at least once before, individual packages can then be (re)built like so:

yarn workspace @thi.ng/transducers run build

# or

(cd packages/transducers && yarn build)

# or

(cd packages/transducers && yarn build:esbuild)

Note: The yarn build script alias will also generate TS type declaration files. This step is only needed if updating the public API of a package. If you're confident it's not needed, using the build:esbuild alias is sufficient and much faster. Also, TS declaration files can be manually rebuilt via build:decl...

Building example projects

Please see the example build instructions in the wiki for further details.

Testing

(Most, but not all packages have tests)

Tests for almost all packages are run via bun.sh. For a couple of packages we're still using our own minimal test runner @thi.ng/testament

yarn test

# or individually
yarn workspace @thi.ng/rstream run test

Documentation

Autogenerated documentation (using TypeDoc) will be saved under /packages/*/doc/ and is also available at docs.thi.ng.

yarn doc

Extracting code examples from readme files & comments

All packages in this repo have prepared infrastructure to extract various code examples & snippets from their README files and from comments in the source code. Altogether, there're approx. 800 of them in this repo, but currently only ~15-20% are prepared for extraction so far (it's an ongoing timeconsuming manual process to prepare & check each of them, but work is under way!)

The code extraction is handled via thi.ng/tangle, itself a part of thi.ng/umbrella. You can read more details about this process here:

• Mastodon post #1
• Mastodon post #2
• thi.ng/tangle readme

To extract code blocks as source files from readmes:

# in the repo root (to process all packages)
yarn doc:readme

# for a single package only
(cd packages/<name> && yarn doc:readme)

To extract example code blocks from doc strings (API docs) in source code:

# in the repo root (to process all packages)
yarn tool:tangle

# for a single package only
(cd packages/<name> && yarn tool:tangle)

In all cases, the extracted files will be saved in each package's /export folder and can then be run directly via bun:

bun packages/arrays/export/topo-sort.ts

Acknowledgements

Maintaining a large monorepo like this requires a lot of infrastructure and I'm grateful for the tooling provided by the following projects to simplify those tasks:

• bun
• esbuild
• html-minifier-terser
• typedoc
• TypeScript
• Vite
• @thi.ng/monopub

License

© 2015 - 2024 Karsten Schmidt // Apache Software License 2.0

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Karsten Schmidt 💻 📖 🚧 💵	Alberto 💻 💡 🐛 🤔 💵	Arthur Carabott 💻 🤔 💡 📝 💵	André Wachter 💻 🤔 🐛	Gavin Cannizzaro 💻 🐛 🤔	Logan Powell 📖 🐛 🤔 💵	Marcin Ignac 🐛
arcticnoah 💻	allforabit 🐛 💻 🤔 💵	Yifan Wu 🐛 📖	stwind 💻 🐛	evilive 💻	Bnaya Peretz 💻 🐛 🤔	oljeger 🐛
Nik Shevchenko 🐛 💻	Matei Adriel 💻 🐛 🤔	Pierre Grimaud 📖	Matt Huebert 💵	Raphael Saunier 💵	Eric Ihli 💵	David Pham 💵
TBD 🐛 🤔	Pedro Henriques dos Santos Teixeira 💵	Jamie Owen 💻 🐛	Robert Kesteson 🐛 💻	Chancy Kennedy 💵	Jarred Sumner 🐛	Jamie Slome 🐛 🛡️
d3v53c 🐛 🛡️	Jannis Pohlmann 🐛	Shakthi Prasad G S 🐛 💻	Robin Gower 🐛	Michael Latzoni 🐛	Z Yin 🐛	Damien Seguin 🐛 💻
Rui Gil 🐛	Ja|nz 💻 🚇 🚧	Tyler Freeman 🐛 💻	blackhuman 🐛	David Negstad 💻	Muhammad Ridho 🐛 💻	MarcusWagberg 💻
Masashi Yoshikawa 🐛	Guido Schmidt 🐛 💵	tsukamotohideki 💵	Dave Meehan 💻 🤔	Aurélien Bottazini 🐛	Alex 🐛 💵	Ross Cairns 💵
Ruslan Prakapchuk 💵	Jarrod Davis 💵	Nicolas Lebrun 💡	Dawid Górny 💵	Michael Reitzenstein 💵	Sam Nosenzo 💵	Igor Loskutov 💻 🤔
Yury 💻 🤔 💵	Jean-Frédéric Faust 💻 🐛

This project follows the all-contributors specification. Contributions of any kind welcome!

umbrella's Issues

[transducers-stats] Build fail with UglifyJSPlugin (Webpack)

It all works fine with dev, but when you build the production:

ERROR in app/vendor.e8d8a5c89728897d6c01.bundle.js from UglifyJs
Unexpected token: name (r) [app/vendor.e8d8a5c89728897d6c01.bundle.js:1,302965]

[rstream] fromPromise() error handling

Attach then() handler outside the stream source and cache result in closure. That way errors will always be caught, not just once we have a subscriber and the stream source fn has been called.

[hdom-canvas] update arc handling & SVG compatibility

Due to an earlier oversight/misunderstanding, currently hdom-canvas paths only support circular arcs (via arcTo() canvas API function), which are not compatible with their SVG counterparts (elliptic arcs). The current workaround is to use @thi.ng/geom to convert such paths to cubic splines / polylines, but we should integrate elliptic arc path segments handling directly, since actually available.

Current proposal:

1. When processing A or a segments, check num args to determine circular vs. elliptic arc
2. If elliptic arc, use geom's arcFrom2Points() function to compute arc center and start/end theta, required for canvas API ellipse() function

Issues:

• introduces new dependency on @thi.ng/geom package

Is it maybe worth to extract low-level parts of geom package into their own to avoid bloat?

[csp] Why does the channel log every consume?

Why does the channel console.log() messages every time it consumes? Won't this cause performance issues? channel.read() by itself doesn't remove the message from the channel.

Also how can you make it so that await channel.write(value) doesn't resolve until something on the channels reads (a channel with no buffer, ie: the default in Go).

[hdom] IE11 Support

IE doesn't support ChildNode.remove() (and probably other things) and throws TypeError: Object doesn't support property or method 'remove' here

umbrella/packages/hdom/src/dom.ts

Line 307 in e36a7ee

n.remove();

Any particular reason not to use parent.removeChild(child)?

In the meantime I'm using polyfill from here https://developer.mozilla.org/en-US/docs/Web/API/ChildNode/remove#Polyfill

[hdom-canvas] add gradient support

Just recording some notes about adding linear & radial gradient support:

1. Add SVG-like gradient def elements to construct gradient object in canvas ctx:

["linearGradient", {id: "foo", from: [0,0], to: [100,100]}, [0, "#f00"], [1, "#0cf"]]

2. Gradient defs will need to be stored in DrawState (add gradients field)

3. Use $ prefix for gradient refs and add special handling for stroke & fill shape attribs, i.e. to check for $foo

Logo

Hi. I am a graphic designer. I volunteer to design a logo for open source projects. I can design it for you to use it in the readme file. What dou you say?

[hdom-canvas][hiccup-svg] syntax alignment

The hdom-canvas package relies on hiccup-encoded shape elements, similar to their SVG counterparts, however with the following differences:

// hdom-canvas
// only style information in attribs
// geometry / points as raw data
["circle", { fill: "red" }, [0,0], 100]
["polygon", { stroke: "blue" }, [[0,0], [100,0], [100,100]]]

// svg
// both style & geometry are given as stringified attribs
["circle", { fill: "red", cx: 0, cy: 0, r: 100}]
["polygon", { stroke: "blue", points: "0,0 100,0, 100,100"}]

Additionally, hiccup-svg currently provides wrapper functions to create elements like the hiccup SVG version above, whereas hdom-canvas doesn't (so far).

For DRY purposes and to support use cases where both high-performance canvas drawing and SVG output are required, users should not have to choose between two similar-but-not-quite-the-same flavors. Since it's easier to turn the hdom-canvas syntax into the semi-stringified version required for SVG serialization, it would make most sense to use that format as the canonical version for both routes and then provide some form of "adapter" / converter to recursively convert such a scene tree into the SVG-friendly hiccup flavor...

[rstream-gestures] enhancement: widgetGestureStream

My main use case for gesture streams is for a number of canvas UI elements on a page that can be scrolled. The current gestureStream doesn't quite support this because:

Streams only return results when inside the element

Usually an interaction should only start when clicked/touched inside the element, but dragging should still work outside of the element. Attaching the gesture to the element means this doesn't work.

Attaching to the body prevents scrolling

To make the above work, you can attach the event to the document.body and calculate the x/y offset from the element. The problem is that preventDefault is required while interacting (to prevent scrolling), but when applied to the whole body, it prevents scrolling when not touching the element

Solution

I'm using this stream function (could do with a better name!), do you think it would make a good addition to the lib?

import { clamp } from "@thi.ng/math";
import { GestureStreamOpts, GestureType } from "@thi.ng/rstream-gestures";
import { fromEvent } from "@thi.ng/rstream/from/event";
import { merge, StreamMerge } from "@thi.ng/rstream/stream-merge";
import { filter, map } from "@thi.ng/transducers";
import { comp } from "@thi.ng/transducers/func/comp";

export interface WidgetGestureInfo {
  pos: number[];
  click?: number[];
  delta?: number[];
  zoom: number;
  isInsideEl: boolean;
}

export interface WidgetGestureEvent {
  type: GestureType;
  info: WidgetGestureInfo;
}

export interface WidgetGestureStreamOpts extends GestureStreamOpts {
  /**
   * If true (default: true), scroll events on the element will prevent the
   * document from scrolling.
   * If false, the scroll event will use the eventOpts.passive argument
   * (default: true) which should be true in most cases for performance reasons:
   * https://www.chromestatus.com/feature/5745543795965952
   */

  preventScrollOnZoom: boolean;
}

export function widgetGestureStream(el: HTMLElement, stopEl: HTMLElement,
                                    userOpts?: Partial<WidgetGestureStreamOpts>):
                                    StreamMerge<any, WidgetGestureEvent> {
  const defaultOptions: WidgetGestureStreamOpts = {
    id: "gestures",
    zoom: 1,
    minZoom: 0.25,
    maxZoom: 4,
    smooth: 1,
    eventOpts: { capture: true, passive: true },
    preventScrollOnZoom: true,
    preventDefault: true,
    local: true,
    scale: false,
  };

  const opts = Object.assign(defaultOptions, userOpts);
  const devicePixelRatio = window.devicePixelRatio || 1;

  let isDown = false;
  let clickPos: number[] = [0, 0];
  let zoom = Math.min(Math.max(opts.zoom, opts.minZoom), opts.maxZoom);

  return merge({
    id: opts.id,

    src: [
      "mousedown", "mousemove", "mouseup",
      "touchstart", "touchmove", "touchend", "touchcancel",
      "wheel",
    ].map((e) => {
      const eventOpts = typeof opts.eventOpts === "boolean"
        ? { passive: opts.eventOpts }
        : Object.assign({}, opts.eventOpts);

      if (e === "wheel") { eventOpts.passive = eventOpts.passive && !opts.preventScrollOnZoom; }
      return fromEvent(stopEl, e, eventOpts);
    }),

    xform: comp(
      map((e: MouseEvent | TouchEvent | WheelEvent): WidgetGestureEvent => {
        const { type, evt } = (e as TouchEvent).touches === undefined
          ? {
            type: {
              mousedown: GestureType.START,
              mousemove: isDown ? GestureType.DRAG : GestureType.MOVE,
              mouseup: GestureType.END,
              wheel: GestureType.ZOOM,
            }[e.type],

            evt: (e as MouseEvent | WheelEvent),
          }
          : {
            type: {
              touchstart: GestureType.START,
              touchmove: GestureType.DRAG,
              touchend: GestureType.END,
              touchcancel: GestureType.END,
            }[e.type],

            evt: (e as TouchEvent).changedTouches[0],
          };

        const pos = [evt.clientX, evt.clientY];
        const rect = el.getBoundingClientRect();
        const isInsideEl = pos[0] >= rect.left && pos[0] <= rect.right &&
                           pos[1] >= rect.top  && pos[1] <= rect.bottom;

        if (opts.local) {
          pos[0] -= rect.left;
          pos[1] -= rect.top;
        }

        if (opts.scale) {
          pos[0] *= devicePixelRatio;
          pos[1] *= devicePixelRatio;
        }

        const info: WidgetGestureInfo = { pos, zoom, isInsideEl };
        if (type === GestureType.START) {
          if (!isInsideEl) { return; }
          if (opts.preventDefault) { e.preventDefault(); }

          isDown = true;
          clickPos = [...pos];
        }
        else if (type === GestureType.END) {
          if (!isDown) { return; }
          if (opts.preventDefault) { e.preventDefault(); }

          isDown = false;
        }
        else if (type === GestureType.DRAG) {
          if (!isDown) { return; }
          if (opts.preventDefault) { e.preventDefault(); }

          info.click = clickPos;
          info.delta = [pos[0] - clickPos[0], pos[1] - clickPos[1]];
        }
        else if (type === GestureType.ZOOM) {
          if (!isInsideEl) { return; }
          if (opts.preventDefault && opts.preventScrollOnZoom) { e.preventDefault(); }

          zoom = clamp(zoom + (e as WheelEvent).deltaY * opts.smooth, opts.minZoom, opts.maxZoom);
          info.zoom = zoom;
        }

        return { type, info };
      }),
      filter(e => e !== undefined),
    ),
  });
}

Example:

import { canvas } from "@thi.ng/hdom-canvas";
import { clamp } from "@thi.ng/math";
import { GestureType } from "@thi.ng/rstream-gestures";
import { fromRAF } from "@thi.ng/rstream/from/raf";
import { merge } from "@thi.ng/rstream/stream-merge";
import { trigger } from "@thi.ng/rstream/trigger";
import { labeled, range } from "@thi.ng/transducers";
import { updateDOM } from "@thi.ng/transducers-hdom";
import { map } from "@thi.ng/transducers/xform/map";

import { WidgetGestureEvent, widgetGestureStream } from "./widgetGestureStream";

const app = (_stream) => {
  function canvasFactory(id: string) {
    return {
      ...canvas,
      init: (el: HTMLCanvasElement) => {
        _stream.add(
          widgetGestureStream(el, document.body, { preventScrollOnZoom: true }).transform(labeled(id)),
          id,
        );
      },
    };
  }

  const width = 355;
  const height = 200;

  const numCanvases = 10;
  const canvasIds = Array(...Array(numCanvases)).map((_, i) => `gesture-${i}`);
  const canvasComponents = new Map(canvasIds.map((id): [string, ReturnType<typeof canvasFactory>] =>
    [id, canvasFactory(id)]));
  const positions = new Map(canvasIds.map((id): [string, number[]] =>
    [id, [width / 2, height / 2]]));
  const radii = new Map(canvasIds.map((id): [string, number] => [id, 20]));


  return (eventStream) => {
    const eventLabel: string = eventStream[0];
    if (eventLabel.startsWith("gesture")) {
      const { type, info } = (eventStream[1] as WidgetGestureEvent);

      if (type === GestureType.DRAG) {
        positions.set(eventLabel, [
          clamp(info.pos[0], 0, width),
          clamp(info.pos[1], 0, height),
        ]);
      }

      if (type === GestureType.ZOOM) {
        radii.set(eventLabel, 20 * info.zoom);
      }
    }

    return ["div",
      map(([id, canvasComponent]) => ["div",
        [canvasComponent,
          { width, height },
          ["circle",
            { fill: "black" },
            positions.get(id),
            20 + radii.get(id) * Math.abs(Math.sin(Date.now() * 0.001)),
          ],
        ],
        ["div", { style: { height: "75px" } }, "spacer"],
      ], canvasComponents),
    ];
  };
};

const stream = merge({
  src: [
    fromRAF().transform(labeled("time")),
    trigger().transform(labeled("trigger")),
  ],
});

stream.transform(
  map(app(stream)),
  updateDOM(),
);

[rstream-gestures] incorrect local offset

rstream-gestures uses el.offsetLeft and el.offsetTop to calculate local positions (https://github.com/thi-ng/umbrella/blob/master/packages/rstream-gestures/src/index.ts#L143)

Unfortunately this fails when the page is scrolled, see example:

import { canvas } from "@thi.ng/hdom-canvas";
import { GestureEvent, gestureStream, GestureType } from "@thi.ng/rstream-gestures";
import { merge } from "@thi.ng/rstream/stream-merge";
import { trigger } from "@thi.ng/rstream/trigger";
import { labeled } from "@thi.ng/transducers";
import { updateDOM } from "@thi.ng/transducers-hdom";
import { filter } from "@thi.ng/transducers/xform/filter";
import { map } from "@thi.ng/transducers/xform/map";

const app = (_stream) => {
  const _canvas = {
    ...canvas,
    init: (el: HTMLCanvasElement) => {
      _stream.add(
        gestureStream(el, { local: true })
          .transform(
            filter((e: GestureEvent) => e[0] === GestureType.START ||
                                        e[0] === GestureType.DRAG),
            labeled("local"),
          ),
        "local",
      );

      _stream.add(
        gestureStream(el, { local: false })
          .transform(
            filter((e: GestureEvent) => e[0] === GestureType.START ||
                                        e[0] === GestureType.DRAG),
            map(e => {
              const pos = e[1].pos;
              const rect = el.getBoundingClientRect();
              return [pos[0] - rect.left, pos[1] - rect.top];
            }),
            labeled("clientRect"),
          ),
        "clientRect",
      );
    },
  };

  let localPos = [300, 30];
  let clientRectPos = [100, 30];

  const marker = (label: string, pos: number[], radius: number, fill: string, offset: number) => {
    return [
      ["circle", { fill }, pos, radius],
      ["text", { fill, align: "left", baseline: "middle" }, [pos[0] + offset, pos[1]], label],
    ];
  };

  return (eventStream) => {
    if (eventStream[0] === "local") { localPos = eventStream[1][1].pos; }
    if (eventStream[0] === "clientRect") { clientRectPos = eventStream[1]; }

    return ["div",
      ["p", "Click and drag in the rectangle"],
      ["p", "Then scroll down and try the same!"],
      [_canvas,
        { width: 400, height: 200, style: { "margin-top": `${window.innerHeight - 200}px` }},

        marker("local", localPos, 30, "black", 35),
        marker("clientRect", clientRectPos, 20, "green", -80),
      ],
    ];
  };
};

const stream = merge({
  src: [ trigger().transform(labeled("trigger")) ],
});

stream.transform(
  map(app(stream)),
  updateDOM(),
);

Solution is to use el.getBoundingClientRect() instead, working on a patch now...

rfn is not iterable

Hi,

Tried running the example from the readme but it raises an exception:

import { Channel } from "@thi.ng/csp";
import * as tx from "@thi.ng/transducers";

Channel.merge([
  Channel.range(0, 3),
  Channel.range(10, 15),
  Channel.range(100, 110)
]).reduce(tx.push).then(console.log);

channel.js:363 Uncaught (in promise) TypeError: rfn is not iterable
    at Channel.<anonymous> (channel.js:363)

Support context in hdom

Maybe there's already a way of achieving this using the existing api or via native es6 methods but I am looking to pass values down deeply into the hdom tree. Thus far I have explored ideas of a higher order function to mimic behaviour found in the redux connect method. The idea of this is to wrap the component and pass in relevant props to the child. To do this, it uses context to pull out the store which has been injected higher up in the component tree (using a HOC called a provider). It's this second part that I'm hitting a stumbling block with.

From digging into the code I can see that the bulk of the work that is carried out to render the hiccup with hdom is done in the normalizeTree function. Would this be a good place to add the possibility to inject globally available state (well globally to the hdom tree).

I understand that an option would be to make the state available in an outer scope but I am hoping to avoid this so that everything is as self contained as possible in a similar way that redux components are.

I don't see a way of adding labels here, I'm wondering if I have access? Thanks!

[@thi.ng/interceptors] Slight error in example

There's a slight error in the router basics example that might lead to some confusion in the following file: https://github.com/thi-ng/umbrella/blob/master/examples/router-basics/src/app.ts. The code:

this.ctx.bus.addHandlers({
            [EVENT_ROUTE_CHANGED]: valueSetter("route"),
            [ev.ROUTE_TO]: (_, [__, route]) => ({ [ev.ROUTE_TO]: route })
        });

should be:

this.ctx.bus.addHandlers({
            [EVENT_ROUTE_CHANGED]: valueSetter("route"),
            [ev.ROUTE_TO]: (_, [__, route]) => ({ [fx.ROUTE_TO]: route })
        });

So it should be using fx.ROUTE_TO and not ev.ROUTE_TO. It actually still works because they both coincidentally have the same string identifier. It would probably lead to confusion though when trying to understand the example. I've have a little trouble trying to mentally figure out the differences between events, event handlers, interceptors and effects which is why I noticed the error.

On a related note, again this is being very pernickity but could help to make a more solid and clear division between effects and interceptors. The trace interceptor is really an interceptor and an effect handler rolled into one. Maybe it would be better to divide this out into an interceptor and a trace effect? To enforce this division more strictly, perhaps interceptors shouldn't be allowed to return undefined? Although I'm sure there's a good reason that you allow for that.

[hdom] - iterators disappear after initial render

I was looking for a shorter way to write components that dynamically create inner child-nodes from a collection.

For example, a basic list component:

[
  "ul",
  tx.transduce(tx.map(i => ["li", i]), tx.push(), tx.range(1, 6))
]

The transduce transduce and the push() seem superfluous to me. I tried:

["ul", tx.map(i => ["li", i], tx.range(1, 6))]

and

["ul", tx.iterator(tx.map(i => ["li", i]), [...tx.range(1, 6)])],

to compose transducers.

Both approaches work fine on first render and then never again. I assume that's because the overloaded map and iterator turn into generator functions? Generator functions are probably not very useful for hdom if they can only run once. However, it looks like iterators can also be computed several times though I'm not sure that's desirable.

Getting back to the issue, is there a shorter way to express the list component above?

[all] Support multiple distribution formats

As per Discord chat, we should look into easy-to-maintain solutions supporting distributions in different flavors.

Proposed initial solution to follow along these lines:
https://www.reddit.com/r/javascript/comments/6ngjkk/how_to_distribute_library_written_with_es6/dka3g5u/

[rstream-dot] refactor to use @thi.ng/dot

The new @thi.ng/dot package is more general and offers better support for various Graphviz options, incl. node record shapes with labeled inputs/outputs. rstream-dot should be rewritten using these features.

• replace DotOpts with GraphAttribs from @thi.ng/dot
• update walk process with special support for StreamSync, StreamMerge and possibly PubSub to make use of multi-input/output nodes for cleaner visualizations

The issue with that 2nd point is that StreamSync and StreamMerge use internal/hidden subscriptions, which in the graphviz model will need to be translated as ports of the owner node. This breaks the 1:1 relationship between actual graph topology & the visualization, but will be easier and less confusing to read, however makes the whole graph traversal and edge collection harder...

walk() might have to be rewritten to use breadth-first (rather than the current depth-first) traversal, in order to correctly map StreamSync/Merge nodes in the new/proposed model.

[hiccup-dom] add mechanism to cancel RAF update loop

Reference & use case: https://twitter.com/denik/status/959890745367900160 by @den1k

[rstream] sync with req and opt sources?

I was implementing a file-browser and ran into the following issue:

The file-browser views a list of files and lets the user select on of them. It has files and selectedFile streams as it's sources.

const files = rs.stream()
const selectedFile = rs.stream()
const selectedFileData = selectedFile.subscribe(tx.map(processFile...))

const fileBrowser = rs.sync({
  src: { files, selectedFile },
  xform: tx.map(src => [
    "ul",
    tx.iterator(
      tx.comp(
        tx.take(5),
        tx.map(fl => [
          "li",
          {
            onclick: () => selectedFile.next(fl),
            style: {
              "font-weight": fl == src.selectedFile.name ? "bold" : "normal"
            }
          },
          fl.name
        ])
      ),
      src.files
    )
  ])
});

fileBrowser won't return until selectedFile got a next value. Usually, like in the commit-table-ssr example, one could just run selectedFile.next(null). But here the problem is that several transformations subscribe to selectedFile that would then need logic to terminate early in order not to error if the value isn't a file. That's not optimal. However, if selectedFile is changed by another component, it's still desirable for it to update fileBrowser.

So I'm wondering if sync could have req and opt (names inspired by clojure.spec) sources, where the first result is emitted when all req sources returned a value and following results whenever any of the values returns a value.

[sax] HTML specific parser additions / fixes

• Add support for HTML5 boolean attribs (present or not) - update ATTRIB_NAME state
• Wrap text as node to support cases like: <a>a1<b>b1</b>a2</a> - currently fails @ a2
• Add option to define void tags not requiring /> (list & spec)

hiccup-css: question about how to use pseudo classes

Hi Karsten 👋

First off thanks for all the work on these packages! I'm finding them very useful for a lot of things at the moment.

One thing I just ran into though with the hiccup-css package, and perhaps it's an oversight on my part and I simply haven't come across the corresponding code/example, but is there a way to use css pseudo classes with the package? I'd like to set a hover state for something but I don't see an obvious way to do so.

Thanks!

[vectors] static scaleWithCenter produces NaNs

import { Mat23, Vec2 } from "@thi.ng/vectors";
console.log(Mat23.scaleWithCenter(new Vec2([1, 1]), 1, 1).buf);
// [1, 0, 0, 1, NaN, NaN]

I believe it's because the im param (matrix index?) is missing in the call to scaleWithCenter23. Default parameters considered harmful? :)

Putting in a fix now...

Add support for array of children

Tag: hiccup-dom

The following code

var grid = require('grid-cells')
var cells = grid(1024, 512, 4, 2, 10);
var parent = ["div", {}]
parent.push(['svg', { width: 512, height: 512 },
  cells.map((cell, index) => {
        var i = index % nx
        var j = Math.floor(index / nx)
        return ['rect', { x: cell[0], y: cell[1], width: cell[2], height: cell[3] }]
     })
  ])

Creates data as follows

And fails with error

That's because svg first child is an array of items

var element = ["svg", {}, [
 ["rect", {}], ["rect", {}], ["rect", {}], ...
]

which hiccup seems to interpret as invalid element without name.

The are multiple possible fixes at the moment:

1. Use spread to unnest array

...cells.map((cell, index) => {
}

as it produces

2. Push or reduce to an array initialized with a wrapper tag name

  cells.reduce((children, cell, index) => {
        var i = index % nx
        var j = Math.floor(index / nx)
        children.push(['rect', { x: cell[0], y: cell[1], width: cell[2], height: cell[3] }])
        return children
     }, ["g"])

as it produces

[rstream] infinite StreamMerge

Related to #6 - add option to StreamMerge to avoid going into DONE state when last input is done.

[umbrella/hdom] - innerHTML as attrib?

in React one can set the innerHTML of a component from a string through dangerouslySetInnerHTML.__html on a component's props. Is there a way to do this in hdom?

Example:

["div", {__html: "abc"}]

renders => <div>abc</div>

[hdom] generalize diffElement()

Currently diffElement() used hardcoded delegates for any mutation tasks (adding / removing / replacing elements / attribs) resulting from the computed diff. This means the implementation is only working with an actual DOM. There's no reason to artificially limit the project to purely this use case and we also need better mocking for tests.

Refactor to support alternative impls (e.g. for test mocks, WebGL) by passing in optional implementation config. Also update start() to accept new arg...

Channel.SCHEDULE cpu usage

Hello,

When using Channel.select in the browser, I'm noticing high cpu usage.
Is this "expected" from the library / are there any best practices for this?

I was wondering if setImmediate is the way to go on browser environments.
I was hoping that await/async could help on this to avoid pulling..

thanks for the great work!

hiccup-dom status?

Hi there!

hiccup-dom looking neat, was missing something like that after jumping cljs -> jsx.

Is the project still to be considered alpha?

Would be a great thing to add that to the README.

Cheers - Victor

[examples/commit-table-ssr] - yarn dev errors

Can't run the code using the steps in the readme:

git clone https://github.com/thi-ng/umbrella.git
cd umbrella/examples/commit-table-ssr
yarn install
yarn dev
Cloning into 'umbrella'...
remote: Counting objects: 18184, done.
remote: Compressing objects: 100% (602/602), done.
remote: Total 18184 (delta 1062), reused 1316 (delta 964), pack-reused 16591
Receiving objects: 100% (18184/18184), 26.15 MiB | 4.19 MiB/s, done.
Resolving deltas: 100% (13898/13898), done.
yarn install v1.7.0
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 31.43s.
yarn run v1.7.0
$ tsc && node build/server/index.js
node_modules/@thi.ng/transducers/rfn/group-binary.d.ts:74:181 - error TS2307: Cannot find module '@thi.ng/transducers/src/api'.

74 export declare function groupBinary<T>(bits: number, key: (x: T) => number, branch?: () => IObjectOf<T[]>, leaf?: Reducer<any, T>, left?: PropertyKey, right?: PropertyKey): import("@thi.ng/transducers/src/api").ReductionFn<any, T>[];
                                                                                                                                                                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

src/client/index.ts:6:26 - error TS2307: Cannot find module '@thi.ng/transducers-hdom'.

6 import { updateUI } from "@thi.ng/transducers-hdom";
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~

error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[atom] Async effects in events bus

This is related to the previous issue but it's probably separate enough to warrant it's own issue.

How would you go about adding an async event such as an an ajax request? I can't see a way to do it via the event handler config because they don't get a handle on the event bus. They only get the state and event. I can see that you could do it after the event bus is instantiated but it would be nice to be able to be able to do it through the event config. It would probably need to be another built in effect similar to "dispatch" that might look like this: [FX_DISPATCH_ASYNC]: {effect: [FETCH_JSON: {url: "http://thi.ng/feed.json"}], handler: "myHandler"}. The effect function would then need to take a callback as an argument to callback into the event system.

I can attempt a pull request for this!

[hdom] update diffTree docs

Where do i get HDOMImplementation for the new diffTree implementation?

I've been using hdom@3 for quite while as follows

const curr = normalizeTree(render, null, [0], true, true)
if (curr !== null) {
    diffElement(rootElement, prev, curr)
    prev = curr
}

I'm trying to port it to v5 and use diffTree as before

Takes a DOM root element and two hiccup trees, prev and curr.

but i'm getting Trigger "in" failed: undefined is not an object (evaluating 'curr[1]')

const curr = normalizeTree({}, render)
if (curr !== null) {
    diffTree(rootElement, prev, curr)
    prev = curr
}

after looking at the source I can see that first param is opts like in normalizeTree but second is impl:HDOMImplementation and i'm not sure where do i get that from.

[hiccup, hdom](examples/commit-table-ssr) - react-like hydrate?

Just got around to reading the code and it looks like the browser and SSR version are separate in the sense that unlike react's hydrate we get either one or the other but not the best of both worlds. hydrate is what I meant to express (admittedly, through a handshake emoji) in this tweet.

Is there a hydration story for hiccup / hdom?

[examples] update .gitignore files

Remove *.html from .gitignore files, since they're currently never shown and I keep forgetting to check them in manually. This has repeatedly caused unexpected issues with users... e.g. #46

[resolve-map] update lookup path separators

Currently, value reference paths for resolveMap() are written like:

• ->../../foo.bar (relative) or
• ->/foo.bar (absolute)

This is because we're using @thi.ng/paths toPath() for the substring after the last slash and that function uses . as separator.

Proposal: Update path syntax to always use / as sepatator in this package so that:

->../../foo.bar.baz becomes ->../../foo/bar/baz

Possibly update toPath() to accept optional custom separator arg, but might not be needed

[hiccup / hiccup-svg] add support for context object

In order to keep hiccup & hiccup-svg compatible with components created for hdom, we need to update its handling of component fns and update serialize() to accept an additional / optional context arg, which is then passed to all embedded component fns as first arg (as done in hdom). Currently this is broken for components relying on the recent changes done in hdom.

Updating SVG components in hiccup-svg with the extra context arg will most likely result in breaking changes, but seemingly unavoidable and better done rather sooner than later...

[math] fitClamped with range of 0 results in NaN

Because norm does a divide by range, it is possible to get NaNs, e.g.

const minNote = 10;
const maxNote = 10;
const y = fitClamped(note.note, minNote, maxNote, 0, 1000);

The use case is where the input min and max are unknown or computed.

It may ruin the elegance of the 1 liners, but a check such as this would avoid this

if (x <= a) { return c; }
if (x >= b) { return d; }
...usual code

[rstream] - Hot Module Replacement

Developing an app using a few rs.streams to hold state. Currently working on getting HMR to work. @postspectacular any pointers or ideas?

Will post progress here.

[hiccup/hdom] support clj/cljs vectors/maps

Currently the hiccup/hdom tree normalization stage assumes components are defined using native JS types only (arrays / objects), which precludes idiomatic usage in Clojure/ClojureScript, where one would usually use persistent vectors & maps to define components.

Adding support for these custom types could be done in several ways:

1. Add additional type checks for namespaced clojure.core.* types / protocols

Pros: trivial implementation
Cons: Additional work at runtime, possible performance impact

2. Port (part of) to Clojure/ClojureScript

Pros: Idiomatic usage, no perf impact (compared to 1)
Cons: implementation & maintenance effort

3. Refactor normalizeTree() / normalizeElement() to inject type checks via IOC / config object

Pros: most flexible (also potentially interesting for other langs)
Cons: none really?

Thoughts?

Tweet for reference: https://twitter.com/danpeddle/status/1034706731790729216

[examples] Getting a 404 for commit-table-ssr example

Hi Karsten, I'm really excited to see all the new developments with rstream and hdom. I'm having a little trouble running the commit-table-ssr example. I'm getting a 404 for the client code and the following text is output: Cannot GET /out/ I'm also getting this (possibly unrelated error) after yarn install:

Done in 0.09s.
cp: cannot stat 'commits.json': No such file or directory
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

P.s. I know I said I'd look into this issue a while back. #23 It's still on my radar and hope to get back to it soon! I've been busy doing react based dev work and unfortunately umbrella has taken a backseat for now.

[rstream] Meta streams

Is it possible to get subscriptions to generate new streams via transducers? I can see there's a 'resolve' subscription class to handle promises but I don't see an equivalent for streams. In other words, the final transducer would generate new streams which would be merged (using something similar to 'resolve') and could be subscribed to. This seems to be a common pattern with Rx and is done using the mergemap operator.

[thi.ng/atom] - Extending classes for Observablehq

I extended umbrella/atom to work idiomatically with observable (e.g. viewof and reactive cells), result here - this now works with deeply nested objects!

extended Atom code here

But I'm struggling to get cursors to work, I think because I can't pass my the extended Atom class in the Cursor impl: https://github.com/thi-ng/umbrella/blob/master/packages/atom/src/cursor.ts#L97

If successful, I'd love to do the same for rstream and derived libs!

[transducers-stats] Supported indicators MACD

Is the library supporting the MACD indicators?
If not, can indicators be made soon?

[all] Usage in beta.observablehq.com?

Discussion based on this tweet by @den1k:

do you think umbrella would play well in https://beta.observablehq.com/ ? since it has a reactive (but I think promise based) update model straightforward interpolation of HTML / svg, e.g. html<span>foo</span>

[hdom-components] ADR-0002 discussion

Please use this issue to discuss aspects and/or change requests to the component design decisions in ADR-0002

[resolve-map] ES6 object destructuring

Add a note to the readme and doc strings to make it more explicit that the object destructuring form only works for ES6 targets and fails (obviously) when transpiling to ES5, in which case the deps needs to be resolved via legacy approach...

Reference: https://twitter.com/denik/status/1034581588531654656

[transducers, resolve-map, rstream-graph] - xform steps in a graph?

imports to replicate:

import { Atom } from "@thi.ng/atom/atom";
import * as tx from "@thi.ng/transducers";
import * as txs from "@thi.ng/transducers-stats";
import * as rsg from "@thi.ng/rstream-graph";
import * as rs from "@thi.ng/rstream";

const data = [...tx.range(1, 21)].map(x => ({ date: x, close: x * 10 }));

As wonderful as transducers are the one thing they give mea headache with is nesting. For example, this transducer computes a moving average from a stream of ohlc data and merges it back into the original:

const anaTx = tx.comp(
  tx.multiplexObj({
    identity: tx.identity,
    hma: tx.comp(
      tx.pluck("close"),
      txs.hma(5)
    )
  }),
  tx.map(({ identity, ...all }) => ({ ...identity, ...all }))
);

Now imagine we also wanted to compute the deltas. No problem:

const deltaTx = tx.comp(tx.partition(2, 1), tx.map(([a, b]) => b - a));

const anaTx = tx.comp(
  tx.multiplexObj({
    identity: tx.identity,
    delta: tx.comp(tx.pluck("close"), deltaTx),
    hma: tx.comp(
      tx.pluck("close"),
      txs.hma(5)
    )
  }),
  tx.map(({ identity, ...all }) => ({ ...identity, ...all }))
);

Now say we wanted to compute the change of the delta relative to the current price. For that we need to access both the close price and the delta. To make this work with transducers alone and without recomputing the delta in another transducer, we need to chain another transducer and merge again:

const anaTx = tx.comp(
  tx.multiplexObj({
    identity: tx.identity,
    delta: tx.comp(tx.pluck("close"), deltaTx),
    hma: tx.comp(
      tx.pluck("close"),
      txs.hma(5)
    )
  }),
  tx.map(({ identity, ...all }) => ({ ...identity, ...all })),
  tx.multiplexObj({
    identity: tx.identity,
    deltaChange: tx.map(({close, delta}) => close / delta)
  }),
  tx.map(({ identity, ...all }) => ({ ...identity, ...all })),
);

We're duplicating more and more code. This gets even more complicated when we want to compute more complex values, like averages of averages. Besides the duplication, these are hard to compose. For example, the keys close and delta are hardcoded which would require a higher order function or another transducer to rename them if say low and delta is to be computed. This is where rstream-graph comes in:

const state = new Atom({});

const deltaTx = tx.comp(tx.partition(2, 1), tx.map(([a, b]) => b - a));

const graph = rsg.initGraph(state, {
  ohlc: () => rs.fromIterable(data),
  hma: {
    fn: rsg.node1(txs.hma(5)),
    ins: { src: { stream: "close/node" } }
  },
  delta: {
    fn: rsg.node1(deltaTx),
    ins: { src: { stream: "close/node" } }
  },
  close: {
    fn: rsg.node1(tx.pluck("close")),
    ins: { src: { stream: "ohlc" } }
  },
  deltaChange: {
    fn: rsg.node(tx.map(({ delta, close }) => delta / close)),
    ins: {
      delta: { stream: "delta/node" },
      close: { stream: "close/node" }
    }
  },
  ana: {
    fn: rsg.node(tx.map(({ohlc, ...all}) => ({...ohlc, ...all}))),
    ins: {
      ohlc: { stream: "ohlc" },
      hma: { stream: "/hma/node" },
      delta: { stream: "delta/node" },
      deltaChange: { stream: "/deltaChange/node" }
    }
  }
});

graph.ana.node.subscribe({
  next: x => console.log("result:", x)
});

But the problem is that graph.ana.node.subscribe triggers 4 times for each item, rightfully, once for every subscription. However, we're only interested in the value of the last update per item, since that one will contain the data for all transformations for the item.

Since I care about one result per item, I think what I'm really looking for is something like a tx.multiplexObj + resolve-map's resolve multiplexGraph transducer, within which transducers can reference each other and run in topological dependency order. Does that make sense?

[hdom] use rAF render loop only on state changes

On my machine start("app", () => ["div", "foo"]) idles at 5% CPU. It seems diffing happens regardless of state changes. Is there a way to only re-diff after an atom any atom inside has been mutated?

Add demo gif to README

Disclaimer: This is a bot

It looks like your repo is trending. The github_trending_videos Instgram account automatically shows the demo gifs of trending repos in Github.

Your README doesn't seem to have any demo gifs. Add one and the next time the parser runs it will pick it up and post it on its Instagram feed. If you don't want to just close this issue we won't bother you again.

[strings] SyntaxError: Invalid regular expression: invalid group specifier name

const kebab = (x, delim = "-") => lower(x.replace(/(?<=[a-z0-9\u00e0-\u00fd])(?=[A-Z\u00c0-\u00dd])/g, (_, i) => (i ? delim : "")));

Is broken in Safari and Firefox

Publish *.ts source?

(I'm .ts noob)

For any package when using editor (vscode) features such as jump to source, it currently opens published, generated *t.ts file within the package in node_modules because the *.ts file does not seem to be published? This means to see the inner workings I must have downloaded the source repo of any library and manually track down the source code. If I'm wrong here I'd appreciate any advice on how to get this to work and apologize for the noise.

microsoft/TypeScript#12358 (comment)

[vectors] vector ops codegen

Current @thi.ng/vectors package size too large and too repetitive. Many vector ops for vec2/3/4 can/should be replaced with generated versions for DRY purposes & less scope for copy&paste errors.

Preliminary benchmarks don't seem to show a noticeable negative impact.

• add codegen templates & helper fns
• add more interfaces for various types of vector op
• extend codegen scope to matrix ops