composewell/streamly-examples

Examples for Streamly

Streamly Examples

Practical examples to demonstrate the features and performance of Streamly. Includes examples about file IO, text processing, networking, concurrent programming, reactive programming and more.

These examples also serve as a beginner's guide to express practical programs using the dataflow programming (streaming) model. Please visit the Streamly homepage for more details and comprehensive documentation.

Running The Examples

When running the unstable version (downloaded from the git repository) you must use --project-file cabal.project.user option otherwise the build might fail. For example:

$ cabal run --project-file cabal.project.user AcidRain

Executable names are the same as the filenames. To run an example:

$ cabal run AcidRain

To run SDL2 based examples, make sure that you have the OS package for the sdl2 library installed on your system and the headers are visible to cabal:

$ cabal run --flag sdl2 CirclingSquare

To run interop examples, use the interop build flag:

$ cabal run --flag interop Interop.Pipes

List of examples

General

• Intro: Simple, introductory examples - loops, text processing, networking, concurrency.
• MergeSort: Sort a stream concurrently using merge sort.
• Rate: Run an action at a given rate.

FileSystem

• CoreUtils: Implement simplified coreutils like cat, cp, tee, grep using Streamly.FileSystem.File API.
• CoreUtilsHandle: Implement simplified coreutils using Streamly.FileSystem.Handle API.
• Split: Implement file splitting utility split.
• FileSystemEvent: File watching/fsnotify API example.
• ListDir: List a directory tree recursively and concurrently.

Text Processing

• CamelCase: Convert a file to camel case.
• WordCount: Simple word counting (wc) program.
• WordCount.c: C equivalent for perf comparison.
• WordCountModular: Modular version.
• WordCountParallel: Concurrent version.
• WordCountParallelUTF8: Concurrent version with full UTF8 handling.
• WordFrequency: Count word frequency in a file and print top 25 words.
• CSVParser: Process a CSV file
• DateTimeParser: Parse a Date/Time string.
• LogParser: Parse a quoted string with escaping.

Networking

• EchoServer: A concurrent TCP server that echoes everything that it receives.
• MergeServer: Merges lines received from several client connections and writes them to a file.
• FileSender: Send many files concurrently to a server over multiple connections. Can be used to test MergeServer.
• CmdServer: Receive a stream of commands from many clients and respond to them using command handlers.
• CmdClient: Run multiple concurrent clients sending streams of commands to a server and receiving responses. Can be used to test CmdServer.
• WordServer: A word look up (dictionary) server, instead of performing a real DB query the server just adds a time delay to simulate the IO.

FRP/Games/Animation

• AcidRain: A console game with deteriorating health that can be modified by typing "harm" and "potion" commands.
• CirclingSquare: Use SDL2 to display a square that moves in a circle, and follows the mouse pointer.

Interoperation

• Interop.Streaming: Converting streamly stream type to and from streaming stream type.
• Interop.Pipes: Converting streamly stream type to and from pipes stream type.
• Interop.Conduit: Converting streamly stream type to and from conduit stream type.
• Interop.Vector: Converting streamly stream type to and from vector stream type.

Licensing

Available under Apache-2.0 license.

streamly-examples's Issues

Update the readme links to be absolute github links

CIs need to be fixed

Circle-ci and appveyor CIs are not working.

Align major version number with streamly

Release a 0.8.0 along with the same version of streamly.

Most examples are broken in 9.4.4.

Some of these won't compile, others seem to be based on broken libraries, etc...

I like streamly very much, but it's much harder to pick up if the examples are guaranteed to be broken.

Enable sdl2 flag in CI on windows and macOS too

We will need to install libsdl2 .

tag the repo with a streamly version tag with which it is compatible

See composewell/streamly#782 .

Add a 0.7.2 tag on the commit which was compatible with 0.7.2.
We can add a 0.8.0 tag when we release streamly-0.8.0.

An example to implement a traffic router idiomatically

To track finishing the example on the traffic-router branch.

Add CIs

• for major GHC versions
• GHCJS
• Linux/macOS/Windows

We can just copy the CIs from streamly

Fix the windows CI build

Changes to cabal file

1. Network package now works with ghcjs, so we can remove the conditional build where network is a dependency
2. We can factor out dependencies of all examples in a common stanza
3. Add version bounds to all dependencies

Benchmark the examples

• The examples can act as real world benchmarks
• Possibly can be compared with alternate language implementations
• We can run them with restricted heap and stack to find any memory issues

Change the license to Apache-2.0

Use streamly mutable arrays instead of vector in WorCountParallelUTF8 example

Remove the unnecessary `-main-is` ghc options in the cabal file

cabal sdist complains:

Warnings:
- 'ghc-options: -main-is' is not portable.

Also, we do not really need this for at least those executables which are not required to have a module name other than Main.

AccessDenied on example links

e.g. https://streamly.composewell.com/streamly-examples-0.1.3/examples/Intro.hs

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>0G3TJK6VK333FZ34</RequestId>
<HostId>dn7zbWYXFtycN7ISPb+C9xpuWAeNRK9GOYLhLPN66dAZPyDawhJR8QWFzB3hu5dypL9cNesaQoY=</HostId>
</Error>

Make the examples as modules in a library or use .lhs?

By making it a library haddock can work on it.

Alternatively, we can write them as .lhs.

Once we do that we should be able to integrate the examples directly on the streamly website.

8.10.4 macOS CI fails to install cabal/ghc

https://github.com/composewell/streamly-examples/actions/runs/3604666675/jobs/6074261657

Add a CI with fusion-plugin

Now that fusion-plugin is essential for performance with parsers and folds, we should be testing everything with it by default.

Limited example of interoperability with the Streaming library.

The Interop.Streaming.fromStreaming example assumes "r ~ ()", but I rarely find that to be the case in code that uses Streaming. Is this an area where Streaming's model is more "powerful" than Streamly's due to different trade-offs, or is there a way to represent "Stream (Of a) m r" in Streamly?

Check which Internal imports can be converted to public APIs

We should try to make the Internal streamly APIs used in examples public wherever possible.