roman/haskell-capataz

OTP-like supervision trees in Haskell

Capataz

Our greatest glory is not in never failing, but in rising every time we fail.– Confucius

Table Of Contents

• Installation
• Documentation
• Development

Installation

Make sure you include the following entry on your cabal file's dependecies section.

library:
  build-depends: capataz

Or on your package.yaml

dependencies:
- capataz

Documentation

The documentation website can be found here. Some relevant links:

• Purpose
• FAQ

Development

Follow the developer guidelines

In future releases

• Replace Protolude in favor of RIO
• Documentation of performance analysis
• Ensure unit tests always finish on all concurrent scenarios (dejafu experiment)

haskell-capataz's Issues

Stabalize test-suite execution

The current version of Capataz uses concurrent tests to assert it's behaving as expected, and this has the unfortunate side-effect of the test-suite not being stable.

Ideas:

• Use deja-fu to mock schedulers -- this seems tricky, specially because we are planning to move to RIO and not certain it will work well with deja-fu
• Have a different (stable) test-suite and move existing one to a different project

Add how-to guides on gitbook

A few how-to pages that would be nice:

• How to create a dynamic worker
• How to create a static worker
• How to create a dynamic supervisor
• How to create a static supervisor
• How to create a stateful worker
• How to terminate a supervisor
• How to terminate a worker
• How to create a stateful supervisor

Update async upper bounds

Stackage required updating upper-bounds for the async package on issue commercialhaskell/stackage#3268

Acceptance Criteria

• A new release v0.1.0.1

Support latest nightly

The current version doesn't work on the latest nightly of stackage

Replace Protolude in favor of RIO

Allow Worker sub-routines to receive their WorkerId

As a software developer, I want to re-use a WorkerId so that I may use that identifier in other APIs and then be able to unify the telemetry of many APIs using the same id.

Implementation Details:

• Change WorkerAction to have a WorkerId parameter

• Supply workerId argument to invocation of workerAction

• Decorate workerAction provided on buildWorkerOptions using a const call. (e.g. workerAction = const workerAction0, assuming the input parameter is renamed to workerAction0)

• Add a new function buildWorkerOptions1 here similar to the buildWorkerOptions but with a workerAction input of type WorkerId -> IO () and export it to the public API.

Create capataz-statsd project

Description

As an operations guy, I want to have out-of-the-box metrics for the capataz library, so that I'm able to know quickly what is going on in my production systems

Supervisor fails when it shouldn't

Consider the following code that uses the capataz library:

• https://gist.github.com/ChShersh/7ce645338c961869757262652b39492d

This code creates supervisor and two threads:

• Thread 1: does nothing, waits infinitely
• Thread 2: waits random time from 1 to 3 seconds and throws an exception

Observed behaviour

Main thread exits

Expected behaviour

Supervisor tries to restart failing worker infinitely.

Here are the logs:

• https://lpaste.net/8240404574289002496

Improve documentation structure

From this reddit comment:

https://www.reddit.com/r/haskell/comments/7ulnn4/ann_capataz_01_now_with_multilevel_supervision/dtmawzt/

A few things are evident:

• There needs to be a quick links section in the README

Move all the documentation found in the README to the gitbook site and just provide links that tell differences, the why, what and how.

Upgrade capataz-repo-watcher to use latest version of rio

The capataz-repo-watcher example project uses rio as a dependency. This dependency removed not long ago it's RIO.Logger module, making the example hard to compile.

Upgrade the example to use the latest version of rio, and fix the version on the dependencies.

Broken link

link for "developer guidelines" from README (github root page) gives 404

Make capataz compatible with ghc-8.4

Need to ease the boundaries of dep versions.

See: commercialhaskell/stackage#3389

Enhance ProcessId from a type alias to a newtype

Summary

As the API owner, I want to make sure that the only way a terminateProcess invocation can be made, is that you call a fork function from the library.

Context

Currently, terminateProcess receives any UUID type, we want to make sure that invalid calls to terminateProcess cannot be made

Implementation Notes

Follow the compilation errors once we transform a type to a newtype, behavior of the library should stay the same as this returned value was being treated as an opaque one

Support GHC-8.6.3

I tried to make it myself but some decisions should be made in here.

The thing is that the latest version of stm –– stm-2.5.0.0 has some breaking change in it, so the interface of the capataz library is conflicting with it. The Changelog for 2.5.0.0 says

Switched newTBQueue and newTBQueueIO to accept Natural as size (gh-17)

And here is the error for capataz:

Haskell-capataz/src/Control/Concurrent/Capataz/Util.hs:75:30: error:
        • Couldn't match expected type ‘Natural’ with actual type ‘Int’
        • In the first argument of ‘newTBQueueIO’, namely ‘bufferSize’
          In a stmt of a 'do' block: inputQueue <- newTBQueueIO bufferSize
          In the expression:
            do inputQueue <- newTBQueueIO bufferSize
               let myLogFunc = mkLogFunc $ ...
                   loggerSpec
                     = workerSpec
                         procName
                         (runLoggerThread logOptions inputQueue)
                         (modOptions . set workerRestartStrategyL Permanent)
               return (loggerSpec, myLogFunc)
       |
    75 |   inputQueue <- newTBQueueIO bufferSize
       |                              ^^^^^^^^^^

The old version of stm -- 2.4.5.1 is not compatible with GHC-8.6.3.
So I'm not sure how you'd like this to be solved but either way, this is going to be breaking for capataz, so the new release should be made in order to support building with GHC-8.6.3.

If you'd like I can continue working on this if we agree on the strategy. The question is: is it better to change the type of the buildLogWorkerSpec to receive Natural or only change the internal of the function? Or probably you have some better ways to solve this?

Add performance analysis documentation