well-typed/generics-sop

Generic Programming using True Sums of Products

sop-core and generics-sop

generics-sop is a library to support the definition of generic functions. Datatypes are viewed in a uniform, structured way: the choice between constructors is represented using an n-ary sum, and the arguments of each constructor are represented using an n-ary product.

Since version 0.4.0.0, generics-sop is now based on sop-core. The core package contains all the functionality of n-ary sums and products, whereas generics-sop provides the datatype-generic programming support on top.

This is the development repository for the packages. For releases, look on Hackage.

The module Generics.SOP is the main module of this library and contains more detailed documentation.

Examples of using generics-sop are provided by the following packages:

• basic-sop basic examples,
• pretty-sop generic pretty printing,
• lens-sop generically computed lenses,
• json-sop generic JSON conversions.

A detailed description of the ideas behind this library is provided by the paper:

• Edsko de Vries and Andres Löh. True Sums of Products. Workshop on Generic Programming (WGP) 2014.

generics-sop's Issues

Create a benchmark suite

I currently have two branches I'm experimenting with that are in some sense performance-oriented: static-case and compact-representation.

Before merging any of these, I'd like to measure relative performance, so it'd be good to have a number of performance tests that'd allow to test how various generics-sop functions perform. It'd also be good to measure the overhead of generics-sop itself, by comparing it to generic-deriving / built-in GHC generics, and to hand-written code.

I may end up doing this ultimately, but I don't have a lot of time. So if anyone wants to help with this, it'd be much appreciated.

Compile failure w/ base-4.6

I went ahead and revised in a tighter lower bound on base for the affected release (https://hackage.haskell.org/package/generics-sop-0.3.0.0/revisions/)

Configuring component lib from generics-sop-0.3.0.0...
Preprocessing library generics-sop-0.3.0.0...
[ 1 of 14] Compiling Generics.SOP.Sing ( src/Generics/SOP/Sing.hs, /tmp/matrix-worker/1493460027/dist-newstyle/build/x86_64-linux/ghc-7.6.3/generics-sop-0.3.0.0/build/Generics/SOP/Sing.o )
[ 2 of 14] Compiling Generics.SOP.Constraint ( src/Generics/SOP/Constraint.hs, /tmp/matrix-worker/1493460027/dist-newstyle/build/x86_64-linux/ghc-7.6.3/generics-sop-0.3.0.0/build/Generics/SOP/Constraint.o )
[ 3 of 14] Compiling Generics.SOP.BasicFunctors ( src/Generics/SOP/BasicFunctors.hs, /tmp/matrix-worker/1493460027/dist-newstyle/build/x86_64-linux/ghc-7.6.3/generics-sop-0.3.0.0/build/Generics/SOP/BasicFunctors.o )
[ 4 of 14] Compiling Generics.SOP.Classes ( src/Generics/SOP/Classes.hs, /tmp/matrix-worker/1493460027/dist-newstyle/build/x86_64-linux/ghc-7.6.3/generics-sop-0.3.0.0/build/Generics/SOP/Classes.o )
[ 5 of 14] Compiling Generics.SOP.NP  ( src/Generics/SOP/NP.hs, /tmp/matrix-worker/1493460027/dist-newstyle/build/x86_64-linux/ghc-7.6.3/generics-sop-0.3.0.0/build/Generics/SOP/NP.o )
[ 6 of 14] Compiling Generics.SOP.Metadata ( src/Generics/SOP/Metadata.hs, /tmp/matrix-worker/1493460027/dist-newstyle/build/x86_64-linux/ghc-7.6.3/generics-sop-0.3.0.0/build/Generics/SOP/Metadata.o )
[ 7 of 14] Compiling Generics.SOP.Type.Metadata ( src/Generics/SOP/Type/Metadata.hs, /tmp/matrix-worker/1493460027/dist-newstyle/build/x86_64-linux/ghc-7.6.3/generics-sop-0.3.0.0/build/Generics/SOP/Type/Metadata.o )

src/Generics/SOP/Type/Metadata.hs:144:7:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:144:22:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:148:8: Not in scope: `symbolVal'

src/Generics/SOP/Type/Metadata.hs:149:8: Not in scope: `symbolVal'

src/Generics/SOP/Type/Metadata.hs:153:7:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:153:22:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:157:8: Not in scope: `symbolVal'

src/Generics/SOP/Type/Metadata.hs:158:8: Not in scope: `symbolVal'

src/Generics/SOP/Type/Metadata.hs:196:11:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:197:44: Not in scope: `symbolVal'

src/Generics/SOP/Type/Metadata.hs:200:7:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:200:45:
    Not in scope: type constructor or class `KnownNat'

src/Generics/SOP/Type/Metadata.hs:204:8: Not in scope: `symbolVal'

src/Generics/SOP/Type/Metadata.hs:206:21: Not in scope: `natVal'

src/Generics/SOP/Type/Metadata.hs:208:11:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:210:15: Not in scope: `symbolVal'

src/Generics/SOP/Type/Metadata.hs:246:10:
    Not in scope: type constructor or class `KnownSymbol'

src/Generics/SOP/Type/Metadata.hs:247:36: Not in scope: `symbolVal'

Why HTraversable_/HSequence instead of HFoldable/HTraversable ?

This hinders discoverability a bit. It also seems that while the docs for the methods themselves refer the ones they generalize, there's no such association being made between the classes. Is that related to the less standard names?

`All2 c xss` should imply `All SListI xss`

This came up during ekmett's haskell stream: There's no nice way to do a double fold on an All2 c xss => xss because the type system doesn't know the inner lists are lists, i.e All SListI xss. The workaround was to write direct recursion on nil/cons cases.

Add a way to determine the index of an n-ary sum

I regularly find myself writing a function like this:

index_NS :: forall f xs . NS f xs -> Int
index_NS = go 0
  where
    go :: forall ys . Int -> NS f ys -> Int
    go !acc (Z _) = acc
    go !acc (S x) = go (acc + 1) x

I think it would be useful to add this function to the library.

An additional argument is that if we ever switch to compact internal representations, then index_NS is trivial to define. The compact representation of an NS is defined as follows:

data NS (f :: k -> *) (xs :: [k]) = NS !Int (f Any)

So index_NS becomes just extracting the first component of the pair. On the other hand, a manual reimplementation of index_NS in user code like the one shown above becomes unnecessarily inefficient.

A different option might be to change collapse_NS and collapse_SOP instead to return the index as well as the stored component. The argument being that while collapse_NP and collapse_POP are lossless operations, the current definitions of collapse_NS and collapse_SOP are not. The code would look as follows:

collapse_NS  :: forall a xs  .               NS  (K a) xs  ->  (Int, a)
collapse_NS = go 0
  where
    go :: forall ys . Int -> NS (K a) ys -> (Int, a)
    go !acc (Z (K x)) = (acc, x)
    go !acc (S xs)    = go (acc + 1) xs

collapse_SOP :: forall a xss . SListI xss => SOP (K a) xss ->  (Int, [a])
collapse_SOP = collapse_NS . hliftA (K . collapse_NP) . unSOP

type instance CollapseTo NS  a = (Int, a)
type instance CollapseTo SOP a = (Int, [a])

instance HCollapse NS  where hcollapse = collapse_NS
instance HCollapse SOP where hcollapse = collapse_SOP

Then index_NS can (at least theoretically) be defined in terms of collapse_NS:

index_NS :: SListI xs => NS f xs -> Int
index_NS = fst . collapse_NS . map_NS (const (K ()))

An argument in favour of the adapted hcollapse definitions is that with the current representation, if you need to call hcollapse anyway, then calling index_NS will lead to an additional and unnecessary traversal of the constructors of NS. On the other hand, this would be a significant breaking change.

Feedback welcome.

/cc @edsko @phadej

Documentation bug: hcollapse for NS

From the docs:

hcollapse, collapse_NS  :: NS  (K a) xs  ->   a

But:

instance HCollapse NS  where hcollapse = I . collapse_NS

so in reality the types of hcollapse and collapse_NS differ by I.

Not submitting a pull request since this creates an ugly non-uniformity in the docs and I don't know how you prefer to resolve it. Perhaps change the type of collapse_NS to return an I? I don't know if anyone actually use the specialized versions.

Haddock generation error on GHC 7.6

Discovered via Stackage:

Resolving dependencies...
Configuring generics-sop-0.1.0.2...
Running Haddock for generics-sop-0.1.0.2...
Running hscolour for generics-sop-0.1.0.2...
Preprocessing library generics-sop-0.1.0.2...
Preprocessing library generics-sop-0.1.0.2...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: tagged-0.7.3
Haddock coverage:
Warning: Generics.SOP.Sing: Instances of type and data families are not yet supported. Instances of the following families will be filtered out:
  Sing
 100% (  8 /  8) in 'Generics.SOP.Sing'
Warning: Generics.SOP.Constraint: Instances of type and data families are not yet supported. Instances of the following families will be filtered out:
  All, All2, Map
  86% (  6 /  7) in 'Generics.SOP.Constraint'
 100% (  7 /  7) in 'Generics.SOP.BasicFunctors'
 100% ( 20 / 20) in 'Generics.SOP.Classes'
Warning: Generics.SOP.NP: Instances of type and data families are not yet supported. Instances of the following families will be filtered out:
  AllMap, Prod, CollapseTo
 100% ( 40 / 40) in 'Generics.SOP.NP'
 100% ( 11 / 11) in 'Generics.SOP.Metadata'
Warning: Generics.SOP.NS: Instances of type and data families are not yet supported. Instances of the following families will be filtered out:
  Prod, CollapseTo
 100% ( 33 / 33) in 'Generics.SOP.NS'
Warning: Generics.SOP.GGP: Instances of type and data families are not yet supported. Instances of the following families will be filtered out:
  ToSingleCode, ToProductCode, ToSumCode
 100% (  8 /  8) in 'Generics.SOP.GGP'
 100% (  4 /  4) in 'Generics.SOP.Universe'
 100% (  3 /  3) in 'Generics.SOP.TH'
Warning: Generics.SOP.Instances: We do not support associated types in instances yet. These instances are affected:
Generics.SOP.Universe.Generic (Generics.SOP.BasicFunctors.I a_a8l3), Generics.SOP.Universe.Generic (Generics.SOP.BasicFunctors.K a_a8l5 b_a8l6), Generics.SOP.Universe.Generic (Generics.SOP.BasicFunctors.:.: f_a8l0 g_a8l1 p_a8l2), Generics.SOP.Universe.Generic GHC.Types.Bool, Generics.SOP.Universe.Generic GHC.Types.Ordering, Generics.SOP.Universe.Generic (Data.Maybe.Maybe a_a8or), Generics.SOP.Universe.Generic (Data.Either.Either a_a9Du b_a9Dv), Generics.SOP.Universe.Generic (), Generics.SOP.Universe.Generic ((,) a_12 b_13), Generics.SOP.Universe.Generic ((,,) a_12 b_13 c_14), Generics.SOP.Universe.Generic ((,,,) a_12 b_13 c_14 d_15), Generics.SOP.Universe.Generic ((,,,,) a_12 b_13 c_14 d_15 e_16), Generics.SOP.Universe.Generic ((,,,,,) a_12 b_13 c_14 d_15 e_16 f_17), Generics.SOP.Universe.Generic ((,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18), Generics.SOP.Universe.Generic ((,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19), Generics.SOP.Universe.Generic ((,,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19 i_1a), Generics.SOP.Universe.Generic ((,,,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19 i_1a j_1b), Generics.SOP.Universe.Generic ((,,,,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19 i_1a j_1b k_1c), Generics.SOP.Universe.Generic ((,,,,,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19 i_1a j_1b k_1c l_1d), Generics.SOP.Universe.Generic ((,,,,,,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19 i_1a j_1b k_1c l_1d m_1e), Generics.SOP.Universe.Generic ((,,,,,,,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19 i_1a j_1b k_1c l_1d m_1e n_1f), Generics.SOP.Universe.Generic ((,,,,,,,,,,,,,,) a_12 b_13 c_14 d_15 e_16 f_17 g_18 h_19 i_1a j_1b k_1c l_1d m_1e n_1f o_1g), Generics.SOP.Universe.Generic ([] a_12), Generics.SOP.Universe.Generic GHC.IO.Exception.IOException, Generics.SOP.Universe.Generic GHC.Exception.ArithException, Generics.SOP.Universe.Generic GHC.IO.Exception.ArrayException, Generics.SOP.Universe.Generic GHC.IO.Exception.AssertionFailed, Generics.SOP.Universe.Generic GHC.IO.Exception.AsyncException, Generics.SOP.Universe.Generic Control.Exception.Base.NonTermination, Generics.SOP.Universe.Generic Control.Exception.Base.NestedAtomically, Generics.SOP.Universe.Generic GHC.IO.Exception.BlockedIndefinitelyOnMVar, Generics.SOP.Universe.Generic GHC.IO.Exception.BlockedIndefinitelyOnSTM, Generics.SOP.Universe.Generic GHC.IO.Exception.Deadlock, Generics.SOP.Universe.Generic Control.Exception.Base.NoMethodError, Generics.SOP.Universe.Generic Control.Exception.Base.PatternMatchFail, Generics.SOP.Universe.Generic Control.Exception.Base.RecConError, Generics.SOP.Universe.Generic Control.Exception.Base.RecSelError, Generics.SOP.Universe.Generic Control.Exception.Base.RecUpdError, Generics.SOP.Universe.Generic GHC.Exception.ErrorCall, Generics.SOP.Universe.Generic GHC.IO.MaskingState, Generics.SOP.Universe.Generic Data.Char.GeneralCategory, Generics.SOP.Universe.Generic (Data.Complex.Complex a_arM4), Generics.SOP.Universe.Generic Data.Data.DataRep, Generics.SOP.Universe.Generic Data.Data.Fixity, Generics.SOP.Universe.Generic Data.Data.ConstrRep, Generics.SOP.Universe.Generic (Data.Fixed.Fixed a_arS4), Generics.SOP.Universe.Generic (Data.Monoid.Dual a_a8nz), Generics.SOP.Universe.Generic (Data.Monoid.Endo a_a8JA), Generics.SOP.Universe.Generic Data.Monoid.All, Generics.SOP.Universe.Generic Data.Monoid.Any, Generics.SOP.Universe.Generic (Data.Monoid.Sum a_a8nf), Generics.SOP.Universe.Generic (Data.Monoid.Product a_a8nk), Generics.SOP.Universe.Generic (Data.Monoid.First a_a8nu), Generics.SOP.Universe.Generic (Data.Monoid.Last a_a8np), Generics.SOP.Universe.Generic (Data.Ord.Down a_arZi), Generics.SOP.Universe.Generic (Data.Proxy.Proxy s_a6hV), Generics.SOP.Universe.Generic Data.Version.Version, Generics.SOP.Universe.Generic Foreign.C.Error.Errno, Generics.SOP.Universe.Generic Foreign.C.Types.CChar, Generics.SOP.Universe.Generic Foreign.C.Types.CSChar, Generics.SOP.Universe.Generic Foreign.C.Types.CUChar, Generics.SOP.Universe.Generic Foreign.C.Types.CShort, Generics.SOP.Universe.Generic Foreign.C.Types.CUShort, Generics.SOP.Universe.Generic Foreign.C.Types.CInt, Generics.SOP.Universe.Generic Foreign.C.Types.CUInt, Generics.SOP.Universe.Generic Foreign.C.Types.CLong, Generics.SOP.Universe.Generic Foreign.C.Types.CULong, Generics.SOP.Universe.Generic Foreign.C.Types.CPtrdiff, Generics.SOP.Universe.Generic Foreign.C.Types.CSize, Generics.SOP.Universe.Generic Foreign.C.Types.CWchar, Generics.SOP.Universe.Generic Foreign.C.Types.CSigAtomic, Generics.SOP.Universe.Generic Foreign.C.Types.CLLong, Generics.SOP.Universe.Generic Foreign.C.Types.CULLong, Generics.SOP.Universe.Generic Foreign.C.Types.CIntPtr, Generics.SOP.Universe.Generic Foreign.C.Types.CUIntPtr, Generics.SOP.Universe.Generic Foreign.C.Types.CIntMax, Generics.SOP.Universe.Generic Foreign.C.Types.CUIntMax, Generics.SOP.Universe.Generic Foreign.C.Types.CClock, Generics.SOP.Universe.Generic Foreign.C.Types.CTime, Generics.SOP.Universe.Generic Foreign.C.Types.CUSeconds, Generics.SOP.Universe.Generic Foreign.C.Types.CSUSeconds, Generics.SOP.Universe.Generic Foreign.C.Types.CFloat, Generics.SOP.Universe.Generic Foreign.C.Types.CDouble, Generics.SOP.Universe.Generic (System.Console.GetOpt.ArgOrder a_asnp), Generics.SOP.Universe.Generic (System.Console.GetOpt.OptDescr a_asoT), Generics.SOP.Universe.Generic (System.Console.GetOpt.ArgDescr a_asoX), Generics.SOP.Universe.Generic GHC.IO.Exception.ExitCode, Generics.SOP.Universe.Generic GHC.IO.IOMode.IOMode, Generics.SOP.Universe.Generic GHC.IO.Handle.Types.BufferMode, Generics.SOP.Universe.Generic GHC.IO.Device.SeekMode, Generics.SOP.Universe.Generic GHC.IO.Handle.Types.Newline, Generics.SOP.Universe.Generic GHC.IO.Handle.Types.NewlineMode, Generics.SOP.Universe.Generic Text.Read.Lex.Lexeme
 100% (  1 /  1) in 'Generics.SOP.Instances'
  99% ( 82 / 83) in 'Generics.SOP'
Warning: Generics.SOP.GGP: could not find link destinations for:
    Generics.SOP.GGP.ToSumCode Generics.SOP.GGP.GSumFrom Generics.SOP.GGP.GSumTo Generics.SOP.GGP.GDatatypeInfo' Generics.SOP.GGP.isNewtype
haddock: internal error: synifyKind

Add special code aliases

type IsRecordType a xs = (Generic a, Code a ~ '[ xs ])
type IsEnumType a = (Generic a, All ((~) '[]) (Code a))
type IsWrappedType a x = (Generic a, Code a ~ '[ '[ x ] ])

I have IsRecordType (inlined) and IsWrappedType (with different name) used in our code. And now as I think about it, I'll should use IsEnumType too as I have quite a few enumerations with very very similar boilerplate just copypasted around.

So 👍 to all of these.

---

About naming, record implies it has field names; which isn't always true. Would IsProductType be ~correct and not clash with records-sop?

Relax TH upper bound of 0.2.5.0 on Hackage (as a Hackage revision)

Could you bump TH dependency of generics-sop-0.2.5.0 on Hackage from <2.13 to <2.14? I tested that it compiles fine. Thank you! Alternatively, I will gladly do that myself, given Hackage credentials.

The story: I'm using postgresql-simple-sop (maintainer currently not responsive), which depends on generics-sop-0.2.5.0 and to compile it under GHC 8.4.4 I'd need the TH dependency of generics-sop-0.2.5.0 to be relaxed. This is a stop-gap measure, but will do fine while I keep nagging the postgresql-simple-sop maintainer.

Define `SListI` in terms of `All`

This is a reminder for myself.

I think it would be nicer to refactor the definitions such that

type SListI = All Top

Currently, All Top implies SListI (via superclass constraint), but not vice versa. I think it'd be nicer if both are equivalent. In many cases, this would make the c and non-c versions of functions coincide.

Consider for example the following two functions as they are currently:

cpure_NP :: All c xs => proxy c -> (forall x . c x => f x) -> NP f xs
pure_NP :: SListI xs => (forall x . f x) -> NP f xs

With the equivalence, we have

pure_NP f = cpure_NP (Proxy :: Proxy Top) f

Improve documentation / point to available tutorials

I've given quite a few talks and written a few tutorials by now (others might have done the same), yet the package itself only points to the original paper.

We might want to create more documentation / an official tutorial. Or at least have a place where we collect and point to available materials.

I'm not even sure if I'm aware of all, but certainly there are at least:

• my HaskellX Bytes 2017 talk: https://skillsmatter.com/skillscasts/10181-fun-with-sum-and-product-types, https://github.com/kosmikus/sop-haskellx-bytes-2017
• my ZuriHac 2016 talk: https://www.youtube.com/watch?v=sQxH349HOik
• my SSGEP 2015 lecture notes: https://github.com/kosmikus/SSGEP
• the WGP 2014 talk and paper: https://youtu.be/jzgfM6NFE3Y, https://www.andres-loeh.de/TrueSumsOfProducts/

Possibly also some SO answers such as these (to a large extent these are by @danidiaz):

• https://stackoverflow.com/questions/44193630/how-to-customise-show-read-instances-on-the-lines-of-fromjson-tojson/44195842#44195842
• https://stackoverflow.com/questions/44071503/how-to-map-data-elements-to-data-types/44073088#44073088
• https://stackoverflow.com/questions/43855273/build-more-advanced-generic-parser/43857293#43857293
• https://stackoverflow.com/questions/42692384/how-to-map-over-a-long-tuple-of-instances-of-the-same-class/42705416#42705416
• https://stackoverflow.com/questions/42674354/how-do-i-map-over-parameters/42693764#42693764
• https://stackoverflow.com/questions/42583838/generically-build-parsers-from-custom-data-types/42585429#42585429
• https://stackoverflow.com/questions/41793202/is-it-possible-to-re-implement-enum-deriving-using-ghc-generics/41822243#41822243
• https://stackoverflow.com/questions/41522984/haskell-is-there-a-way-of-mapping-over-an-algebraic-data-type/41524511#41524511
• https://stackoverflow.com/questions/40593441/how-can-i-produce-a-tag-type-for-any-datatype-for-use-with-dsum-without-templat/40621740#40621740
• https://stackoverflow.com/questions/39991581/how-can-i-implement-generalized-zipn-and-unzipn-in-haskell/39997122#39997122
• https://stackoverflow.com/questions/39020787/is-there-a-way-to-apply-maybe-constructor-to-each-field-of-record-with-generics/39027753#39027753
• https://stackoverflow.com/questions/38271220/constructing-haskell-data-types-with-many-fields/38272190#38272190
• https://stackoverflow.com/questions/38248692/whats-a-better-way-of-managing-large-haskell-records/38251692#38251692
• https://stackoverflow.com/questions/36667541/indexing-into-containers-the-mathematical-underpinnings/36672495#36672495
• https://stackoverflow.com/questions/36369414/how-to-generically-extract-all-values-in-haskell-records/36371431#36371431
• https://stackoverflow.com/questions/36254349/retrieving-record-function-in-generic-sop/36255015#36255015
• https://stackoverflow.com/questions/35709737/is-it-possible-to-derive-recursion-principles-generically/35804233#35804233

Confusing error with `QuantifiedConstraints`

First, I apologise in advance, if this is not a valid place for such issues, so please feel free to close this, if that's the case.

The problem is that using QuantifiedConstraints to enforce Code constraint results in a confusing type error, that doesn't appears to make sense.

The goal is to:

1. enforce a particular structure ('[xs])
2. on a type derived from a typeclass parameter (using Code)
3. without introducing a new typeclass parameter (abstracting over detail)

This seems to require QuantifiedConstraints, because:
I. I could find no other way to encode the Code a ~ '[xs] constraint (f.e. Code a ~ '[Head (Code a)] makes GHC blow reduction stack.
II. Code a ~ '[xs] requires a type variable, which I'm hesitant to admit into the typeclass, because of requirement 3

..however, using QuantifiedConstraints to introduce a fresh variable fails in a confusing way:

{-# LANGUAGE AllowAmbiguousTypes #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE QuantifiedConstraints #-}
{-# LANGUAGE TypeInType #-}
{-# LANGUAGE UndecidableSuperClasses #-}
{-# LANGUAGE UnicodeSyntax #-}
{-# OPTIONS_GHC -Wextra -Wno-unused-imports -Wno-unticked-promoted-constructors -Wno-type-defaults -Wno-missing-signatures #-}

module Main where

import           Control.Monad.IO.Class
import qualified GHC.Generics as GHC
import           Generics.SOP
import qualified Generics.SOP as SOP


class    ( GHC.Generic a, SOP.Generic a, HasDatatypeInfo a
         , forall xs. Code a ~ '[xs]) => Foo m a where

instance ( GHC.Generic a, SOP.Generic a, HasDatatypeInfo a
         , forall xs. Code a ~ '[xs]
         , Monad m)                  => Foo m a where

..leads to:

error:
    • Could not deduce: Code a ~ '[xs]
        arising from the superclasses of an instance declaration
      from the context: (GHC.Generic a, HasDatatypeInfo a,
                         forall (xs :: [*]). Code a ~ '[xs], Monad m)
        bound by the instance declaration
        at /run/user/1000/danteazsW1Q.hs:(22,10)-(24,47)
    • In the instance declaration for ‘Foo m a’

..which appears contradictory, because:

1. Code a ~ '[xs] is explicitly stated in the instance
2. There is no indication of type variables not unifying -- i.e. no mismatch between xs and xs0 or somesuch is reported (which is typical when unification fails due to existentials not aligning).

Repository reorganisation

Now that we've split the packages, we should:

• rename the repository from generics-sop to sop,
• possibly integrate some of the other sop-related packages.

Release 0.2.5.0

@phadej Any other things that need to be done before 0.2.5.0? Or can I release it?

Windows 10: can't build 0.4.0.1 using GHC 8.6.3 (stack, lts-13.0)

I do not know where the problem lies, but I am unable to build generics-sop-0.4.0.1 using stack with resolver lts-13.0 (GHC 8.6.3) on Windows 10 64-bit. After the configure and build steps are reported by stack, there is no further progress (even though ghc.exe continues to use most of the CPU for long periods of time - on a laptop with an Intel Core i7-8550U CPU).

(GHC 8.6.3 seems to be part of the problem: changing the resolver to lts-12.25 - GHC 8.4.4 - sees generics-sop-0.3.2.0 build fine and quite quickly. Adding extra-deps generics-sop-0.4.0.1 and sop-core-0.4.0.0 also sees a quick successful build with GHC 8.4.4.)

For example, the following sequence of activities results in no progress beyond Progress 0\2, even if I wait a long time:

>stack new sopTest
>cd sopTest
[Edit `package.yaml` to add `- generics-sop` as a dependency]
>stack build
generics-sop-0.4.0.1: configure
generics-sop-0.4.0.1: build
Progress 0/2

stack build --dry-run yields:

No packages would be unregistered.

Would build:
generics-sop-0.4.0.1: database=snapshot, source=generics-sop-0.4.0.1@sha256:6474ad47a9dbcedc3c721b8d582d00ea52ec9fe6c5327562d0aeaa6f878ace91,5529 (from Hackage)
sopTest-0.1.0.0: database=local, source=C:\Users\mikep\Documents\Code\Haskell\sopTest\, after: generics-sop-0.4.0.1

No executables to be installed.

stack --verbose build yields for the last three lines of logging (before no further progress is made):

[debug] Finished writing C:\Users\mikep\AppData\Local\Temp\stack25088\generics-sop-0.4.0.1\.stack-work\dist\e626a42b\stack-config-cache
2018-12-29 16:53:34.344575: [info] generics-sop-0.4.0.1: build
2018-12-29 16:53:34.344575: [debug] Run process within C:\Users\mikep\AppData\Local\Temp\stack25088\generics-sop-0.4.0.1\: C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_2.4.0.1_ghc-8.6.3.exe --builddir=.stack-work\dist\e626a42b build --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
Progress 0/2

For completeness, stack --version is Version 1.10.0, Git revision 8e3490566b8e2846d99219156bee47047bd44564 (6723 commits) x86_64 hpack-0.31.1.

Type level constructor/record information

The Code type family gives the list of argument types to each constructor of a type, but there's no obvious way to get (at the type-level) whether those constructors are records or not. It be wonderful if there were such a type family (IsRecord a :: '[Bool]), that maintained the same order as Code.

Strictness information

I was using this to provide the generics interface for some code I was writing on stream recently.

However, I found I needed to know whether or not my fields were strict, which is something I can extract from the field selectors in GHC.Generics, but which doesn't seem to be present in the generics-sop metadata.

I wound up having to pull out all my generics-sop code out as a result, but I figured I should note here why in the hopes you might be able to address it in future releases.

Zip NP and NS to produce NP

Here's an alternative generalization of <*>:

hap' :: forall f (xs :: [k]) . NS (f -.-> f) xs -> NP f xs -> NP f xs
hap' (Z (Fn f)) (h :* t) = f h :* t
hap' (S f) (h :* t) = h :* hap' f t
hap' _ _ = error "impossible"

Unless I'm missing something, it can't be implemented in terms of the existing combinators. If so, I suggest adding it under some name.

Should we add hapInjs'_POP

The NP variant is simply hap injections, but POP variant is:

hapInjs'_POP :: SListI xss => POP f xss -> NP (K (SOP f xss)) xss
hapInjs'_POP = hmap (kfirst SOP) . hap injections . unPOP

which is quite lenghty. Or, is there a shorter way to say that?

Hackage Haddocks give incorrect signatures

The Hackage Haddocks for generics-sop give the wrong signature for the constructors of NS, NP, and possibly others. For NS, it shows:

The kind signatures of these types are also incorrect.
The signatures are shown correctly in the documentation of sop-core:

Re-export entities from `Generics.SOP`

Looks like some identifiers such as WrappedCode are not re-exported from Generics.SOP, even though they should be.

Allow non-TH building?

Is TH used only for Generic and HasDatatypeInfo instances? If so, would it be possible to have a flag that allows generics-sop to compile without TH by skipping all the instance generation? I'm trying to use generics-sop to add some functionality to a library that needs to have the option of no TH (for deployment on mobile). The core use just relies on the NS and NP types and doesn't require any instances. Some other features would be unavailable without instances but that could be handled.

Functions => NP

I've created this piece of code with the help of lyxia: https://gist.github.com/reactormonk/c4dd738b4f23281b6ab35b53c8661c91 - maybe it's useful somewhere? Should I open a PR?

Does not compile on GHC 8.6.3 Windows x86_64

Neither 0.4.0.1 nor 0.3.2.0 compile on Windows in less than an hour (they were still running at 100% CPU when I killed them)

Use ghc-proofs package in testsuite

This is connection with the work on static-case and optimisation (see #40).

By now, I think it's feasible to get the static-case branch into a shape where it produces mostly the same code as handwritten code. Once we have that, we should be able to "prove" this using the ghc-proofs package and thereby add regression tests for these properties.

/cc @nomeata @andorp

Quadratic compile times with AllZip constraints

The cut down code below takes approximately 6s to compile on my machine (including Windows and cabal overhead) without -DBIGGER, and 18s with -DBIGGER. Adding another row of Ints takes that to 40s. If the AllZipF constraint is removed from dumpFields then it takes ~6s regardless of how many Ints there are.

{-# LANGUAGE CPP #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeFamilies #-}
module Code
  ( dumpRow
  ) where

import Generics.SOP
import Generics.SOP.Constraint

type CodeRecord =
      '[
         Int, Int, Int, Int, Int, Int, Int, Int, Int, Int
#ifdef BIGGER
       , Int, Int, Int, Int, Int, Int, Int, Int, Int, Int
       , Int, Int, Int, Int, Int, Int, Int, Int, Int, Int
       , Int, Int, Int, Int, Int, Int, Int, Int, Int, Int
#endif
      ]

dumpRow :: NS I CodeRecord -> [String]
dumpRow = dumpFields

dumpFields :: (AllZipF (~) as as) => NS I as -> [String]
dumpFields = const []

I tracked it down to the definition of AllZipF:

  AllZipF (c :: a -> b -> Constraint) (xs :: [a]) (ys :: [b])
    :: Constraint where
  AllZipF _c '[]      '[]        = ()
  AllZipF  c (x ': xs) (y ': ys) = (c x y, AllZip c xs ys)

class
  ( SListI xs, SListI ys
  , SameShapeAs xs ys, SameShapeAs ys xs
  , AllZipF c xs ys
  ) => AllZip (c :: a -> b -> Constraint) (xs :: [a]) (ys :: [b])
instance
  ( SListI xs, SListI ys
  , SameShapeAs xs ys, SameShapeAs ys xs
  , AllZipF c xs ys
  ) => AllZip c xs ys

I think the recursive dependency to AllZip means that the SameShape constraints have to be solved again and again.

I tried changing the recursive case to depend on AllZipF directly:

AllZipF c (x ': xs) (y ': ys) = (c x y, AllZipF c xs ys)

Both sop-core and generics-sop compile fine, and the generics-sop tests pass. The slowdown in the code above is gone. (My real application also subjectively seems a lot better, but I think there may be another performance problem to track down after this one.)

Intuitively it seems like the same fix might be needed in AllF, but that causes a compile failure in sop-core.

I'm not sure if there are any other possible gotchas with the code change, particularly as it'll break the symmetry with AllF. I'll send in a pull request anyway, but thought a separate issue would be the best way to document the problem it's fixing.

Add kfirst

kfirst :: forall a b (x :: k). (a -> b) -> K a x -> K b x

Bifunctor's first would restrict kind of x, it would be handy to have polykinded kfirst.

0.4 broken on windows (GHC-8.2.2)

[13 of 14] Compiling Generics.SOP.Instances ( src\Generics\SOP\Instances.hs, dist\build\Generics\SOP\Instances.o )
src\Generics\SOP\Instances.hs:51:1: error:
    Could not find module `GHC.Event'
    Use -v to see a list of the files searched for.
   |
51 | import GHC.Event -- new
   | ^^^^^^^^^^^^^^^^
Failed to install generics-sop-0.4.0.0
cabal.exe: Error: some packages failed to install:

Make `Generic` a superclass of `HasDatatypeInfo`

Is there any good reason not to do this? I'm a bit surprised we did not do this from the beginning.

The type signature of datatypeInfo mentions Code, and Code is associated with Generic, so it cannot possibly be defined for a type without a Generic instance. It thus seems as if there is no way to ever have a HasDatatypeInfo instance without a Generic instance.

@edsko Do you have an opinion here?

Split into core and generics packages

See #70 and #61 .

The goal would be to provide two packages:

sop-core (or a better name) - definition of data structures and classes for working with them: NP, HTraverse etc
generics-sop - depends on sop-core and re-exports everything so the API remains the same. Any generics specific code will live here.

Question: What should the core package be called?

I am keen on providing specialisations for heterogenous lists and extensible records in the core package, as well as curry/uncurry functionality. The core package will then be in the same category as packages like HList and Vinyl. My reasoning for this is that the sop approach in generics-sop is great, but at the moment it is not immediately obvious that it can or should be used for general purpose programming with heterogenous data structures. The name should ideally be something that reflects its general purpose possibilities, rather than something specifically related to generics.

ghc 8.8 / template-haskell 2.15.0 build failure

[12 of 14] Compiling Generics.SOP.TH  ( src/Generics/SOP/TH.hs, dist/build/Generics/SOP/TH.o )

src/Generics/SOP/TH.hs:196:17: error:
    • Couldn't match expected type ‘(TypeQ -> TySynEqnQ) -> DecQ’
                  with actual type ‘Q Dec’
    • The first argument of ($) takes one argument,
      but its type ‘Q Dec’ has none
      In the expression:
        tySynInstD ''Code $ tySynEqn [typ] (codeFor f cons)
      In an equation for ‘codeSyn’:
          codeSyn = tySynInstD ''Code $ tySynEqn [typ] (codeFor f cons)
    |
196 |   let codeSyn = tySynInstD ''Code $ tySynEqn [typ] (codeFor f cons)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

src/Generics/SOP/TH.hs:196:28: error:
    • Couldn't match type ‘Name’ with ‘Q TySynEqn’
      Expected type: TySynEqnQ
        Actual type: Name
    • In the first argument of ‘tySynInstD’, namely ‘''Code’
      In the expression: tySynInstD ''Code
      In the expression:
        tySynInstD ''Code $ tySynEqn [typ] (codeFor f cons)
    |
196 |   let codeSyn = tySynInstD ''Code $ tySynEqn [typ] (codeFor f cons)
    |                            ^^^^^^

src/Generics/SOP/TH.hs:196:46: error:
    • Couldn't match expected type ‘Maybe [TyVarBndr]’
                  with actual type ‘[Q Type]’
    • In the first argument of ‘tySynEqn’, namely ‘[typ]’
      In the second argument of ‘($)’, namely
        ‘tySynEqn [typ] (codeFor f cons)’
      In the expression:
        tySynInstD ''Code $ tySynEqn [typ] (codeFor f cons)
    |
196 |   let codeSyn = tySynInstD ''Code $ tySynEqn [typ] (codeFor f cons)
    |                                              ^^^^^

src/Generics/SOP/TH.hs:304:3: error:
    • Couldn't match expected type ‘(TypeQ -> TySynEqnQ) -> Q Dec’
                  with actual type ‘Q Dec’
    • The function ‘tySynInstD’ is applied to two arguments,
      but its type ‘TySynEqnQ -> Q Dec’ has only one
      In the expression:
        tySynInstD
          ''DatatypeInfoOf
          (tySynEqn [typ] (metadataType' isNewtype typeName cs))
      In an equation for ‘metadataType’:
          metadataType typ isNewtype typeName cs
            = tySynInstD
                ''DatatypeInfoOf
                (tySynEqn [typ] (metadataType' isNewtype typeName cs))
    |
304 |   tySynInstD ''DatatypeInfoOf (tySynEqn [typ] (metadataType' isNewtype typeName cs))
    |   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

src/Generics/SOP/TH.hs:304:14: error:
    • Couldn't match type ‘Name’ with ‘Q TySynEqn’
      Expected type: TySynEqnQ
        Actual type: Name
    • In the first argument of ‘tySynInstD’, namely ‘''DatatypeInfoOf’
      In the expression:
        tySynInstD
          ''DatatypeInfoOf
          (tySynEqn [typ] (metadataType' isNewtype typeName cs))
      In an equation for ‘metadataType’:
          metadataType typ isNewtype typeName cs
            = tySynInstD
                ''DatatypeInfoOf
                (tySynEqn [typ] (metadataType' isNewtype typeName cs))
    |
304 |   tySynInstD ''DatatypeInfoOf (tySynEqn [typ] (metadataType' isNewtype typeName cs))
    |              ^^^^^^^^^^^^^^^^

src/Generics/SOP/TH.hs:304:41: error:
    • Couldn't match expected type ‘Maybe [TyVarBndr]’
                  with actual type ‘[Q Type]’
    • In the first argument of ‘tySynEqn’, namely ‘[typ]’
      In the second argument of ‘tySynInstD’, namely
        ‘(tySynEqn [typ] (metadataType' isNewtype typeName cs))’
      In the expression:
        tySynInstD
          ''DatatypeInfoOf
          (tySynEqn [typ] (metadataType' isNewtype typeName cs))
    |
304 |   tySynInstD ''DatatypeInfoOf (tySynEqn [typ] (metadataType' isNewtype typeName cs))
    |                                         ^^^^^

Avoiding partial pattern matches using type-level metadata

Let’s say I have the following fieldNames function which gives me back a list of the names of record fields:

{-# LANGUAGE DataKinds, FlexibleContexts, GADTs, ScopedTypeVariables, TypeOperators #-}
module PartialSOP
  ( fieldNames
  ) where

import           GHC.TypeLits
import           Generics.SOP
import qualified Generics.SOP.Type.Metadata as T

fieldNames :: forall a proxy modName tyName constrName fields.
  ( Generic a
  , HasDatatypeInfo a
  , KnownSymbol modName
  , KnownSymbol tyName
  , DatatypeInfoOf a ~ 'T.ADT modName tyName '[ 'T.Record constrName fields]
  ) => proxy a -> [String]
fieldNames proxy =
  case datatypeInfo proxy of
    ADT _modName _tyName constrInfos ->
      case constrInfos of
        (constr :* Nil) ->
          case constr of
            Record _name fields ->
              let f :: forall f. FieldInfo f -> K String f
                  f (FieldInfo name) = K name
              in hcollapse (hliftA f fields)

This seems to work just fine and the equality constraint on DatatypeInfoOf should prevent users from using it on non-record types. However, datatypeInfo is giving me the term-level metadata and at that point I’ve lost all info about the metadata so GHC rightfully complains that the pattern matches are partial. I can convince it that the pattern match on constrInfos is not partial using a Code a ~ '[r] constraint (where r is a fresh type variable) but I’m failing to do something similar for the other two pattern matches.

It seems like convincing GHC that this is safe would require some kind of GADT which includes metadata. Does something like this already exist? If not have you thought about adding this? I haven’t tried creating one so far so I don’t know what problems will result from them but if you’re open to adding it I’ll give it a shot.

Build failure on GHC 8

setupCompilerEnvironmentPhase
Build with /nix/store/fih8xk5p1yxjawyagm7ns3l3grjwmh41-ghc-8.1.20161109.
ghc-pkg: unable to decommit memory: Invalid argument
ghc-pkg: unable to decommit memory: Invalid argument
unpacking sources
unpacking source archive /nix/store/7qqjfh7xfv8y78cvj9qa7s0gk4gq6mqy-generics-sop-0.2.2.0.tar.gz
source root is generics-sop-0.2.2.0
setting SOURCE_DATE_EPOCH to timestamp 1468130830 of file generics-sop-0.2.2.0/test/Example.hs
patching sources
compileBuildDriverPhase
setupCompileFlags: -package-db=/tmp/nix-build-generics-sop-0.2.2.0.drv-0/package.conf.d -j1 -threaded
[1 of 1] Compiling Main             ( Setup.hs, /tmp/nix-build-generics-sop-0.2.2.0.drv-0/Main.o )
Linking Setup ...
configuring
configureFlags: --verbose --prefix=/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0 --libdir=$prefix/lib/$compiler --libsubdir=$pkgid --with-gcc=gcc --package-db=/tmp/nix-build-generics-sop-0.2.2.0.drv-0/package.conf.d --ghc-option=-optl=-Wl,-rpath=/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/lib/ghc-8.1.20161109/generics-sop-0.2.2.0 --enable-split-objs --disable-library-profiling --disable-profiling --enable-shared --enable-library-vanilla --enable-executable-dynamic --enable-tests
Configuring generics-sop-0.2.2.0...
Dependency base >=4.6 && <5: using base-4.9.0.0
Dependency generics-sop -any: using generics-sop-0.2.2.0
Dependency ghc-prim >=0.3 && <0.6: using ghc-prim-0.5.0.0
Dependency template-haskell >=2.8 && <2.12: using template-haskell-2.11.0.0
Setup: unable to decommit memory: Invalid argument
Source component graph:
    component lib
    component test:generic-sop-examples dependency lib
Configured component graph:
    component generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x
        include base-4.9.0.0
        include ghc-prim-0.5.0.0
        include template-haskell-2.11.0.0
    component generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x-generic-sop-examples
        include base-4.9.0.0
        include generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x
Linked component graph:
    unit generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x
        include base-4.9.0.0
        include ghc-prim-0.5.0.0
        include template-haskell-2.11.0.0
    unit generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x-generic-sop-examples
        include base-4.9.0.0
        include generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x
Ready component graph:
    definite generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x
        depends base-4.9.0.0
        depends ghc-prim-0.5.0.0
        depends template-haskell-2.11.0.0
    definite generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x-generic-sop-examples
        depends base-4.9.0.0
        depends generics-sop-0.2.2.0-9w3u9oOObG6ENcL8IZfj8x
Using Cabal-1.25.0.0 compiled by ghc-8.1
Using compiler: ghc-8.1.20161109
Using install prefix:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0
Binaries installed in:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/bin
Libraries installed in:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/lib/ghc-8.1.20161109/generics-sop-0.2.2.0
Dynamic Libraries installed in:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/lib/ghc-8.1.20161109/x86_64-linux-ghc-8.1.20161109
Private binaries installed in:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/libexec
Data files installed in:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/share/x86_64-linux-ghc-8.1.20161109/generics-sop-0.2.2.0
Documentation installed in:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/share/doc/x86_64-linux-ghc-8.1.20161109/generics-sop-0.2.2.0
Configuration files installed in:
/nix/store/d7w3m1p8hbgxx35ydayiqvwsc23s3jz5-generics-sop-0.2.2.0/etc
No alex found
Using ar found on system at:
/nix/store/9kxhc2f3j5lcyh00zcrifvfly33n5m4r-binutils-2.27/bin/ar
No c2hs found
No cpphs found
Using gcc version 5.4.0 given by user at:
/nix/store/d78lwdgpd05c7qjdg4bvsx38jfy3vs43-gcc-wrapper-5.4.0/bin/gcc
Using ghc version 8.1.20161109 found on system at:
/nix/store/fih8xk5p1yxjawyagm7ns3l3grjwmh41-ghc-8.1.20161109/bin/ghc
Using ghc-pkg version 8.1.20161109 found on system at:
/nix/store/fih8xk5p1yxjawyagm7ns3l3grjwmh41-ghc-8.1.20161109/bin/ghc-pkg
No ghcjs found
No ghcjs-pkg found
No greencard found
Using haddock version 2.17.2 found on system at:
/nix/store/fih8xk5p1yxjawyagm7ns3l3grjwmh41-ghc-8.1.20161109/bin/haddock
No happy found
Using haskell-suite found on system at: haskell-suite-dummy-location
Using haskell-suite-pkg found on system at: haskell-suite-pkg-dummy-location
No hmake found
Using hpc version 0.67 found on system at:
/nix/store/fih8xk5p1yxjawyagm7ns3l3grjwmh41-ghc-8.1.20161109/bin/hpc
Using hsc2hs version 0.68 found on system at:
/nix/store/fih8xk5p1yxjawyagm7ns3l3grjwmh41-ghc-8.1.20161109/bin/hsc2hs
Using hscolour version 1.24 found on system at:
/nix/store/1akjfhrqd0xandv1iyqs0sy4cgi5cvva-hscolour-1.24.1/bin/HsColour
No jhc found
Using ld found on system at:
/nix/store/d78lwdgpd05c7qjdg4bvsx38jfy3vs43-gcc-wrapper-5.4.0/bin/ld
No lhc found
No lhc-pkg found
No pkg-config found
Using strip version 2.27 found on system at:
/nix/store/9kxhc2f3j5lcyh00zcrifvfly33n5m4r-binutils-2.27/bin/strip
Using tar found on system at:
/nix/store/ipsnpg6csk11j07lzlsd7z5mm30wsb4x-gnutar-1.29/bin/tar
No uhc found
Setup: unable to decommit memory: Invalid argument
Setup: unable to decommit memory: Invalid argument
building
Preprocessing library generics-sop-0.2.2.0...
[ 1 of 13] Compiling Generics.SOP.BasicFunctors ( src/Generics/SOP/BasicFunctors.hs, dist/build/Generics/SOP/BasicFunctors.o )
[ 2 of 13] Compiling Generics.SOP.Sing ( src/Generics/SOP/Sing.hs, dist/build/Generics/SOP/Sing.o )
[ 3 of 13] Compiling Generics.SOP.Constraint ( src/Generics/SOP/Constraint.hs, dist/build/Generics/SOP/Constraint.o )
[ 4 of 13] Compiling Generics.SOP.Classes ( src/Generics/SOP/Classes.hs, dist/build/Generics/SOP/Classes.o )
[ 5 of 13] Compiling Generics.SOP.NP  ( src/Generics/SOP/NP.hs, dist/build/Generics/SOP/NP.o )
[ 6 of 13] Compiling Generics.SOP.NS  ( src/Generics/SOP/NS.hs, dist/build/Generics/SOP/NS.o )
[ 7 of 13] Compiling Generics.SOP.Metadata ( src/Generics/SOP/Metadata.hs, dist/build/Generics/SOP/Metadata.o )
[ 8 of 13] Compiling Generics.SOP.GGP ( src/Generics/SOP/GGP.hs, dist/build/Generics/SOP/GGP.o )
[ 9 of 13] Compiling Generics.SOP.Dict ( src/Generics/SOP/Dict.hs, dist/build/Generics/SOP/Dict.o )
[10 of 13] Compiling Generics.SOP.Universe ( src/Generics/SOP/Universe.hs, dist/build/Generics/SOP/Universe.o )
[11 of 13] Compiling Generics.SOP.TH  ( src/Generics/SOP/TH.hs, dist/build/Generics/SOP/TH.o )

src/Generics/SOP/TH.hs:230:25: error:
    • Ambiguous occurrence ‘Newtype’
      It could refer to either ‘Language.Haskell.TH.Syntax.Newtype’,
                               imported from ‘Language.Haskell.TH.Syntax’ at src/Generics/SOP/TH.hs:13:1-48
                            or ‘Generics.SOP.Metadata.Newtype’,
                               imported from ‘Generics.SOP.Metadata’ at src/Generics/SOP/TH.hs:16:1-28
                               (and originally defined at src/Generics/SOP/Metadata.hs:40:3-91)
    • In the Template Haskell quotation
        [| Newtype
             ($(stringE (nameModule' typeName)))
             ($(stringE (nameBase typeName)))
             ($(mdCon (head cs))) |]

Make 0.4 release

I guess the correct way to make sure we're not getting into problems with Stackage is to:

• release sop-core first and submit to Stackage as new package;
• wait for the PR to get merged;
• then release generics-sop.

@phadej, can you confirm that this is the right way?

GHC loop when reporting error

The following (cut down) code causes GHC 8.6.3 to start reporting an error and then loop:

{-# LANGUAGE TypeApplications, ScopedTypeVariables, GADTs, MultiParamTypeClasses,
                         FlexibleContexts, AllowAmbiguousTypes #-}
module Main where

import Data.Functor.Identity
import Generics.SOP

class (f a ~ b) => IsHKD f a b

dump
  :: forall t1
   . (Generic t1, AllZip2 (IsHKD Identity) (Code t1) (Code t1))
  => t1 -> ()
dump = dump_SOP @(Code t1) . from

dump_SOP
  :: forall xss f
   . (AllZip2 (IsHKD f) xss xss)
  => SOP I xss
  -> ()
dump_SOP _ = ()

main = return ()

GHCi, version 8.6.3: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Main             ( src/Main.hs, interpreted )

src/Main.hs:13:8: error:

It's legitimate that there is an error - at the very least I haven't given the compiler enough information to instantiate f to Identity - but having the compiler loop instead of telling me the error is inconvenient :-)

I inlined generics-sop and cut it down further.

{-# LANGUAGE FlexibleContexts, FlexibleInstances, AllowAmbiguousTypes,
                         UndecidableSuperClasses #-}
module M where

data I = I

class All x => All x

class AllZip2 f
instance AllZip2 f

f1 :: (All x, AllZip2 I) => x -> ()
f1 = f2

f2 :: AllZip2 f => x -> ()
f2 _ = ()

The key features of the generics-sop code that trigger it seem to be the recursive instance for All (in the generics-sop codebase this is actually All Top => All c) and the catch-all instance for AllZip2.

I guess that even though generics-sop is using UndecidableSuperClasses, GHC ought not to be looping indefinitely in the middle of trying to report an error. I'll report a bug there after the trac->gitlab migration is done. But I thought it would be useful to at least have a record of it here.

Add more helper functions for working with (type-level) metadata

See #53 for some examples.

fieldNames would be nice to have on the term-level.

IsRecordType would be nice to have on the type-level.

But also type functions for operating on / extracting aspects of the type-level metadata structure would probably be useful. E.g. type-level equivalents of moduleName, datatypeName, constructorInfo, constructorName, fieldName.

Release 0.3.1.0

I think we should merge PR #43 when it is ready and then release 0.3.1.0.

There's already a significant amount of new stuff in this release, in particular with the addition of AllZip and HTrans.

If anyone wants extra features in 0.3.1.0, speak up now.

Resolve name clash between `GHC.Generics.Generic` and `Generics.SOP.Generic`

It's unfortunate that writing a data type with an instance for Generics.SOP.Generic forces you to tinker around with qualified imports or qualified names. The reason is that both classes are called Generic.

Here's an example:

import Generics.SOP
import GHC.Generics

data Foo = Foo
  deriving (GHC.Generics.Generic)

instance Generics.SOP.Generic Foo

One possible solution is to find another name for Generics.SOP.Generic (which of course would be a breaking change). Are there other possible solutions? Maybe I'm just overlooking a better way to write this down.

Improve performance with inlining trick?

*traverse*, *cata*, *pure*, etc. can all potentially be implemented using recursive class instances like

class Thing xs where thing :: thing xs
instance Thing '[] where {-# INLINE thing #-} thing = _
instance Thing (x ': xs) where {-# INLINE thing #-} thing = _

which would allow them to be zero overhead, and, hopefully, for GHC to be able to eliminate generics-sop overheads entirely. I'm not sure whether sList has the effect of tricking GHC into doing this already but that does seem possible.

Cons: yet longer compile times, yet more dictionaries to pass around.

Check / update / auto-generate (?) instances list

The library provides instances for most(?) base types in Generics.SOP.Instances. However, I don't think this list has been updated since the first release. And base has certainly changed significantly.

At least, we should look which new types we should perhaps add.

Even better, we should write a tool that can scan base and generate the module for us.

Super classes and All - improve documentation

I've documented an issue I've encountered using the All constraint on stack overflow but I'm wondering if this is a better place, so I figured I'd drop an issue here as well - if the SO gets answered then I'll close this issue with a reference to it.

Exhaustivity workaround in ap_NS

I was surprised to see, which compiles even on GHC-8.0 and GHC-8.2 without warnings

ap_NS :: NP (f -.-> g) xs -> NS f xs -> NS g xs
ap_NS (Fn f  :* _)   (Z x)   = Z (f x)
ap_NS (_     :* fs)  (S xs)  = S (ap_NS fs xs)
ap_NS _ _ = error "inaccessible"

Is this "bug" recorded in GHC Trac?

Use collapsible headers for examples and instances

This seems to be the advertised way to do this in Haddock now.

See the discussion in #24 for slightly more info.

What is the purpose of the "AllF" type family in the "All" class?

Hi, I have a question about the design of the library.

The All class is used to require a constraint for every element of a list. The constraint is required for each element x two times. One in this instance:

instance (c x, All c xs) => All c (x ': xs) where
  cpara_SList p nil cons =
    cons (cpara_SList p nil cons)

And another in AllF, a type family which creates a "list" of preconditions for All:

class (AllF c xs, SListI xs) => All ...

type family
  AllF (c :: k -> Constraint) (xs :: [k]) :: Constraint where
  AllF _c '[]       = ()
  AllF  c (x ': xs) = (c x, All c xs)

My question is, what necessitates this duplication? Especially considering that AllF seems to require somewhat advanced extensions like UndecidableSuperClasses.

Make a release with type-level metadata

Probably based on the type-level-metadata-simpler branch.

Advantages:

• This would finally solve #8, which is open for a very long time.
• This would allow the release of kosmikus/records-sop which has been requested in kosmikus/records-sop#1.
• The unreleased work I'm doing on datatype-generic migrations using generics-sop is also depending on these features.
• The changes should be mostly backwards compatible.
• I'd welcome more feedback on the interface, and more experience with the type-level features (see below).

Disadvantages:

• I'm not completely happy with the new features yet, so they'll most likely change again in the future.
• There is no easy way to add this without dropping ghc-7 support.

Any thoughts? @phadej @nmattia @jkarni ?

Add NFData instances

If no one picks this up, I'll do that around GHC8.2 time (after new deepseq release, with NFData1/2)

Interaction with OverloadedRecordFields

One thing I like of this library is the ability to wrap every field with a type constructor, usually some Applicative.

In plain records (let's not consider sum types) it would be nice if we could manipulate the corresponding representation using the same "labels" that we use for the original record, allowing us to get and set individual "wrapped" values. Perhaps the upcoming OverloadedRecordFields extension could help with this?

Consider a type like

data Wrapped f r = ...

with a constructor like

makeWrapped :: (Generic r, Code r ~ '[ xs ]) => NP f xs -> Wrapped f r

With OverloadedRecordFields (here is an implementation package based in the proposal) perhaps we could define an instance like

instance (HasField symbol r a) => HasField symbol (Wrapped f r) (f a) where ...

Would this make sense? Or perhaps the idea of directly accessing individual fields of the representation is misguided?

version 0.1.0.3 doesn't compile

A complete build log is at http://hydra.cryp.to/build/286232/log/raw. The significant bit seems to be:

Building generics-sop-0.1.0.3...
Preprocessing library generics-sop-0.1.0.3...

src/Generics/SOP.hs:299:0:
     error: missing binary operator before token "("
     #if !(defined(__HADDOCK_VERSION__)) || MIN_TOOL_VERSION_haddock(2,14,0)
     ^

Maybe the cpp extension isn't declared in the cabal file?

Release 0.2.4.0

I think we should make a new release with the lifted-classes changes and the index functions.

@phadej, any objections or anything else you want to have in?

1.0 ?

I would like this library to reach version 1.0 ultimately, and I think it has been remarkably stable for a while now, despite the fact that it's still relatively young and that it uses a whole lot of different language extensions.

I'd like to collect opinions here as to what (if anything) everyone who's currently using the library would like to see before we label a release to be 1.0.