Comments (4)
dhess
commented on June 5, 2024
Another one:
two interp agree: FAIL (5.57s)
--
| ✗ two interp agree failed at test/Tests/EvalFullInterp.hs:458:60
| after 752 tests, 3 shrinks and 580 discards.
| shrink path: 752/580:qA2
| both terminated 100% ███████████████████▉
| one failed to terminate 0% ····················
|
| ┏━━ test/Tests/Eval/Utils.hs ━━━
| 51 ┃ genDirTm :: PropertyT WT (Dir, Expr, Type' () ())
| 52 ┃ genDirTm = do
| 53 ┃ dir <- forAllT $ Gen.element @[] [Chk, Syn]
| ┃ │ Syn
| 54 ┃ (t', ty) <- case dir of
| 55 ┃ Chk -> do
| 56 ┃ ty' <- forAllT $ genWTType $ KType ()
| 57 ┃ t' <- forAllT $ genChk ty'
| 58 ┃ pure (t', ty')
| 59 ┃ Syn -> forAllT genSyn
| ┃ │ ( App
| ┃ │ ()
| ┃ │ (Let
| ┃ │ ()
| ┃ │ LocalName { unLocalName = "x" }
| ┃ │ (EmptyHole ())
| ┃ │ (App
| ┃ │ ()
| ┃ │ (Var
| ┃ │ ()
| ┃ │ (GlobalVarRef
| ┃ │ GlobalName
| ┃ │ { qualifiedModule =
| ┃ │ ModuleName { unModuleName = "Primitives" :\| [] }
| ┃ │ , baseName = "const"
| ┃ │ }))
| ┃ │ (EmptyHole ())))
| ┃ │ (EmptyHole ())
| ┃ │ , TCon
| ┃ │ ()
| ┃ │ GlobalName
| ┃ │ { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
| ┃ │ , baseName = "Bool"
| ┃ │ }
| ┃ │ )
| 60 ┃ t <- generateIDs t'
| 61 ┃ pure (dir, t, ty)
|
| ┏━━ test/Tests/EvalFullInterp.hs ━━━
| 445 ┃ tasty_two_interp_agree :: Property
| 446 ┃ tasty_two_interp_agree = withTests 1000
| 447 ┃ $ withDiscards 2000
| 448 ┃ $ propertyWT testModules
| 449 ┃ $ do
| 450 ┃ let globs = foldMap' moduleDefsQualified $ create' $ sequence testModules
| 451 ┃ tds <- asks typeDefs
| 452 ┃ (dir, t, _ty) <- genDirTm
| 453 ┃ let optsV = ViewRedexOptions{groupedLets = True, aggressiveElision = True, avoidShadowing = False}
| 454 ┃ let optsR = RunRedexOptions{pushAndElide = True}
| 455 ┃ (_, ss) <- failWhenSevereLogs $ evalFullStepCount @EvalLog UnderBinders optsV optsR tds globs 100 dir t
| 456 ┃ si <- liftIO (evalFullTest' (MicroSec 10_000) tds globs dir $ forgetMetadata t)
| 457 ┃ case (ss, si) of
| 458 ┃ (Right ss', Right si') -> label "both terminated" >> Hedgehog.diff (forgetMetadata ss') alphaEq si'
| ┃ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| ┃ │ ━━━ Failed (- lhs) (+ rhs) ━━━
| ┃ │ - Ann
| ┃ │ - ()
| ┃ │ - (EmptyHole ())
| ┃ │ - (TCon
| ┃ │ - ()
| ┃ │ - GlobalName
| ┃ │ - { qualifiedModule = ModuleName { unModuleName = "Builtins" :\| [] }
| ┃ │ - , baseName = "Bool"
| ┃ │ - })
| ┃ │ + App
| ┃ │ + ()
| ┃ │ + (App
| ┃ │ + ()
| ┃ │ + (Var
| ┃ │ + ()
| ┃ │ + (GlobalVarRef
| ┃ │ + GlobalName
| ┃ │ + { qualifiedModule =
| ┃ │ + ModuleName { unModuleName = "Primitives" :\| [] }
| ┃ │ + , baseName = "const"
| ┃ │ + }))
| ┃ │ + (EmptyHole ()))
| ┃ │ + (EmptyHole ())
| 459 ┃ _ -> label "one failed to terminate"
|
| This failure can be reproduced by running:
| > recheckAt (Seed 8128149785453899409 10689373616275667251) "752/580:qA2" two interp agree
|
| Use "--pattern '$NF ~ /two interp agree/' --hedgehog-replay '752/580:qA2 Seed 8128149785453899409 10689373616275667251'" to reproduce from the command-line.
|
| Use -p '/two interp agree/' to rerun this test only.
|
| 1 out of 854 tests failed (88.19s)
Ref:
Raw logs:
from primer.
dhess
commented on June 5, 2024
Tests
--
| EvalFullInterp
| two interp agree: FAIL (1.45s)
| ✗ two interp agree failed at test/Tests/EvalFullInterp.hs:467:60
| after 136 tests, 9 shrinks and 105 discards.
| shrink path: 136/105:f2DgCbA3
| both terminated 99% ███████████████████▊
|
| ┏━━ test/Tests/Eval/Utils.hs ━━━
| 51 ┃ genDirTm :: PropertyT WT (Dir, Expr, Type' () ())
| 52 ┃ genDirTm = do
| 53 ┃ dir <- forAllT $ Gen.element @[] [Chk, Syn]
| ┃ │ Syn
| 54 ┃ (t', ty) <- case dir of
| 55 ┃ Chk -> do
| 56 ┃ ty' <- forAllT $ genWTType $ KType ()
| 57 ┃ t' <- forAllT $ genChk ty'
| 58 ┃ pure (t', ty')
| 59 ┃ Syn -> forAllT genSyn
| ┃ │ ( Ann
| ┃ │ ()
| ┃ │ (LAM
| ┃ │ ()
| ┃ │ LocalName { unLocalName = "x" }
| ┃ │ (Case
| ┃ │ ()
| ┃ │ (Ann
| ┃ │ ()
| ┃ │ (EmptyHole ())
| ┃ │ (TCon
| ┃ │ ()
| ┃ │ GlobalName
| ┃ │ { qualifiedModule =
| ┃ │ ModuleName { unModuleName = "Primitives" :\| [] }
| ┃ │ , baseName = "Int"
| ┃ │ }))
| ┃ │ []
| ┃ │ (CaseFallback
| ┃ │ (Let
| ┃ │ ()
| ┃ │ LocalName { unLocalName = "x" }
| ┃ │ (EmptyHole ())
| ┃ │ (Hole
| ┃ │ ()
| ┃ │ (Case
| ┃ │ ()
| ┃ │ (Ann
| ┃ │ ()
| ┃ │ (EmptyHole ())
| ┃ │ (TCon
| ┃ │ ()
| ┃ │ GlobalName
| ┃ │ { qualifiedModule =
| ┃ │ ModuleName { unModuleName = "Primitives" :\| [] }
| ┃ │ , baseName = "Char"
| ┃ │ }))
| ┃ │ []
| ┃ │ (CaseFallback
| ┃ │ (Letrec
| ┃ │ ()
| ┃ │ LocalName { unLocalName = "y" }
| ┃ │ (EmptyHole ())
| ┃ │ (TEmptyHole ())
| ┃ │ (Var
| ┃ │ ()
| ┃ │ (GlobalVarRef
| ┃ │ GlobalName
| ┃ │ { qualifiedModule = ModuleName { unModuleName = "M" :\| [] }
| ┃ │ , baseName = "idChar"
| ┃ │ }))))))))))
| ┃ │ (TEmptyHole ())
| ┃ │ , TEmptyHole ()
| ┃ │ )
| 60 ┃ t <- generateIDs t'
| 61 ┃ pure (dir, t, ty)
|
| ┏━━ test/Tests/EvalFullInterp.hs ━━━
| 454 ┃ tasty_two_interp_agree :: Property
| 455 ┃ tasty_two_interp_agree = withTests 1000
| 456 ┃ $ withDiscards 2000
| 457 ┃ $ propertyWT testModules
| 458 ┃ $ do
| 459 ┃ let globs = foldMap' moduleDefsQualified $ create' $ sequence testModules
| 460 ┃ tds <- asks typeDefs
| 461 ┃ (dir, t, _ty) <- genDirTm
| 462 ┃ let optsV = ViewRedexOptions{groupedLets = True, aggressiveElision = True, avoidShadowing = False}
| 463 ┃ let optsR = RunRedexOptions{pushAndElide = True}
| 464 ┃ (_, ss) <- failWhenSevereLogs $ evalFullStepCount @EvalLog UnderBinders optsV optsR tds globs 100 dir t
| 465 ┃ si <- liftIO (evalFullTest' (MicroSec 10_000) tds globs dir $ forgetMetadata t)
| 466 ┃ case (ss, si) of
| 467 ┃ (Right ss', Right si') -> label "both terminated" >> Hedgehog.diff (forgetMetadata ss') alphaEq si'
| ┃ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| ┃ │ ━━━ Failed (- lhs) (+ rhs) ━━━
| ┃ │ Ann
| ┃ │ ()
| ┃ │ LAM
| ┃ │ ()
| ┃ │ LocalName { unLocalName = "x" }
| ┃ │ Hole
| ┃ │ ()
| ┃ │ - Ann
| ┃ │ - ()
| ┃ │ - (Lam
| ┃ │ - ()
| ┃ │ - LocalName { unLocalName = "x" }
| ┃ │ - (Var () (LocalVarRef LocalName { unLocalName = "x" })))
| ┃ │ - (TFun
| ┃ │ - ()
| ┃ │ - (TCon
| ┃ │ - ()
| ┃ │ - GlobalName
| ┃ │ - { qualifiedModule =
| ┃ │ - ModuleName { unModuleName = "Primitives" :\| [] }
| ┃ │ - , baseName = "Char"
| ┃ │ - })
| ┃ │ - (TCon
| ┃ │ - ()
| ┃ │ - GlobalName
| ┃ │ - { qualifiedModule =
| ┃ │ - ModuleName { unModuleName = "Primitives" :\| [] }
| ┃ │ - , baseName = "Char"
| ┃ │ - }))
| ┃ │ + Lam
| ┃ │ + ()
| ┃ │ + LocalName { unLocalName = "x" }
| ┃ │ + (Var () (LocalVarRef LocalName { unLocalName = "x" }))
| ┃ │ TEmptyHole ()
| 468 ┃ _ -> label "one failed to terminate"
|
| This failure can be reproduced by running:
| > recheckAt (Seed 18338648843675604054 720643755453659811) "136/105:f2DgCbA3" two interp agree
|
| Use "--pattern '$NF ~ /two interp agree/' --hedgehog-replay '136/105:f2DgCbA3 Seed 18338648843675604054 720643755453659811'" to reproduce from the command-line.
|
| Use -p '/two interp agree/' to rerun this test only.
|
| 1 out of 870 tests failed (106.09s)
Ref:
Raw logs:
from primer.
dhess
commented on June 5, 2024
It seems plausible that at least one of these is related to #1247.
from primer.
dhess
commented on June 5, 2024
Yet another:
https://buildkite.com/hackworthltd/primer-ci/builds/113#018fdac8-a37c-4d44-8b25-5fec6014c7e3
from primer.
Related Issues (20)
- Are we building (should we build) dependencies with `-O2`
- More robust Wasm support
- When looking for matches for holes, prefer local bindings over top-level/in-scope module binding
- Future work on interpreter
- wasm: always build with `-O2`
- Property test failure (possibly Wasm-related?) HOT 1
- Primer language -> Wasm compiler HOT 1
- Compile Primer programs to Wasm
- Only run Wasm tests on merge queue or workflow dispatch HOT 2
- Use Buildkite artifacts to cache Wasm build artifacts HOT 1
- Benchmark results aren’t fetched from Cachix HOT 3
- `primer-service`: look into RFC 9457
- Duplication in interpreter implementation
- Hook interpreter up to API
- `tasty_redex_independent` property test failure
- `tasty_multiple_requests_accepted` property test failures HOT 2
- `RecordPair TyConName ValConName` does not serialize nicely in the OpenAPI API
- Interpreter can't reduce top-level definitions
- Investigate `weeder-nix`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from primer.