facebook / haxl Goto Github PK
View Code? Open in Web Editor NEWA Haskell library that simplifies access to remote data, such as databases or web-based services.
License: BSD 3-Clause "New" or "Revised" License
A Haskell library that simplifies access to remote data, such as databases or web-based services.
License: BSD 3-Clause "New" or "Revised" License
Most relevant details:
tests/DataCacheTest.hs:33:15: error:
• Couldn't match type ‘GHC.Ptr.Ptr GHC.Stack.CCS.CostCentreStack
-> IVar u0 w0 a’
with ‘IVar u w a’
Expected type: IO (IVar u w a)
Actual type: IO
(GHC.Ptr.Ptr GHC.Stack.CCS.CostCentreStack -> IVar u0 w0 a)
• In the expression: IVar <$> newIORef (IVarFull (Ok a NilWrites))
In an equation for ‘newResult’:
newResult a = IVar <$> newIORef (IVarFull (Ok a NilWrites))
• Relevant bindings include
a :: a (bound at tests/DataCacheTest.hs:33:11)
newResult :: a -> IO (IVar u w a)
(bound at tests/DataCacheTest.hs:33:1)
|
33 | newResult a = IVar <$> newIORef (IVarFull (Ok a NilWrites))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/DataCacheTest.hs:36:13: error:
• The constructor ‘IVar’ should have 2 arguments, but has been given 1
• In the pattern: IVar ref
In an equation for ‘takeResult’:
takeResult (IVar ref)
= do e <- readIORef ref
case e of
IVarFull a -> return a
_ -> error "takeResult"
|
36 | takeResult (IVar ref) = do
| ^^^^^^^^
Complete build log: haxl-build-failure.log
With a little work I can supply a minimal nix derivation to reproduce the issue.
Current workaround is to pretend that haxl doesn't have any executables so we don't enable executable profiling.
From documentation it is clear that fetch requests are cached between rounds of fetch. But It is not clear (at least for me) if DataSource fetch can get more than one equal Requests. For example
data SqlRequest a where
RequestComments :: PostId -> SqlRequest [Comment]
instance DataSource () SqlRequest where
fetch state _ _ fetches = -- can fetches here contain more than one RequestComments to same PostId?
When I was trying to build the Haxl-2.3.0.0 using stack, I got this error message.
The version of GHC for building this is 8.8.3
Haxl\Core\Exception.hs:303:20: error:
haxl > Ambiguous occurrence `MonadFail'
haxl > It could refer to
haxl > either `Prelude.MonadFail',
haxl > imported from `Prelude' at Haxl\Core\Exception.hs:31:8-26
haxl > (and originally defined in `Control.Monad.Fail')
haxl > or `Haxl.Core.Exception.MonadFail',
haxl > defined at Haxl\Core\Exception.hs:300:1
perhaps i've missed something - I have some datasources that rely on other datasources to get their data. Is there some way of indicating in the Datasource implementation that fetch should be a Haxl expression, rather than just IO?
Missing test file:
tests/TestExampleDataSource.hs: loadcache <- readFile "sigma/haxl/core/tests/LoadCache.txt"
I have taken look at some talks of @simonmar about haxl:
And looks like Haxl is not lawful as it's ignoring law (<*>) = ap
.
also as (>>) = (*>)
is implied by that law it causes issues like one in first link (1.) about sequencing monadic actions in do even when applicative do was off:
run $ do
cmd "touch" ["foo"]
cmd "rm" ["foo"]
And there was need to add _ <-
in order that to be sequential:
run $ do
_ <- cmd "touch" ["foo"]
cmd "rm" ["foo"]
My question are:
Also I have taken look at the original paper about Applicative and did get reasons why this law was intorduced, so if you know any discussion on that topic you can link would be great!
Also in js community some implementations of Futures also Ignore that law (fantasy-land#179), looks like this law is not that widely known.
While trying to compile with ghc-8.2
/tmp/stack12680/haxl-0.5.0.0/Haxl/Core/Types.hs:112:1: error:
Could not find module ‘Data.Typeable.Internal’
it is a hidden module in the package ‘base-4.10.0.0’
Use -v to see a list of the files searched for.
|
112 | import Data.Typeable.Internal
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Replacing the import with Data.Typeable
seemed to fix the issue.
compiler : The Glorious Glasgow Haskell Compilation System, version 8.2.0.20170507
I propose to rename Show1
class to ShowP
or ShowU
(for "show parametric" or "show universally").
the current Show1
clashes with Show1
class in base
, which is a bit un-ergonomic.
There is already throw
and catch
.
Is it ok to make a PR for this?
Currently, the fetch
method of the type class lives in IO
. Flaxl showed that it is possible to free it into any monad by relaxing the constraints from IO
. Any plans for Haxl
to move to or also provide a monad transformer interface?
Hi,
Sorry to bother, but is there any plans on releasing a new version to Hackage as the current highest version 2.1.2.0 doesn't work on ghc 8.8?
I'm using Haxl to talk to a REST API, and for the sake of example, we can pretend like there are two endpoints. One endpoint lets me fetch a Message
given an ID, and another endpoint lets me fetch, say, n
messages that were sent before a given ID.
data FooRequest a where
GetMessage :: Id -> FooRequest Message
GetMessagesBefore :: -> Id -> Int -> FooRequest [Message]
Whenever I perform a GetMessagesBefore
request, I would like to be able to cache the results of this request such that performing a GetMessage
request with an id fetched previously via GetMessagesBefore
hits the cache.
I can do something like:
-- let's pretend we have `env`
getMessagesBefore id_ count = runHaxl env $ do
res <- dataFetch (GetMessagesBefore id_ count)
for res $ \message -> cacheRequest (GetMessage (messageId message)) (Right message)
However, cacheRequest
throws an exception if the request has already been cached.
Thanks for your time!
I am getting stuck right when I call runHaxl
. I enabled tracing and the only output is:
schedule: 1
What usually causes this?
... but this repository has version: 0.3.1.0
. Is the development branched privately?
http://hdiff.luite.com/cgit/haxl/diff?id=0.3.1.0&id2=0.3.0.0 (seems that not, but still weird).
:Haxl/ (master✗) $ ./Setup configure [16:14:51]
Configuring haxl-0.3.1.0...
Setup: Encountered missing dependencies:
aeson >=0.6 && <0.12, exceptions ==0.8., text >=1.1.0.1 && <1.3
:Haxl/ (master✗) $ cabal install aeson [16:14:53]
Resolving dependencies...
All the requested packages are already installed:
aeson-0.11.2.0
Use --reinstall if you want to reinstall anyway.
:Haxl/ (master✗) $ cabal install exceptions [16:15:02]
Resolving dependencies...
All the requested packages are already installed:
exceptions-0.8.3
Use --reinstall if you want to reinstall anyway.
:Haxl/ (master✗) $ cabal install text [16:15:10]
Resolving dependencies...
All the requested packages are already installed:
text-1.2.2.1
Use --reinstall if you want to reinstall anyway.
:Haxl/ (master✗) $ ./Setup configure [16:15:17]
Configuring haxl-0.3.1.0...
Setup: Encountered missing dependencies:
aeson >=0.6 && <0.12, exceptions ==0.8., text >=1.1.0.1 && <1.3
Hi there,
I've been using this great library for some time now, thanks for opening it up to the public!
My use case involves large amounts of data, so I had to change the library slightly to be able to use a bounded cache. I have made it configurable so that each type of request can either use a bounded cache of a given size, or an unbounded cache.
In order to configure it, I have added an additional typeclass that has a DataSource has to be an instance of. It has a single function, cacheSize, that determines the size of the cache for each request (which may be infinite).
If this is useful to other people, I'd be glad to do a pull request. Notice however, that this would be a breaking change, since all DataSources have to add an instance of the new typeclass (which is easy to do).
I have not yet added tests that check that the cache stays within the specified bounds, nor done benchmarks (but it's on my agenda).
Best,
Philipp
I have a learning algorithm that chooses variables to make decisions on that i'm trying to integrate into my Haxl-based project. The snag here is that the learning alg chooses those variable dynamically, making it hard to write a datasource that doesn't just fetch everything it might look at.
I think I can see a way around this using template haskell and introspecting on the generated decision tree, but if I'm overcomplicating it I'd love to hear about it.
haxl.cabal
claims compatiblity with base == 4.*
, however GHC 7.8.4 falsifies this:
Configuring library for haxl-2.0.1.0..
Preprocessing library for haxl-2.0.1.0..
Building library for haxl-2.0.1.0..
[ 1 of 18] Compiling Haxl.Core.Util ( Haxl/Core/Util.hs, /tmp/matrix-worker/1535645265/dist-newstyle/build/x86_64-linux/ghc-7.6.3/haxl-2.0.1.0/build/Haxl/Core/Util.o )
[ 2 of 18] Compiling Haxl.Core.DataCache ( Haxl/Core/DataCache.hs, /tmp/matrix-worker/1535645265/dist-newstyle/build/x86_64-linux/ghc-7.6.3/haxl-2.0.1.0/build/Haxl/Core/DataCache.o )
[ 3 of 18] Compiling Haxl.Core.StateStore ( Haxl/Core/StateStore.hs, /tmp/matrix-worker/1535645265/dist-newstyle/build/x86_64-linux/ghc-7.6.3/haxl-2.0.1.0/build/Haxl/Core/StateStore.o )
Haxl/Core/StateStore.hs:44:19:
Not in scope: type constructor or class `Proxy'
Haxl/Core/StateStore.hs:45:18: Not in scope: `typeRep'
Haxl/Core/StateStore.hs:61:41:
Not in scope: data constructor `Proxy'
Haxl/Core/StateStore.hs:61:50:
Not in scope: type constructor or class `Proxy'
Haxl/Core/StateStore.hs:69:24:
Not in scope: data constructor `Proxy'
Haxl/Core/StateStore.hs:69:33:
Not in scope: type constructor or class `Proxy'
Haxl/Core/StateStore.hs:75:22:
Not in scope: data constructor `Proxy'
Haxl/Core/StateStore.hs:75:31:
Not in scope: type constructor or class `Proxy'
<<ghc: 1030926808 bytes, 1905 GCs, 15138471/60948896 avg/max bytes residency (8 samples), 118M in use, 0.00 INIT (0.00 elapsed), 0.58 MUT (0.69 elapsed), 0.59 GC (0.59 elapsed) :ghc>>
I've already revised the affected releases on Hackage,
so there's no immediate need for action on your part; however please take this into account for future releases of haxl
.
If Haxl is implemented to support streaming throughout the tree, from each leaf up to the root. With this feature, Haxl will (?) be usable as a framework to perform operations on a data which does not necessary fit into the memory.
Hi there,
I'm a researcher studying software evolution. As part of my current research, I'm studying the implications of open-sourcing a proprietary software, for instance, if the project succeed in attracting newcomers. However, I observed that some projects, like Haxl, deleted their software history.
Knowing that software history is indispensable for developers (e.g., developers need to refer to history several times a day), I would like to ask Haxl developers the following four brief questions:
Thanks in advance for your collaboration,
Gustavo Pinto, PhD
http://www.gustavopinto.org
while building with ghc-8.6.1 we get the following warnings with --allow-newer
WARNING: Ignoring out of range dependency (allow-newer enabled): containers-0.6.0.1. haxl requires: ==0.5.*
WARNING: Ignoring out of range dependency (allow-newer enabled): stm-2.5.0.0. haxl requires: ==2.4.*
The master
branch has lifted bounds. Please make a revision / new release to Hackage. Thanks.
Just a reminder :D
I'm not sure if this is the right place to report this, but with the new Haxl release there are a couple of broken links in hackage:
I am happy to raise a PR if needed
Is there any obstacle preventing addition of MoandMask
instance for GenHaxl u w
? It already has MonadThrow
and MonadCatch
instances.
My usecase for this is that I would like to dump caches to file at the end of GenHaxl
computation, to be able to reproduce any problem that ocurred. I would really like to use finally
there, but it requires MonadMask
instance.
Pardon my ignorance if there's an obvious reason not to do this, I'm really new to functional programming, but I was playing around with porting Haxl to different environments (in particular, Purescript to try to ease in the nodejs community to the idea), and I realized that it's pretty straightforward to implement a <|>
operator. This implementation is Purescript so I'm not sure how well it translates to real Haskell
https://github.com/jqyu/purescript-rad/blob/master/src/Rad/Core/Monad.purs#L236-L243
I don't know much about the Haskell runtime but I suppose the fact that it's a recursive implementation may make memory bounds hard to predict, but this allows for a graceful way of recovering from errors, as well as straightforward implementations of retried requests (i.e., chaining a fixed number of <|>
calls to catch failures)
In particular, I'm working on a GraphQL implementation using Haxl and it would be really useful for me to be able to catch errors and return a null result for a branch of a query so I figured other people might be able to benefit from this as well.
The latest version on hackage still throws errors for GHC 9, the fix is already in main. Now with the aeson PR merged, could you release a new version?
Thanks
cabal install
failed until I upped filepath
to 1.4.* and time
to 1.5.*
I'm on GHC 7.10.1.
$ cabal install
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: haxl-0.1.0.0 (user goal)
trying: vector-0.10.12.3/installed-cbb... (dependency of haxl-0.1.0.0)
next goal: time (dependency of haxl-0.1.0.0)
rejecting: time-1.5.0.1/installed-f60..., 1.5.0.1, 1.5 (conflict: haxl =>
time==1.4.*)
rejecting: time-1.4.2, 1.4.1, 1.4.0.2, 1.4.0.1, 1.4 (conflict: vector =>
deepseq==1.4.1.1/installed-936..., time => deepseq>=1.1 && <1.4)
rejecting: time-1.3, 1.2.0.5, 1.2.0.4, 1.2.0.3, 1.2.0.2, 1.2.0.1, 1.2, 1.1.4,
1.1.3, 1.1.2.4, 1.1.2.3, 1.1.2.2, 1.1.2.1, 1.1.2.0, 1.0 (conflict: haxl =>
time==1.4.*)
Dependency tree exhaustively searched.
I'm trying to get Haxl running against a real database from the sql example.
Is this the only function you need to implement?
sql :: SQLResult a => String -> IO a
Browsing this example it seems the user of the library keeps track of the DB connection pool, not Haxl?
The Cabal file declares this code to be licensed under the BSD-3-Clause license, but that is actually not the case. The code is licensed under BSD3 plus Facebook's "patent termination clause", which is an entirely different thing.
Not really an issue, just a question - how do you handle logging? I can see ifTrace in there, but it's going to return something in GenHaxl - is it legit to hook that up to unsafeLiftIO and use something like monad-logger?
As seen on the Stackage build server:
Preprocessing test suite 'test' for haxl-0.4.0.0...
[1 of 1] Compiling Main ( tests/TestMain.hs, dist/build/test/test-tmp/Main.o )
tests/TestMain.hs:6:1: error:
Failed to load interface for ‘AllTests’
Use -v to see a list of the files searched for.
On most builds The collectsdata
test passes fine, but occasionally it fails with no error output.
Hi!
While experimenting with Haxl for our MSc thesis we ran into a weird memory issue that we can only reproduce with a combination of Haxl and stack-run
. We cannot reproduce with only Haxl (without using stack-run
), or purely with only stack-run
(no Haxl involved), but are ot sure exactly where the problem lies.
A strict foldl'
eats lots of memory depending on whether there is a type annotation for the result or not. See this gist for a minimal code example. The issue persists over different machines.
dumpCacheAsHaskell
is a neat thing. Unfortunately this relies on the fact that Show
emits valid code. Not always are the instances under control of the developer. Just recently I tried to dumpCacheAsHaskell
with types from the time
library which resulted in invalid Haskell code being generated.
Could we introduce a new type class, maybe with a default implementation for types that are showable?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.