chadaustin / buffer-builder Goto Github PK
View Code? Open in Web Editor NEWHaskell library for efficiently building up buffers
License: BSD 3-Clause "New" or "Revised" License
Haskell library for efficiently building up buffers
License: BSD 3-Clause "New" or "Revised" License
Is it trivial to adapt deriveJson
from Aeson
to automatically derive ToJson
instances? I looked into using buffer builder for a project but got stopped in tracks because I use automatic derivation everywhere (instead of hand-coding it tediously). An example derivation is like this through Data.Aeson.TH
:
deriveJSON defaultOptions{fieldLabelModifier = Prelude.drop 5,constructorTagModifier = Prelude.map toLower,omitNothingFields = True} ''ARecordType
It seems that in order to use encodeJson
, ToJson
instance must be provided which must be hand-coded for now. If it can somehow be derived from Data.Aeson.ToJSON
, then it will make it very useful!
Unless I missed something about how to adapt an existing Aeson ToJSON
to ToJson
?
On naive implementations of realloc(), the current growth implementation copies (lg N) times. If we chunked into a linked list, we'd have the option to produce a Lazy ByteString, avoiding a copy, or we could copy once into a Strict ByteString.
I can't imagine a situation where this matters yet, but in case it comes up someday:
Plan:
Have an internal 'dead' bit in BufferWriter
bw_new return 0 on failure, checked on Haskell side
bw_append functions don't fail, they just mark as dead on allocation failure
bw_get_and_trim returns failure code, Haskell throws memory error if dead
I've got some code where the buffered value needs to exceed 1MB before being flushed over the network, and I'm currently having to do my own bookkeeping of the size. I'm pretty sure that the builder tracks its utilization under the hood, so would it be realistic to make that data publicly accessible?
The haskell-perf team has a list of benchmarks of different bytestring builders. buffer-builder
should get added there:
https://github.com/haskell-perf/strict-bytestring-builders
The newly added Semigroup
instances are currently not compatible with GHC <= 7.10.
https://matrix.hackage.haskell.org/package/buffer-builder
Would it be possible to fix the lower bounds for buffer-builder-0.2.4.5
on Hackage so that dependency solvers can catch it?
Sorry that title isn't very clear. My use case is described here. Essentially I need to be able to do something like:
writeBuilderOnto :: Builder -> ByteArrayThingIMightHaveToGrow -> IO ()
bytestring
's Builder
almost gets me there with the internal BufferWriter
, but unfortunately we can determine the number of bytes that will be written ahead of time from just the Builder
. This is a hard requirement for me.
buffer-builder
looks like an attractive place for this functionality to live because I think nearly all the instances you provide have O(1) length already, and it looks like your library is super fast to boot. I think only lazy bytestrings and the String
case would need to force the spine to get an O(n) length, which needs to happen anyway.
Does that analysis seem right, and make sense? And would you be open to:
BufferBuilder
such that its allocation area is known before runningBufferWriter
-like function that I can run on a pre-allocated bufferLet me know what you think, and I may hack on this in the next couple months if it's the sort of thing you'd be interested in having here.
It appears the header file branchlut.h
is missing: http://hydra.cryp.to/build/577987/nixlog/2/raw?
This fails to build in the absence of libstdc++
. Looking at the source code it doesn't look like the c-bits actually use any C++ functionality.
buffer-builder > /tmp/nix-shell.6TvZya/stack-cb8b4477c445eb72/buffer-builder-0.2.4.8/src/Data/BufferBuilder.hs:354:9: error: [GHC-76037]
buffer-builder > Not in scope: data constructor ‘ByteArray’
buffer-builder > Suggested fix:
buffer-builder > Add ‘ByteArray’ to the import list in the import of
buffer-builder > ‘Data.Text.Array’ (at src/Data/BufferBuilder.hs:73:1-35).
buffer-builder > |
buffer-builder > 354 | let ByteArray byteArray = arr
buffer-builder > | ^^^^^^^^^
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.