Comments (5)
One possible answer:
We buffer until we can reliably determine which node type will be used.
For example, incremental GLR parser buffering (see https://github.com/lezer-parser/lezer, https://lezer.codemirror.net/docs/guide/#ambiguity)
from micromark.
For indented code etc, could we stream the underlying data instead of buffering it or is there some gotchas?
from micromark.
It seems to be possible to parse Markdown line-by-line, therefore one line would be the farthest one has to buffer.
A higher-level mechanism may have to “buffer” things though:
- For the indented code: blank line blocks are buffered when in an indented code block, and only output if there is a non-blank indented code line after them (and ignore if not)
- For definitions, I’m working on “MaybeDefinition” and “MaybeDefinitionTitle” blocks, which could be turned into “Paragraph”s if they can’t be valid, or “Definition”s if they are
from micromark.
For indented code, it can already stream chunks of code, and blank lines (both as tokens), but to go to HTML you’d need to buffer continued blank lines and ignore them if they are not followed by more code chunks, and use them if they are!
from micromark.
No need to buffer: we can instead queue tokens (at most a line), and later act on that queue. Specified in CMSM: Overview
from micromark.
Related Issues (20)
- 3.0.8 seems to introduce a module level dependency on document HOT 9
- `index.d.ts` is missing in `micromark-util-encode` published files HOT 3
- HTML with excess whitespace is not parsed correctly HOT 2
- List items wrapped in <p> tags due to trailing space HOT 3
- hard break at the end of a paragraph is not properly parsed HOT 3
- Make `definitions` available to extensions HOT 2
- Custom extensions break in development mode, despite working in production HOT 6
- & in image url will be encode to html entity HOT 2
- Configure collapsing newlines into a single paragraph HOT 3
- TokenizeContext.sliceSerialize throws in sliceChunks if first chunk of token is Code instead of string HOT 20
- Reduce execution time by ~11% with a simple reimplementation of TokenizeContext.now HOT 3
- nested ordered lists not starting with 1. are not detected HOT 4
- `TokenizeContext.sliceSerialize` for `Token.type` of `setextHeading` includes non-heading content from outside the range of [`startLine`, `endLine`] HOT 1
- `micromark-util-symbol` can not be imported by typescript HOT 9
- Strings ending with `\n-` are compiled into a level 2 heading HOT 3
- Error - [webpack] 'dist': ./node_modules/micromark-util-decode-numeric-character-reference/index.js 23:11 Module parse failed: Identifier directly after number HOT 12
- control character and puntuation cause extra emphasis to appear HOT 3
- Using power-assert causes Webpack builds to fail HOT 13
- ES5 Compatibility HOT 6
- uvu shouldn't be set in dependencies HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from micromark.