ikatyang/tree-sitter-markdown

Markdown grammar for tree-sitter

tree-sitter-markdown

Markdown (CommonMark Spec v0.29-gfm) grammar for tree-sitter

Note: This grammar is based on the assumption that link label matchings will never fail since reference links can come before their reference definitions, which causes it hard to do incremental parsing without this assumption.

Changelog

Install

npm install tree-sitter-markdown tree-sitter

Usage

const Parser = require("tree-sitter");
const Markdown = require("tree-sitter-markdown");

const parser = new Parser();
parser.setLanguage(Markdown);

const sourceCode = `
# foo
-     bar
  baz
`;

const tree = parser.parse(sourceCode);
console.log(tree.rootNode.toString());
// (document
//   (atx_heading
//     (atx_heading_marker)
//     (heading_content
//       (text)))
//   (tight_list
//     (list_item
//       (list_marker)
//       (indented_code_block
//         (text))
//       (paragraph
//         (text)))))

License

MIT © Ika

tree-sitter-markdown's Issues

Wikilinks support?

Hello! I would love to use this parser for a lsp I'm working on (https://github.com/lsp-zettelkasten), but I use a different type of linking (wikilink style):

[[this is a link]]

Is it possible to add this to the current parser?

binding for rust

Hey @ikatyang it would be great if there's a binding for rust. I tried to do it my self using tree-sitter guide but failed to do so many times. Here how tree-sitter-rust does it https://github.com/tree-sitter/tree-sitter-rust/tree/master/bindings/rust.

Thanks

Paragraph longer than 255 characters fails assertion

paste

acdskjnnkljadslknjadcskljnacdsknljcdsnjlkacdslnjkcdnjklcdlsnjkacndljkslnjkacdlnjkcdnljkcdsnjlkcdnjklcdanjlkadcsnljkcdlkjnsackljndsajknlcdsnkjalcdnkjlcdcdaknjlklnjdcaskljncadskjnlcdasjnkcdasknjcadkjsnlacdskjnlcdajsknldcasjlknacdsjnlkcadsjlnkjnlkcdasaacdaaaa

into https://ikatyang.github.io/tree-sitter-markdown/ . The tree stops updating until a character is removed.

Assertion fails here:

tree-sitter-markdown/src/tree_sitter_markdown/inline_delimiter.cc

Line 17 in 9ff8d34

assert(len_ <= 0b11111111);

I was able to work around this with pfgithub@c9d9711 , but that doesn't seem like a good solution

Assertion Errors for some markdown files

Some markdown files such as
failure-1.md and
failure-2-minimal.md cause the tree-sitter-markdown parser crash with a nl_dlm.sym() == SYM_EXT_AUT_LNK_BGN || inl_dlm.sym() == SYM_EXT_AUT_LNK_CTN” assertion error.

emacs: src/./tree_sitter_markdown/inline_delimiter.cc:240: void tree_sitter_markdown::InlineDelimiterList::transfer_to(tree_sitter_markdown::MinimizedInlineDelimiterList&): Assertiva “inl_dlm.sym() == SYM_EXT_AUT_LNK_BGN || inl_dlm.sym() == SYM_EXT_AUT_LNK_CTN” falhou.
Fatal error 6: Aborted
Backtrace:
emacs[0x52d3ab]
emacs[0x4222ff]
emacs[0x422801]
emacs[0x52ba08]
emacs[0x52bae9]
/lib64/libc.so.6(+0x3db70)[0x7fc622290b70]
/lib64/libc.so.6(+0x8e844)[0x7fc6222e1844]
/lib64/libc.so.6(gsignal+0x1e)[0x7fc622290abe]
/lib64/libc.so.6(abort+0xdf)[0x7fc62227987f]
/lib64/libc.so.6(+0x2679b)[0x7fc62227979b]
/lib64/libc.so.6(+0x36147)[0x7fc622289147]
/home/kassick/.emacs.d/elpa/29.1/develop/tree-sitter-langs-20230910.547/bin/markdown.so(_ZN20tree_sitter_markdown19InlineDelimiterList11transfer_toERNS_28MinimizedInlineDelimiterListE+0x16e)[0x7fc5b06e0c9e]
/home/kassick/.emacs.d/elpa/29.1/develop/tree-sitter-langs-20230910.547/bin/markdown.so(tree_sitter_markdown_external_scanner_scan+0x841)[0x7fc5b071eb51]
/home/kassick/.emacs.d/elpa/29.1/develop/tsc-20230916.113114/tsc-dyn.so(+0x14da92)[0x7fc5c614da92]
/home/kassick/.emacs.d/elpa/29.1/develop/tsc-20230916.113114/tsc-dyn.so(+0x12d0d1)[0x7fc5c612d0d1]
/home/kassick/.emacs.d/elpa/29.1/develop/tsc-20230916.113114/tsc-dyn.so(+0x128a6a)[0x7fc5c6128a6a]
/home/kassick/.emacs.d/elpa/29.1/develop/tsc-20230916.113114/tsc-dyn.so(+0x12d526)[0x7fc5c612d526]
emacs[0x5d2172]
emacs[0x5e148e]
emacs[0x5994e6]
emacs[0x5995a9]
emacs[0x59841c]
emacs[0x598629]
emacs[0x5994e6]
/usr/local/bin/../lib/emacs/29.1.50/native-lisp/29.1.50-a92a42e2/preloaded/subr-13adf6a6-8f3898dc.eln(F72756e2d6d6f64652d686f6f6b73_run_mode_hooks_0+0x389)[0x7fc61f65fd09]
emacs[0x5994e6]
/usr/local/bin/../lib/emacs/29.1.50/native-lisp/29.1.50-a92a42e2/preloaded/elisp-mode-90dbfe40-2f48c113.eln(F656d6163732d6c6973702d6d6f6465_emacs_lisp_mode_0+0x62c)[0x7fc61f246a7c]
emacs[0x5e1450]
emacs[0x5e3c8b]
emacs[0x59d020]
emacs[0x59d5d8]
emacs[0x59cf07]
emacs[0x59f4d6]
/home/kassick/.emacs.d/eln-cache/29.1.50-a92a42e2/yasnippet-1427ec8a-8df7bbfb.eln(top_level_run+0xaf58)[0x7fc5b080c6c3]
emacs[0x5ec8e8]
emacs[0x5ecd73]
emacs[0x5cb863]
emacs[0x5cc0b8]
emacs[0x5991c1]
emacs[0x59b5a2]
emacs[0x59c975]
...

Unable to install

Following the readme install command, I got prompted the following error log:

0 verbose cli [
0 verbose cli   '/home/joe/.nvm/versions/node/v16.13.0/bin/node',
0 verbose cli   '/home/joe/.nvm/versions/node/v16.13.0/bin/npm',
0 verbose cli   'install',
0 verbose cli   'tree-sitter-markdown',
0 verbose cli   'tree-sitter'
0 verbose cli ]
1 info using [email protected]
2 info using [email protected]
3 timing npm:load:whichnode Completed in 0ms
4 timing config:load:defaults Completed in 2ms
5 timing config:load:file:/home/joe/.nvm/versions/node/v16.13.0/lib/node_modules/npm/npmrc Completed in 1ms
6 timing config:load:builtin Completed in 1ms
7 timing config:load:cli Completed in 2ms
8 timing config:load:env Completed in 1ms
9 timing config:load:file:/home/joe/projet/todo_txt_parser/.npmrc Completed in 0ms
10 timing config:load:project Completed in 3ms
11 timing config:load:file:/home/joe/.npmrc Completed in 1ms
12 timing config:load:user Completed in 1ms
13 timing config:load:file:/home/joe/.nvm/versions/node/v16.13.0/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:validate Completed in 0ms
16 timing config:load:credentials Completed in 2ms
17 timing config:load:setEnvs Completed in 1ms
18 timing config:load Completed in 13ms
19 timing npm:load:configload Completed in 13ms
20 timing npm:load:setTitle Completed in 0ms
21 timing npm:load:setupLog Completed in 1ms
22 timing config:load:flatten Completed in 4ms
23 timing npm:load:cleanupLog Completed in 2ms
24 timing npm:load:configScope Completed in 0ms
25 timing npm:load:projectScope Completed in 0ms
26 timing npm:load Completed in 22ms
27 timing arborist:ctor Completed in 1ms
28 timing arborist:ctor Completed in 0ms
29 timing idealTree:init Completed in 13ms
30 timing idealTree:userRequests Completed in 3ms
31 silly idealTree buildDeps
32 silly fetch manifest tree-sitter-markdown@*
33 http fetch GET 200 https://registry.npmjs.org/tree-sitter-markdown 908ms (cache revalidated)
34 silly fetch manifest tree-sitter@*
35 http fetch GET 200 https://registry.npmjs.org/tree-sitter 722ms (cache revalidated)
36 silly placeDep ROOT [email protected] OK for:  want: *
37 silly placeDep ROOT [email protected] OK for:  want: *
38 silly fetch manifest nan@^2.14.0
39 silly fetch manifest prebuild-install@^6.0.1
40 http fetch GET 200 https://registry.npmjs.org/nan 65ms (cache revalidated)
41 http fetch GET 200 https://registry.npmjs.org/prebuild-install 133ms (cache revalidated)
42 timing idealTree:#root Completed in 1803ms
43 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.14.0
44 silly placeDep ROOT [email protected] OK for: [email protected] want: ^6.0.1
45 silly fetch manifest detect-libc@^1.0.3
46 silly fetch manifest expand-template@^2.0.3
47 silly fetch manifest [email protected]
48 silly fetch manifest minimist@^1.2.3
49 silly fetch manifest mkdirp-classic@^0.5.3
50 silly fetch manifest napi-build-utils@^1.0.1
51 silly fetch manifest node-abi@^2.21.0
52 silly fetch manifest npmlog@^4.0.1
53 silly fetch manifest pump@^3.0.0
54 silly fetch manifest rc@^1.2.7
55 silly fetch manifest simple-get@^3.0.3
56 silly fetch manifest tar-fs@^2.0.0
57 silly fetch manifest tunnel-agent@^0.6.0
58 http fetch GET 200 https://registry.npmjs.org/detect-libc 92ms (cache revalidated)
59 http fetch GET 200 https://registry.npmjs.org/expand-template 100ms (cache revalidated)
60 http fetch GET 200 https://registry.npmjs.org/github-from-package 158ms (cache revalidated)
61 http fetch GET 200 https://registry.npmjs.org/mkdirp-classic 157ms (cache revalidated)
62 http fetch GET 200 https://registry.npmjs.org/minimist 175ms (cache revalidated)
63 http fetch GET 200 https://registry.npmjs.org/napi-build-utils 173ms (cache revalidated)
64 http fetch GET 200 https://registry.npmjs.org/tunnel-agent 167ms (cache revalidated)
65 http fetch GET 200 https://registry.npmjs.org/npmlog 173ms (cache revalidated)
66 http fetch GET 200 https://registry.npmjs.org/rc 179ms (cache revalidated)
67 http fetch GET 200 https://registry.npmjs.org/simple-get 179ms (cache revalidated)
68 http fetch GET 200 https://registry.npmjs.org/tar-fs 192ms (cache revalidated)
69 http fetch GET 200 https://registry.npmjs.org/pump 197ms (cache revalidated)
70 http fetch GET 200 https://registry.npmjs.org/node-abi 809ms (cache revalidated)
71 timing idealTree:node_modules/tree-sitter Completed in 835ms
72 timing idealTree:node_modules/nan Completed in 0ms
73 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.3
74 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.0.3
75 silly placeDep ROOT [email protected] OK for: [email protected] want: 0.0.0
76 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.2.3
77 silly placeDep ROOT [email protected] OK for: [email protected] want: ^0.5.3
78 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.1
79 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.21.0
80 silly placeDep ROOT [email protected] OK for: [email protected] want: ^4.0.1
81 silly placeDep ROOT [email protected] OK for: [email protected] want: ^3.0.0
82 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.2.7
83 silly placeDep ROOT [email protected] OK for: [email protected] want: ^3.0.3
84 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.0.0
85 silly placeDep ROOT [email protected] OK for: [email protected] want: ^0.6.0
86 silly fetch manifest semver@^5.4.1
87 silly fetch manifest are-we-there-yet@~1.1.2
88 silly fetch manifest console-control-strings@~1.1.0
89 silly fetch manifest gauge@~2.7.3
90 silly fetch manifest set-blocking@~2.0.0
91 silly fetch manifest end-of-stream@^1.1.0
92 silly fetch manifest once@^1.3.1
93 silly fetch manifest deep-extend@^0.6.0
94 silly fetch manifest ini@~1.3.0
95 silly fetch manifest strip-json-comments@~2.0.1
96 silly fetch manifest decompress-response@^4.2.0
97 silly fetch manifest simple-concat@^1.0.0
98 silly fetch manifest chownr@^1.1.1
99 silly fetch manifest tar-stream@^2.1.4
100 silly fetch manifest safe-buffer@^5.0.1
101 http fetch GET 200 https://registry.npmjs.org/semver 76ms (cache revalidated)
102 http fetch GET 200 https://registry.npmjs.org/gauge 79ms (cache revalidated)
103 http fetch GET 200 https://registry.npmjs.org/set-blocking 82ms (cache revalidated)
104 http fetch GET 200 https://registry.npmjs.org/ini 106ms (cache revalidated)
105 http fetch GET 200 https://registry.npmjs.org/end-of-stream 112ms (cache revalidated)
106 http fetch GET 200 https://registry.npmjs.org/chownr 100ms (cache revalidated)
107 http fetch GET 200 https://registry.npmjs.org/decompress-response 107ms (cache revalidated)
108 http fetch GET 200 https://registry.npmjs.org/simple-concat 109ms (cache revalidated)
109 http fetch GET 200 https://registry.npmjs.org/are-we-there-yet 122ms (cache revalidated)
110 http fetch GET 200 https://registry.npmjs.org/deep-extend 117ms (cache revalidated)
111 http fetch GET 200 https://registry.npmjs.org/strip-json-comments 120ms (cache revalidated)
112 http fetch GET 200 https://registry.npmjs.org/tar-stream 113ms (cache revalidated)
113 http fetch GET 200 https://registry.npmjs.org/console-control-strings 130ms (cache revalidated)
114 http fetch GET 200 https://registry.npmjs.org/once 140ms (cache revalidated)
115 http fetch GET 200 https://registry.npmjs.org/safe-buffer 144ms (cache revalidated)
116 timing idealTree:node_modules/prebuild-install Completed in 222ms
117 timing idealTree:node_modules/detect-libc Completed in 0ms
118 timing idealTree:node_modules/expand-template Completed in 0ms
119 timing idealTree:node_modules/github-from-package Completed in 0ms
120 timing idealTree:node_modules/minimist Completed in 0ms
121 timing idealTree:node_modules/mkdirp-classic Completed in 0ms
122 timing idealTree:node_modules/napi-build-utils Completed in 0ms
123 silly placeDep ROOT [email protected] OK for: [email protected] want: ^5.4.1
124 timing idealTree:node_modules/node-abi Completed in 2ms
125 silly placeDep ROOT [email protected] OK for: [email protected] want: ~1.1.2
126 silly placeDep ROOT [email protected] OK for: [email protected] want: ~1.1.0
127 silly placeDep ROOT [email protected] OK for: [email protected] want: ~2.7.3
128 silly placeDep ROOT [email protected] OK for: [email protected] want: ~2.0.0
129 silly fetch manifest delegates@^1.0.0
130 silly fetch manifest readable-stream@^2.0.6
131 silly fetch manifest aproba@^1.0.3
132 silly fetch manifest has-unicode@^2.0.0
133 silly fetch manifest object-assign@^4.1.0
134 silly fetch manifest signal-exit@^3.0.0
135 silly fetch manifest string-width@^1.0.1
136 silly fetch manifest strip-ansi@^3.0.1
137 silly fetch manifest wide-align@^1.1.0
138 http fetch GET 200 https://registry.npmjs.org/delegates 68ms (cache revalidated)
139 http fetch GET 200 https://registry.npmjs.org/readable-stream 69ms (cache revalidated)
140 http fetch GET 200 https://registry.npmjs.org/signal-exit 72ms (cache revalidated)
141 http fetch GET 200 https://registry.npmjs.org/wide-align 72ms (cache revalidated)
142 http fetch GET 200 https://registry.npmjs.org/strip-ansi 74ms (cache revalidated)
143 http fetch GET 200 https://registry.npmjs.org/has-unicode 88ms (cache revalidated)
144 http fetch GET 200 https://registry.npmjs.org/object-assign 91ms (cache revalidated)
145 http fetch GET 200 https://registry.npmjs.org/string-width 101ms (cache revalidated)
146 http fetch GET 200 https://registry.npmjs.org/aproba 112ms (cache revalidated)
147 timing idealTree:node_modules/npmlog Completed in 125ms
148 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.0
149 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.0.6
150 silly fetch manifest core-util-is@~1.0.0
151 silly fetch manifest inherits@~2.0.3
152 silly fetch manifest isarray@~1.0.0
153 silly fetch manifest process-nextick-args@~2.0.0
154 silly fetch manifest safe-buffer@~5.1.1
155 silly fetch manifest string_decoder@~1.1.1
156 silly fetch manifest util-deprecate@~1.0.1
157 http fetch GET 200 https://registry.npmjs.org/core-util-is 58ms (cache revalidated)
158 http fetch GET 200 https://registry.npmjs.org/inherits 68ms (cache revalidated)
159 http fetch GET 200 https://registry.npmjs.org/util-deprecate 66ms (cache revalidated)
160 http fetch GET 200 https://registry.npmjs.org/string_decoder 68ms (cache revalidated)
161 http fetch GET 200 https://registry.npmjs.org/isarray 87ms (cache revalidated)
162 http fetch GET 200 https://registry.npmjs.org/process-nextick-args 101ms (cache revalidated)
163 timing idealTree:node_modules/are-we-there-yet Completed in 109ms
164 timing idealTree:node_modules/console-control-strings Completed in 0ms
165 timing idealTree:node_modules/delegates Completed in 0ms
166 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.3
167 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.0.0
168 silly placeDep ROOT [email protected] OK for: [email protected] want: ^4.1.0
169 silly placeDep ROOT [email protected] OK for: [email protected] want: ^3.0.0
170 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.1
171 silly placeDep ROOT [email protected] OK for: [email protected] want: ^3.0.1
172 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.1.0
173 silly fetch manifest code-point-at@^1.0.0
174 silly fetch manifest is-fullwidth-code-point@^1.0.0
175 silly fetch manifest ansi-regex@^2.0.0
176 http fetch GET 200 https://registry.npmjs.org/ansi-regex 61ms (cache revalidated)
177 http fetch GET 200 https://registry.npmjs.org/is-fullwidth-code-point 66ms (cache revalidated)
178 http fetch GET 200 https://registry.npmjs.org/code-point-at 93ms (cache revalidated)
179 timing idealTree:node_modules/gauge Completed in 111ms
180 timing idealTree:node_modules/aproba Completed in 0ms
181 timing idealTree:node_modules/has-unicode Completed in 0ms
182 timing idealTree:node_modules/object-assign Completed in 0ms
183 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.1.0
184 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.3.1
185 silly fetch manifest wrappy@1
186 http fetch GET 200 https://registry.npmjs.org/wrappy 63ms (cache revalidated)
187 timing idealTree:node_modules/pump Completed in 75ms
188 timing idealTree:node_modules/end-of-stream Completed in 0ms
189 silly placeDep ROOT [email protected] OK for: [email protected] want: 1
190 timing idealTree:node_modules/once Completed in 3ms
191 silly placeDep ROOT [email protected] OK for: [email protected] want: ^0.6.0
192 silly placeDep ROOT [email protected] OK for: [email protected] want: ~1.3.0
193 silly placeDep ROOT [email protected] OK for: [email protected] want: ~2.0.1
194 timing idealTree:node_modules/rc Completed in 9ms
195 timing idealTree:node_modules/deep-extend Completed in 0ms
196 timing idealTree:node_modules/ini Completed in 0ms
197 silly placeDep ROOT [email protected] OK for: [email protected] want: ~1.0.0
198 silly placeDep ROOT [email protected] OK for: [email protected] want: ~2.0.3
199 silly placeDep ROOT [email protected] OK for: [email protected] want: ~1.0.0
200 silly placeDep ROOT [email protected] OK for: [email protected] want: ~2.0.0
201 silly placeDep ROOT [email protected] OK for: [email protected] want: ~5.1.1
202 silly placeDep ROOT [email protected] OK for: [email protected] want: ~1.1.1
203 silly placeDep ROOT [email protected] OK for: [email protected] want: ~1.0.1
204 timing idealTree:node_modules/readable-stream Completed in 18ms
205 timing idealTree:node_modules/core-util-is Completed in 0ms
206 timing idealTree:node_modules/inherits Completed in 0ms
207 timing idealTree:node_modules/isarray Completed in 0ms
208 timing idealTree:node_modules/process-nextick-args Completed in 0ms
209 timing idealTree:node_modules/safe-buffer Completed in 0ms
210 timing idealTree:node_modules/semver Completed in 0ms
211 timing idealTree:node_modules/set-blocking Completed in 0ms
212 timing idealTree:node_modules/signal-exit Completed in 1ms
213 silly placeDep ROOT [email protected] OK for: [email protected] want: ^4.2.0
214 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.0
215 silly fetch manifest mimic-response@^2.0.0
216 http fetch GET 200 https://registry.npmjs.org/mimic-response 47ms (cache revalidated)
217 timing idealTree:node_modules/simple-get Completed in 53ms
218 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.0.0
219 timing idealTree:node_modules/decompress-response Completed in 2ms
220 timing idealTree:node_modules/mimic-response Completed in 0ms
221 timing idealTree:node_modules/simple-concat Completed in 0ms
222 timing idealTree:node_modules/string_decoder Completed in 0ms
223 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.0
224 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.0
225 silly fetch manifest number-is-nan@^1.0.0
226 http fetch GET 200 https://registry.npmjs.org/number-is-nan 49ms (cache revalidated)
227 timing idealTree:node_modules/string-width Completed in 55ms
228 timing idealTree:node_modules/code-point-at Completed in 0ms
229 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.0
230 timing idealTree:node_modules/is-fullwidth-code-point Completed in 3ms
231 timing idealTree:node_modules/number-is-nan Completed in 0ms
232 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.0.0
233 timing idealTree:node_modules/strip-ansi Completed in 3ms
234 timing idealTree:node_modules/ansi-regex Completed in 0ms
235 timing idealTree:node_modules/strip-json-comments Completed in 0ms
236 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.1.1
237 silly placeDep ROOT [email protected] OK for: [email protected] want: ^2.1.4
238 silly fetch manifest bl@^4.0.3
239 silly fetch manifest fs-constants@^1.0.0
240 silly fetch manifest readable-stream@^3.1.1
241 http fetch GET 200 https://registry.npmjs.org/bl 63ms (cache revalidated)
242 http fetch GET 200 https://registry.npmjs.org/fs-constants 67ms (cache revalidated)
243 timing idealTree:node_modules/tar-fs Completed in 79ms
244 timing idealTree:node_modules/chownr Completed in 0ms
245 silly placeDep ROOT [email protected] OK for: [email protected] want: ^4.0.3
246 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.0.0
247 silly placeDep node_modules/tar-stream [email protected] OK for: [email protected] want: ^3.1.1
248 silly fetch manifest buffer@^5.5.0
249 silly fetch manifest readable-stream@^3.4.0
250 http fetch GET 200 https://registry.npmjs.org/buffer 57ms (cache revalidated)
251 timing idealTree:node_modules/tar-stream Completed in 69ms
252 silly placeDep ROOT [email protected] OK for: [email protected] want: ^5.5.0
253 silly placeDep node_modules/bl [email protected] OK for: [email protected] want: ^3.4.0
254 silly fetch manifest base64-js@^1.3.1
255 silly fetch manifest ieee754@^1.1.13
256 http fetch GET 200 https://registry.npmjs.org/base64-js 55ms (cache revalidated)
257 http fetch GET 200 https://registry.npmjs.org/ieee754 59ms (cache revalidated)
258 timing idealTree:node_modules/bl Completed in 64ms
259 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.3.1
260 silly placeDep ROOT [email protected] OK for: [email protected] want: ^1.1.13
261 timing idealTree:node_modules/buffer Completed in 5ms
262 timing idealTree:node_modules/base64-js Completed in 0ms
263 timing idealTree:node_modules/fs-constants Completed in 0ms
264 timing idealTree:node_modules/ieee754 Completed in 0ms
265 timing idealTree:node_modules/tree-sitter-markdown Completed in 0ms
266 timing idealTree:node_modules/tunnel-agent Completed in 0ms
267 timing idealTree:node_modules/util-deprecate Completed in 0ms
268 timing idealTree:node_modules/wide-align Completed in 0ms
269 timing idealTree:node_modules/wrappy Completed in 0ms
270 timing idealTree:node_modules/bl/node_modules/readable-stream Completed in 0ms
271 timing idealTree:node_modules/tar-stream/node_modules/readable-stream Completed in 0ms
272 timing idealTree:buildDeps Completed in 3656ms
273 timing idealTree:fixDepFlags Completed in 4ms
274 timing idealTree Completed in 3679ms
275 timing reify:loadTrees Completed in 3680ms
276 timing reify:diffTrees Completed in 6ms
277 silly reify moves {}
278 timing reify:retireShallow Completed in 0ms
279 timing reify:createSparse Completed in 24ms
280 timing reify:loadBundles Completed in 0ms
281 silly audit bulk request {
281 silly audit   'tree-sitter': [ '0.20.1' ],
281 silly audit   'tree-sitter-markdown': [ '0.7.1' ],
281 silly audit   nan: [ '2.17.0' ],
281 silly audit   'prebuild-install': [ '6.1.4' ],
281 silly audit   'detect-libc': [ '1.0.3' ],
281 silly audit   'expand-template': [ '2.0.3' ],
281 silly audit   'github-from-package': [ '0.0.0' ],
281 silly audit   minimist: [ '1.2.7' ],
281 silly audit   'mkdirp-classic': [ '0.5.3' ],
281 silly audit   'napi-build-utils': [ '1.0.2' ],
281 silly audit   'node-abi': [ '2.30.1' ],
281 silly audit   npmlog: [ '4.1.2' ],
281 silly audit   pump: [ '3.0.0' ],
281 silly audit   rc: [ '1.2.8' ],
281 silly audit   'simple-get': [ '3.1.1' ],
281 silly audit   'tar-fs': [ '2.1.1' ],
281 silly audit   'tunnel-agent': [ '0.6.0' ],
281 silly audit   semver: [ '5.7.1' ],
281 silly audit   'are-we-there-yet': [ '1.1.7' ],
281 silly audit   'console-control-strings': [ '1.1.0' ],
281 silly audit   gauge: [ '2.7.4' ],
281 silly audit   'set-blocking': [ '2.0.0' ],
281 silly audit   delegates: [ '1.0.0' ],
281 silly audit   'readable-stream': [ '2.3.7', '3.6.0' ],
281 silly audit   aproba: [ '1.2.0' ],
281 silly audit   'has-unicode': [ '2.0.1' ],
281 silly audit   'object-assign': [ '4.1.1' ],
281 silly audit   'signal-exit': [ '3.0.7' ],
281 silly audit   'string-width': [ '1.0.2' ],
281 silly audit   'strip-ansi': [ '3.0.1' ],
281 silly audit   'wide-align': [ '1.1.5' ],
281 silly audit   'end-of-stream': [ '1.4.4' ],
281 silly audit   once: [ '1.4.0' ],
281 silly audit   wrappy: [ '1.0.2' ],
281 silly audit   'deep-extend': [ '0.6.0' ],
281 silly audit   ini: [ '1.3.8' ],
281 silly audit   'strip-json-comments': [ '2.0.1' ],
281 silly audit   'core-util-is': [ '1.0.3' ],
281 silly audit   inherits: [ '2.0.4' ],
281 silly audit   isarray: [ '1.0.0' ],
281 silly audit   'process-nextick-args': [ '2.0.1' ],
281 silly audit   'safe-buffer': [ '5.1.2' ],
281 silly audit   string_decoder: [ '1.1.1' ],
281 silly audit   'util-deprecate': [ '1.0.2' ],
281 silly audit   'decompress-response': [ '4.2.1' ],
281 silly audit   'simple-concat': [ '1.0.1' ],
281 silly audit   'mimic-response': [ '2.1.0' ],
281 silly audit   'code-point-at': [ '1.1.0' ],
281 silly audit   'is-fullwidth-code-point': [ '1.0.0' ],
281 silly audit   'number-is-nan': [ '1.0.1' ],
281 silly audit   'ansi-regex': [ '2.1.1' ],
281 silly audit   chownr: [ '1.1.4' ],
281 silly audit   'tar-stream': [ '2.2.0' ],
281 silly audit   bl: [ '4.1.0' ],
281 silly audit   'fs-constants': [ '1.0.0' ],
281 silly audit   buffer: [ '5.7.1' ],
281 silly audit   'base64-js': [ '1.5.1' ],
281 silly audit   ieee754: [ '1.2.1' ]
281 silly audit }
282 timing reifyNode:node_modules/ansi-regex Completed in 253ms
283 timing reifyNode:node_modules/chownr Completed in 253ms
284 timing reifyNode:node_modules/is-fullwidth-code-point Completed in 254ms
285 timing reifyNode:node_modules/number-is-nan Completed in 254ms
286 timing reifyNode:node_modules/code-point-at Completed in 254ms
287 timing reifyNode:node_modules/strip-json-comments Completed in 253ms
288 timing reifyNode:node_modules/ini Completed in 253ms
289 timing reifyNode:node_modules/wrappy Completed in 253ms
290 timing reifyNode:node_modules/once Completed in 253ms
291 timing reifyNode:node_modules/end-of-stream Completed in 254ms
292 timing reifyNode:node_modules/wide-align Completed in 254ms
293 timing reifyNode:node_modules/strip-ansi Completed in 253ms
294 timing reifyNode:node_modules/string-width Completed in 253ms
295 timing reifyNode:node_modules/process-nextick-args Completed in 255ms
296 timing reifyNode:node_modules/object-assign Completed in 253ms
297 timing reifyNode:node_modules/has-unicode Completed in 253ms
298 timing reifyNode:node_modules/aproba Completed in 253ms
299 timing reifyNode:node_modules/mkdirp-classic Completed in 255ms
300 timing reifyNode:node_modules/ieee754 Completed in 264ms
301 timing reifyNode:node_modules/fs-constants Completed in 263ms
302 timing reifyNode:node_modules/mimic-response Completed in 263ms
303 timing reifyNode:node_modules/decompress-response Completed in 262ms
304 timing reifyNode:node_modules/core-util-is Completed in 262ms
305 timing reifyNode:node_modules/inherits Completed in 262ms
306 timing reifyNode:node_modules/signal-exit Completed in 261ms
307 timing reifyNode:node_modules/set-blocking Completed in 260ms
308 timing reifyNode:node_modules/console-control-strings Completed in 261ms
309 timing reifyNode:node_modules/tunnel-agent Completed in 261ms
310 timing reifyNode:node_modules/simple-get Completed in 261ms
311 timing reifyNode:node_modules/base64-js Completed in 273ms
312 timing reifyNode:node_modules/util-deprecate Completed in 271ms
313 http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 280ms
314 timing auditReport:getReport Completed in 284ms
315 silly audit report {}
316 timing auditReport:init Completed in 0ms
317 timing reify:audit Completed in 286ms
318 timing reifyNode:node_modules/expand-template Completed in 272ms
319 timing reifyNode:node_modules/simple-concat Completed in 277ms
320 timing reifyNode:node_modules/string_decoder Completed in 277ms
321 timing reifyNode:node_modules/safe-buffer Completed in 277ms
322 timing reifyNode:node_modules/pump Completed in 274ms
323 timing reifyNode:node_modules/npmlog Completed in 274ms
324 timing reifyNode:node_modules/napi-build-utils Completed in 274ms
325 timing reifyNode:node_modules/deep-extend Completed in 280ms
326 timing reifyNode:node_modules/isarray Completed in 282ms
327 timing reifyNode:node_modules/delegates Completed in 296ms
328 timing reifyNode:node_modules/are-we-there-yet Completed in 306ms
329 timing reifyNode:node_modules/detect-libc Completed in 305ms
330 timing reifyNode:node_modules/tar-stream Completed in 313ms
331 timing reifyNode:node_modules/semver Completed in 309ms
332 timing reifyNode:node_modules/buffer Completed in 317ms
333 timing reifyNode:node_modules/github-from-package Completed in 314ms
334 timing reifyNode:node_modules/rc Completed in 315ms
335 timing reifyNode:node_modules/bl Completed in 322ms
336 timing reifyNode:node_modules/node-abi Completed in 317ms
337 timing reifyNode:node_modules/prebuild-install Completed in 316ms
338 timing reifyNode:node_modules/gauge Completed in 326ms
339 timing reifyNode:node_modules/minimist Completed in 330ms
340 timing reifyNode:node_modules/readable-stream Completed in 342ms
341 timing reifyNode:node_modules/bl/node_modules/readable-stream Completed in 347ms
342 timing reifyNode:node_modules/tar-stream/node_modules/readable-stream Completed in 346ms
343 timing reifyNode:node_modules/tar-fs Completed in 345ms
344 timing reifyNode:node_modules/nan Completed in 358ms
345 timing reifyNode:node_modules/tree-sitter Completed in 376ms
346 timing reifyNode:node_modules/tree-sitter-markdown Completed in 377ms
347 timing reify:unpack Completed in 385ms
348 timing reify:unretire Completed in 0ms
349 timing build:queue Completed in 5ms
350 timing build:link:node_modules/detect-libc Completed in 5ms
351 timing build:link:node_modules/semver Completed in 4ms
352 timing build:link:node_modules/rc Completed in 4ms
353 timing build:link:node_modules/prebuild-install Completed in 4ms
354 timing build:link Completed in 5ms
355 info run [email protected] install node_modules/tree-sitter prebuild-install || node-gyp rebuild
356 info run [email protected] install node_modules/tree-sitter-markdown node-gyp rebuild
357 info run [email protected] install { code: 1, signal: null }
358 timing reify:rollback:createSparse Completed in 105ms
359 timing reify:rollback:retireShallow Completed in 0ms
360 timing command:install Completed in 6981ms
361 verbose stack Error: command failed
361 verbose stack     at ChildProcess.<anonymous> (/home/joe/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
361 verbose stack     at ChildProcess.emit (node:events:390:28)
361 verbose stack     at maybeClose (node:internal/child_process:1064:16)
361 verbose stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)
362 verbose pkgid [email protected]
363 verbose cwd /home/joe/projet/todo_txt_parser
364 verbose Linux 6.0.2-76060002-generic
365 verbose argv "/home/joe/.nvm/versions/node/v16.13.0/bin/node" "/home/joe/.nvm/versions/node/v16.13.0/bin/npm" "install" "tree-sitter-markdown" "tree-sitter"
366 verbose node v16.13.0
367 verbose npm  v8.1.0
368 error code 1
369 error path /home/joe/projet/todo_txt_parser/node_modules/tree-sitter-markdown
370 error command failed
371 error command sh -c node-gyp rebuild
372 error make: Entering directory '/home/joe/projet/todo_txt_parser/node_modules/tree-sitter-markdown/build'
372 error   CC(target) Release/obj.target/tree_sitter_markdown_binding/src/parser.o
372 error   CXX(target) Release/obj.target/tree_sitter_markdown_binding/bindings/node/binding.o
372 error   CXX(target) Release/obj.target/tree_sitter_markdown_binding/src/scanner.o
372 error make: Leaving directory '/home/joe/projet/todo_txt_parser/node_modules/tree-sitter-markdown/build'
373 error gyp info it worked if it ends with ok
373 error gyp info using [email protected]
373 error gyp info using [email protected] | linux | x64
373 error gyp info find Python using Python version 3.10.6 found at "/usr/bin/python3"
373 error gyp info spawn /usr/bin/python3
373 error gyp info spawn args [
373 error gyp info spawn args   '/home/joe/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
373 error gyp info spawn args   'binding.gyp',
373 error gyp info spawn args   '-f',
373 error gyp info spawn args   'make',
373 error gyp info spawn args   '-I',
373 error gyp info spawn args   '/home/joe/projet/todo_txt_parser/node_modules/tree-sitter-markdown/build/config.gypi',
373 error gyp info spawn args   '-I',
373 error gyp info spawn args   '/home/joe/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
373 error gyp info spawn args   '-I',
373 error gyp info spawn args   '/home/joe/.cache/node-gyp/16.13.0/include/node/common.gypi',
373 error gyp info spawn args   '-Dlibrary=shared_library',
373 error gyp info spawn args   '-Dvisibility=default',
373 error gyp info spawn args   '-Dnode_root_dir=/home/joe/.cache/node-gyp/16.13.0',
373 error gyp info spawn args   '-Dnode_gyp_dir=/home/joe/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp',
373 error gyp info spawn args   '-Dnode_lib_file=/home/joe/.cache/node-gyp/16.13.0/<(target_arch)/node.lib',
373 error gyp info spawn args   '-Dmodule_root_dir=/home/joe/projet/todo_txt_parser/node_modules/tree-sitter-markdown',
373 error gyp info spawn args   '-Dnode_engine=v8',
373 error gyp info spawn args   '--depth=.',
373 error gyp info spawn args   '--no-parallel',
373 error gyp info spawn args   '--generator-output',
373 error gyp info spawn args   'build',
373 error gyp info spawn args   '-Goutput_dir=.'
373 error gyp info spawn args ]
373 error gyp info spawn make
373 error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
373 error In file included from ../src/parser.c:1:
373 error ../src/parser.c: In function ‘ts_lex’:
373 error ../src/tree_sitter/parser.h:136:11: warning: variable ‘lookahead’ set but not used [-Wunused-but-set-variable]
373 error   136 |   int32_t lookahead;            \
373 error       |           ^~~~~~~~~
373 error ../src/parser.c:1756:3: note: in expansion of macro ‘START_LEXER’
373 error  1756 |   START_LEXER();
373 error       |   ^~~~~~~~~~~
373 error In file included from ../bindings/node/binding.cc:2:
373 error /home/joe/.cache/node-gyp/16.13.0/include/node/node.h:821:7: warning: cast between incompatible function types from ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Object>)’ to ‘node::addon_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)’} [-Wcast-function-type]
373 error   821 |       (node::addon_register_func) (regfunc),                          \
373 error       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
373 error /home/joe/.cache/node-gyp/16.13.0/include/node/node.h:855:3: note: in expansion of macro ‘NODE_MODULE_X’
373 error   855 |   NODE_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
373 error       |   ^~~~~~~~~~~~~
373 error ../bindings/node/binding.cc:26:1: note: in expansion of macro ‘NODE_MODULE’
373 error    26 | NODE_MODULE(tree_sitter_markdown_binding, Init)
373 error       | ^~~~~~~~~~~
373 error ../src/./tree_sitter_markdown/block_context.cc: In member function ‘tree_sitter_markdown::ParseState tree_sitter_markdown::BlockContext::pst() const’:
373 error ../src/scanner.cc:9:72: error: exception handling disabled, use ‘-fexceptions’ to enable
373 error     9 | #define TREE_SITTER_MARKDOWN_ASSERT(condition) if (!(condition)) throw 1;
373 error       |                                                                        ^
373 error ../src/./tree_sitter_markdown/block_context.cc:9:40: note: in expansion of macro ‘TREE_SITTER_MARKDOWN_ASSERT’
373 error     9 | ParseState BlockContext::pst() const { TREE_SITTER_MARKDOWN_ASSERT(pst_ != PST_INVALID); return pst_; }
373 error       |                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~
373 error In file included from ../src/scanner.cc:23:
373 error ../src/./tree_sitter_markdown/lexer.cc: In member function ‘void tree_sitter_markdown::Lexer::adv(bool)’:
373 error ../src/./tree_sitter_markdown/lexer.cc:106:37: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<int>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
373 error   106 |     if (cur_idx_ - buf_bgn_idx_ + 1 >= chr_buf_.size()) {
373 error       |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
373 error ../src/./tree_sitter_markdown/lexer.cc: In member function ‘void tree_sitter_markdown::Lexer::jmp_pos(const tree_sitter_markdown::LexedPosition&)’:
373 error ../src/./tree_sitter_markdown/lexer.cc:174:56: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<int>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
373 error   174 |   TREE_SITTER_MARKDOWN_ASSERT(pos.idx() - buf_bgn_idx_ < chr_buf_.size() - 1);
373 error       |                               ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
373 error ../src/scanner.cc:9:54: note: in definition of macro ‘TREE_SITTER_MARKDOWN_ASSERT’
373 error     9 | #define TREE_SITTER_MARKDOWN_ASSERT(condition) if (!(condition)) throw 1;
373 error       |                                                      ^~~~~~~~~
373 error ../src/./tree_sitter_markdown/lexer.cc: In member function ‘bool tree_sitter_markdown::Lexer::has_chr_at_idx(tree_sitter_markdown::LexedCharacter, tree_sitter_markdown::LexedIndex) const’:
373 error ../src/./tree_sitter_markdown/lexer.cc:185:50: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<int>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
373 error   185 |   TREE_SITTER_MARKDOWN_ASSERT(idx - buf_bgn_idx_ < chr_buf_.size());
373 error       |                               ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
373 error ../src/scanner.cc:9:54: note: in definition of macro ‘TREE_SITTER_MARKDOWN_ASSERT’
373 error     9 | #define TREE_SITTER_MARKDOWN_ASSERT(condition) if (!(condition)) throw 1;
373 error       |                                                      ^~~~~~~~~
373 error In file included from ../src/scanner.cc:23:
373 error ../src/./tree_sitter_markdown/lexer.cc:186:14: warning: comparison is always false due to limited range of data type [-Wtype-limits]
373 error   186 |   return idx < 0 ? false : chr_buf_[idx - buf_bgn_idx_] == chr;
373 error       |          ~~~~^~~
373 error ../src/./tree_sitter_markdown/lexer.cc: In member function ‘bool tree_sitter_markdown::Lexer::has_chr_at_idx(bool (*)(tree_sitter_markdown::LexedCharacter), tree_sitter_markdown::LexedIndex) const’:
373 error ../src/./tree_sitter_markdown/lexer.cc:190:50: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<int>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
373 error   190 |   TREE_SITTER_MARKDOWN_ASSERT(idx - buf_bgn_idx_ < chr_buf_.size());
373 error       |                               ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
373 error ../src/scanner.cc:9:54: note: in definition of macro ‘TREE_SITTER_MARKDOWN_ASSERT’
373 error     9 | #define TREE_SITTER_MARKDOWN_ASSERT(condition) if (!(condition)) throw 1;
373 error       |                                                      ^~~~~~~~~
373 error In file included from ../src/scanner.cc:23:
373 error ../src/./tree_sitter_markdown/lexer.cc:191:14: warning: comparison is always false due to limited range of data type [-Wtype-limits]
373 error   191 |   return idx < 0 ? false : is_chr(chr_buf_[idx - buf_bgn_idx_]);
373 error       |          ~~~~^~~
373 error ../src/./tree_sitter_markdown/lexer.cc: In member function ‘bool tree_sitter_markdown::Lexer::has_chr_in_rng(bool (*)(tree_sitter_markdown::LexedCharacter), tree_sitter_markdown::LexedIndex, tree_sitter_markdown::LexedIndex) const’:
373 error ../src/./tree_sitter_markdown/lexer.cc:196:54: warning: comparison of integer expressions of different signedness: ‘int’ and ‘std::vector<int>::size_type’ {aka ‘long unsigned int’} [-Wsign-compare]
373 error   196 |   TREE_SITTER_MARKDOWN_ASSERT(end_idx - buf_bgn_idx_ < chr_buf_.size() - 1);
373 error       |                               ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
373 error ../src/scanner.cc:9:54: note: in definition of macro ‘TREE_SITTER_MARKDOWN_ASSERT’
373 error     9 | #define TREE_SITTER_MARKDOWN_ASSERT(condition) if (!(condition)) throw 1;
373 error       |                                                      ^~~~~~~~~
373 error In file included from ../src/scanner.cc:21:
373 error ../src/./tree_sitter_markdown/inline_scan.cc: In function ‘bool tree_sitter_markdown::scn_blk_txt(tree_sitter_markdown::Lexer&, tree_sitter_markdown::InlineDelimiterList&, tree_sitter_markdown::InlineContextStack&, tree_sitter_markdown::BlockDelimiterList&, tree_sitter_markdown::BlockContextStack&)’:
373 error ../src/./tree_sitter_markdown/inline_scan.cc:307:1: warning: control reaches end of non-void function [-Wreturn-type]
373 error   307 | }
373 error       | ^
373 error In file included from ../src/scanner.cc:23:
373 error ../src/./tree_sitter_markdown/lexer.cc: In member function ‘tree_sitter_markdown::LexedLength tree_sitter_markdown::Lexer::clc_vtr_spc_cnt(tree_sitter_markdown::LexedColumn, tree_sitter_markdown::LexedColumn, tree_sitter_markdown::LexedLength&) const’:
373 error ../src/./tree_sitter_markdown/lexer.cc:242:1: warning: control reaches end of non-void function [-Wreturn-type]
373 error   242 | }
373 error       | ^
373 error In file included from ../src/scanner.cc:24:
373 error ../src/./tree_sitter_markdown/parse_table.cc: In function ‘tree_sitter_markdown::Symbol tree_sitter_markdown::get_blk_cls_sym(tree_sitter_markdown::Symbol)’:
373 error ../src/./tree_sitter_markdown/parse_table.cc:50:5: warning: control reaches end of non-void function [-Wreturn-type]
373 error    50 |     default: TREE_SITTER_MARKDOWN_ASSERT(false);
373 error       |     ^~~~~~~
373 error make: *** [tree_sitter_markdown_binding.target.mk:126: Release/obj.target/tree_sitter_markdown_binding/src/scanner.o] Error 1
373 error gyp ERR! build error
373 error gyp ERR! stack Error: `make` failed with exit code: 2
373 error gyp ERR! stack     at ChildProcess.onExit (/home/joe/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
373 error gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
373 error gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
373 error gyp ERR! System Linux 6.0.2-76060002-generic
373 error gyp ERR! command "/home/joe/.nvm/versions/node/v16.13.0/bin/node" "/home/joe/.nvm/versions/node/v16.13.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
373 error gyp ERR! cwd /home/joe/projet/todo_txt_parser/node_modules/tree-sitter-markdown
373 error gyp ERR! node -v v16.13.0
373 error gyp ERR! node-gyp -v v8.2.0
373 error gyp ERR! not ok
374 verbose exit 1

Atom language plugin?

The current markdown language packages for Atom don't use tree-sitter. I would be great if this package could be used as an Atom Language package.

Assertion failure for specific markdown file

The markdown parser fails for the following input: https://gist.github.com/dozed/411fa82c314772f4c57349aa9d88c86d

Reproduce:

const Parser = require('tree-sitter');
const Markdown = require('./build/Release/tree_sitter_markdown_binding');
const parser = new Parser();
parser.setLanguage(Markdown);
const text = fs.readFileSync('Test.md', 'utf8')
const tree = parser.parse(text);
tree

Results in:

terminate called after throwing an instance of 'int'
[1]    268688 IOT instruction (core dumped)  node

It seems to fail in: https://github.com/ikatyang/tree-sitter-markdown/blob/master/src/scanner.cc#L56

The relevant stacktrace in Emacs:

emacs: src/scanner.cc:56: unsigned int {anonymous}::Scanner::serialize(unsigned char*): Assertion `i <= 1024' failed.

Thread 1 "emacs" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
44	./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007fffee23c406 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007fffee22287c in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007fffee22279b in __assert_fail_base
    (fmt=0x7fffee3b9f68 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7fffe40d66d9 "i <= 1024", file=file@entry=0x7fffe40d66ca "src/scanner.cc", line=line@entry=56, function=function@entry=0x7fffe40d3f20 "unsigned int {anonymous}::Scanner::serialize(unsigned char*)") at ./assert/assert.c:92
#6  0x00007fffee233b86 in __assert_fail
    (assertion=assertion@entry=0x7fffe40d66d9 "i <= 1024", file=file@entry=0x7fffe40d66ca "src/scanner.cc", line=line@entry=56, function=function@entry=0x7fffe40d3f20 "unsigned int {anonymous}::Scanner::serialize(unsigned char*)")
    at ./assert/assert.c:101
#7  0x00007fffe407fa49 in (anonymous namespace)::Scanner::serialize (buffer=0x55555be60f65 "\002", this=0x55555bcc10c0) at src/scanner.cc:56
#8  tree_sitter_markdown_external_scanner_serialize(void*, char*) (payload=0x55555bcc10c0, buffer=0x55555be60f65 "\002") at src/scanner.cc:301
#9  0x00007fffdeb4e340 in ts_parser_parse () at /home/user/.emacs.d/elpa/tsc-20220212.1632/tsc-dyn.so
#10 0x00007fffdeb2d0d1 in tsc_dyn::parser::__emr_O_parse_chunks::h8c1adc3341858b2b () at /home/user/.emacs.d/elpa/tsc-20220212.1632/tsc-dyn.so
#11 0x00007fffdeb28a6a in std::panicking::try::h5d5045dd90c5b416 () at /home/user/.emacs.d/elpa/tsc-20220212.1632/tsc-dyn.so
#12 0x00007fffdeb2d526 in tsc_dyn::parser::__emrs_E_parse_chunks::extern_lambda::hd67f0f0313299272 () at /home/user/.emacs.d/elpa/tsc-20220212.1632/tsc-dyn.so
#13 0x00005555558b6797 in funcall_module (function=..., nargs=3, arglist=0x7fffe87ff228) at emacs-module.c:1183
#14 0x000055555585df8e in funcall_lambda (fun=..., nargs=3, arg_vector=0x7fffe87ff228) at eval.c:3162
#15 0x000055555585d0b4 in funcall_general (fun=..., numargs=3, args=0x7fffe87ff228) at eval.c:2945
#16 0x00005555558c64fb in exec_byte_code (fun=..., args_template=0, nargs=0, args=0x7fffe87ff180) at bytecode.c:811
#17 0x000055555585da49 in fetch_and_exec_byte_code (fun=..., args_template=771, nargs=3, args=0x7fffffffc808) at eval.c:3081
#18 0x000055555585df53 in funcall_lambda (fun=..., nargs=3, arg_vector=0x7fffffffc808) at eval.c:3153
#19 0x000055555585d0b4 in funcall_general (fun=..., numargs=3, args=0x7fffffffc808) at eval.c:2945
#20 0x000055555585d3cc in Ffuncall (nargs=4, args=0x7fffffffc800) at eval.c:2995
#21 0x000055555585c58c in funcall_nil (nargs=4, args=0x7fffffffc800) at eval.c:2677
#22 0x000055555585cb1a in run_hook_with_args (nargs=4, args=0x7fffffffc800, funcall=0x55555585c565 <funcall_nil>) at eval.c:2854
#23 0x000055555585c61d in Frun_hook_with_args (nargs=4, args=0x7fffffffc800) at eval.c:2719
#24 0x00005555557b958d in signal_after_change (charpos=648, lendel=0, lenins=1) at insdel.c:2283
#25 0x00005555557b3507 in insert_and_inherit (string=0x7fffffffc953 "``", nbytes=1) at insdel.c:696
#26 0x00005555557dd4de in internal_self_insert (c=96, n=1) at cmds.c:472
#27 0x00005555557dc911 in Fself_insert_command (n=..., c=...) at cmds.c:294
#28 0x000055555585d674 in funcall_subr (subr=0x555556033c20 <Sself_insert_command>, numargs=1, args=0x7fffe87ff100) at eval.c:3036
#29 0x00005555558c64d8 in exec_byte_code (fun=..., args_template=257, nargs=1, args=0x7fffffffd218) at bytecode.c:809
#30 0x000055555585da49 in fetch_and_exec_byte_code (fun=..., args_template=257, nargs=1, args=0x7fffffffd210) at eval.c:3081
#31 0x000055555585df53 in funcall_lambda (fun=..., nargs=1, arg_vector=0x7fffffffd210) at eval.c:3153
#32 0x000055555585d0b4 in funcall_general (fun=..., numargs=1, args=0x7fffffffd210) at eval.c:2945
#33 0x000055555585d3cc in Ffuncall (nargs=2, args=0x7fffffffd208) at eval.c:2995
#34 0x000055555584ef5c in Ffuncall_interactively (nargs=2, args=0x7fffffffd208) at callint.c:250
#35 0x000055555585d8b1 in funcall_subr (subr=0x55555603a220 <Sfuncall_interactively>, numargs=2, args=0x7fffffffd208) at eval.c:3059
#36 0x000055555585d068 in funcall_general (fun=..., numargs=2, args=0x7fffffffd208) at eval.c:2941
#37 0x000055555585d3cc in Ffuncall (nargs=3, args=0x7fffffffd200) at eval.c:2995
#38 0x00005555558518ab in Fcall_interactively (function=..., record_flag=..., keys=...) at callint.c:787
#39 0x000055555585d6aa in funcall_subr (subr=0x55555603a280 <Scall_interactively>, numargs=3, args=0x7fffe87ff070) at eval.c:3038
#40 0x00005555558c64d8 in exec_byte_code (fun=..., args_template=1025, nargs=1, args=0x7fffffffdb20) at bytecode.c:809
#41 0x000055555585da49 in fetch_and_exec_byte_code (fun=..., args_template=1025, nargs=1, args=0x7fffffffdb18) at eval.c:3081
#42 0x000055555585df53 in funcall_lambda (fun=..., nargs=1, arg_vector=0x7fffffffdb18) at eval.c:3153
#43 0x000055555585d0b4 in funcall_general (fun=..., numargs=1, args=0x7fffffffdb18) at eval.c:2945
#44 0x000055555585d3cc in Ffuncall (nargs=2, args=0x7fffffffdb10) at eval.c:2995
#45 0x000055555575dadb in call1 (fn=..., arg1=...) at /home/user/Code/tools/emacs-29/src/lisp.h:3247
#46 0x00005555557615f1 in command_loop_1 () at keyboard.c:1503
#47 0x00005555558583fe in internal_condition_case (bfun=0x555555760d45 <command_loop_1>, handlers=..., hfun=0x5555557600ce <cmd_error>) at eval.c:1474
#48 0x00005555557608da in command_loop_2 (handlers=...) at keyboard.c:1133
#49 0x00005555558574b5 in internal_catch (tag=..., func=0x5555557608ac <command_loop_2>, arg=...) at eval.c:1197
#50 0x0000555555760868 in command_loop () at keyboard.c:1111
#51 0x000055555575fb50 in recursive_edit_1 () at keyboard.c:720
#52 0x000055555575fd84 in Frecursive_edit () at keyboard.c:803
#53 0x000055555575b047 in main (argc=1, argv=0x7fffffffe048) at emacs.c:2529

Nodes for math mode (mathjax)

Would it be possible to extend the parser to find mathjax expressions ($blah$ $$blah$$) so that we can use that in queries?

Hierarchical syntax tree

I would like to know why this parser doesn't parse the markdown file into a proper hierarchy based on sections, so for example, why the following markdown:

# Test Heading Level 1

## Test Heading Level 2

### Test Heading Level 3

Results in this tree:

document [0, 0] - [4, 24])
  atx_heading [0, 0] - [0, 22])
    atx_heading_marker [0, 0] - [0, 1])
    heading_content [0, 1] - [0, 22])
      text [0, 1] - [0, 22])
  atx_heading [2, 0] - [2, 23])
    atx_heading_marker [2, 0] - [2, 2])
    heading_content [2, 2] - [2, 23])
      text [2, 2] - [2, 23])
  atx_heading [4, 0] - [4, 24])
    atx_heading_marker [4, 0] - [4, 3])
    heading_content [4, 3] - [4, 24])
      text [4, 3] - [4, 24])

Instead of something like this:

document [0, 0] - [4, 24])
  atx_heading [0, 0] - [0, 22])
    atx_heading_marker [0, 0] - [0, 1])
    heading_content [0, 1] - [0, 22])
      text [0, 1] - [0, 22])
    atx_heading [2, 0] - [2, 23])
      atx_heading_marker [2, 0] - [2, 2])
      heading_content [2, 2] - [2, 23])
        text [2, 2] - [2, 23])
      atx_heading [4, 0] - [4, 24])
        atx_heading_marker [4, 0] - [4, 3])
        heading_content [4, 3] - [4, 24])
          text [4, 3] - [4, 24])

The current format seems to make certain things very hard, if not impossible, like for example proper folding.

Is there a proper way to achieve something like folding with the current tree, and if not, why was the design chosen to be the way it is, and would changing it be an option?

Injecting yaml correctly

Hey @ikatyang, I wonder if there a better query to inject yaml.

[
  (thematic_break)
  (setext_heading
   (heading_content
    (_))
   (setext_h2_underline))
] @yaml

This worked but it missed up the syntax tree:

Tasks
---

Memos
---

Timeline
---

setext_heading [7, 0] - [8, 3]
  document [7, 0] - [7, 5]
    flow_node [7, 0] - [7, 5]
      plain_scalar [7, 0] - [7, 5]
        string_scalar [7, 0] - [7, 5]
  document [8, 0] - [8, 3]
  heading_content [7, 0] - [7, 5]
    text [7, 0] - [7, 5]
  setext_h2_underline [8, 0] - [8, 3]
setext_heading [10, 0] - [11, 3]
  document [10, 0] - [10, 5]
    flow_node [10, 0] - [10, 5]
      plain_scalar [10, 0] - [10, 5]
        string_scalar [10, 0] - [10, 5]
  document [11, 0] - [11, 3]
  heading_content [10, 0] - [10, 5]
    text [10, 0] - [10, 5]
  setext_h2_underline [11, 0] - [11, 3]
setext_heading [13, 0] - [14, 3]
  document [13, 0] - [13, 8]
    flow_node [13, 0] - [13, 8]
      plain_scalar [13, 0] - [13, 8]
        string_scalar [13, 0] - [13, 8]
  document [14, 0] - [14, 3]
  heading_content [13, 0] - [13, 8]
    text [13, 0] - [13, 8]
  setext_h2_underline [14, 0] - [14, 3]

expected:

setext_heading [0, 0] - [1, 3]
  heading_content [0, 0] - [0, 5]
    text [0, 0] - [0, 5]
  setext_h2_underline [1, 0] - [1, 3]
setext_heading [3, 0] - [4, 3]
  heading_content [3, 0] - [3, 5]
    text [3, 0] - [3, 5]
  setext_h2_underline [4, 0] - [4, 3]
setext_heading [6, 0] - [7, 3]
  heading_content [6, 0] - [6, 8]
    text [6, 0] - [6, 8]
  setext_h2_underline [7, 0] - [7, 3]

Thanks

SEGFAULTs and assertion failures

We at nvim-treesitter are using this parser for editor support in Neovim. We've received and experienced reports about segfaults of this parser nvim-treesitter/nvim-treesitter#602 neovim/neovim#13122. We would love to use this project, however we can't when it's killing the whole editor.

https://files.gitter.im/5506b96e15522ed4b3dd5317/KR5c/after.gif (referencing scanner 253)

Maybe a way to debug this would be to use tree-sitters libfuzzer feature.

# working dir is tree-sitter repo (https://github.com/tree-sitter/tree-sitter)
apt install libfuzzer-12-dev # or other version
ln -s ~/projects/tree-sitter-markdown ./test/fixtures/grammars/markdown
export LIB_FUZZER_PATH=/usr/lib/llvm-12/lib/libFuzzer.a
mkdir out
# next step probably requires CC to be clang, GCC won't work
./script/build-fuzzers
./out/markdown_fuzzer

python used in script must be python2 (or edit ./script/build-fuzzers by replacing python with python2)

Below you can see a example output achieved by this fuzzer.

#226964 NEW    cov: 9367 ft: 43554 corp: 2889/35Kb lim: 53 exec/s: 879 rss: 519Mb L: 52/53 MS: 1 InsertRepeatedBytes-
markdown_fuzzer: test/fixtures/grammars/markdown/src/./tree_sitter_markdown/inline_scan.cc:103: tree_sitter_markdown::Symbol tree_sitter_markdown::scn_inl(tree_sitter_markdown::Lexer &, tree_sitter_markdown::InlineDelimiterList &, tree_sitter_markdown::InlineContextStack &, tree_sitter_markdown::BlockDelimiterList &, tree_sitter_markdown::BlockContextStack &, InlineDelimiterList::Iterator &, const InlineDelimiterList::Iterator &, tree_sitter_markdown::LexedIndex &, const bool): Assertion `blk_dlms.back().sym() == SYM_LIT_LBK' failed.
==189124== ERROR: libFuzzer: deadly signal
    #0 0x52ae41 in __sanitizer_print_stack_trace (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x52ae41)
    #1 0x473758 in fuzzer::PrintStackTrace() (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x473758)
    #2 0x458f53 in fuzzer::Fuzzer::CrashCallback() (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x458f53)
    #3 0x7f2c56085baf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x14baf)
    #4 0x7f2c55e9a8ca in __libc_signal_restore_set signal/../sysdeps/unix/sysv/linux/internal-signals.h:104:3
    #5 0x7f2c55e9a8ca in raise signal/../sysdeps/unix/sysv/linux/raise.c:47:3
    #6 0x7f2c55e7f863 in abort stdlib/abort.c:79:7
    #7 0x7f2c55e7f748 in __assert_fail_base assert/assert.c:92:3
    #8 0x7f2c55e91a95 in __assert_fail assert/assert.c:101:3
    #9 0x5827bc in tree_sitter_markdown::scn_inl(tree_sitter_markdown::Lexer&, tree_sitter_markdown::InlineDelimiterList&, tree_sitter_markdown::InlineContextStack&, tree_sitter_markdown::BlockDelimiterList&, tree_sitter_markdown::BlockContextStack&, std::_List_iterator<tree_sitter_markdown::InlineDelimiter>&, std::_List_iterator<tree_sitter_markdown::InlineDelimiter> const&, unsigned short&, bool) /home/stephan/projects/tree-sitter/test/fixtures/grammars/markdown/src/./tree_sitter_markdown/inline_scan.cc:103:13
    #10 0x5829fa in tree_sitter_markdown::scn_inl(tree_sitter_markdown::Lexer&, tree_sitter_markdown::InlineDelimiterList&, tree_sitter_markdown::InlineContextStack&, tree_sitter_markdown::BlockDelimiterList&, tree_sitter_markdown::BlockContextStack&) /home/stephan/projects/tree-sitter/test/fixtures/grammars/markdown/src/./tree_sitter_markdown/inline_scan.cc:38:10
    #11 0x6097d4 in (anonymous namespace)::Scanner::scan(TSLexer*, bool const*) /home/stephan/projects/tree-sitter/test/fixtures/grammars/markdown/src/scanner.cc:261:18
    #12 0x6084a2 in tree_sitter_markdown_external_scanner_scan /home/stephan/projects/tree-sitter/test/fixtures/grammars/markdown/src/scanner.cc:305:19
    #13 0x64ec30 in ts_parser__lex (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x64ec30)
    #14 0x63c4b6 in ts_parser__advance (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x63c4b6)
    #15 0x6367d1 in ts_parser_parse (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x6367d1)
    #16 0x646fe9 in ts_parser_parse_string_encoding (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x646fe9)
    #17 0x646c13 in ts_parser_parse_string (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x646c13)
    #18 0x555293 in LLVMFuzzerTestOneInput (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x555293)
    #19 0x45a6c1 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x45a6c1)
    #20 0x459c0a in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x459c0a)
    #21 0x45b8b7 in fuzzer::Fuzzer::MutateAndTestOne() (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x45b8b7)
    #22 0x45c455 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x45c455)
    #23 0x44a5fb in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x44a5fb)
    #24 0x473f32 in main (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x473f32)
    #25 0x7f2c55e81cb1 in __libc_start_main csu/../csu/libc-start.c:314:16
    #26 0x41f56d in _start (/home/stephan/projects/tree-sitter/out/markdown_fuzzer+0x41f56d)

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 2 CrossOver-CopyPart-; base unit: 8c7fd13d377b76f756fea15bdc24d58fa6c383a9
0x2a,0x1,0x3,0x24,0x24,0xa,0x2d,0x3a,0xa,0x3c,0x3c,0x3c,0x2a,0x1,0x3,0x24,0x24,0xa,0x2d,0x3a,0xa,0x3c,0x3c,0x3c,0x2b,0x2d,0x2b,0x2d,
*\x01\x03$$\x0a-:\x0a<<<*\x01\x03$$\x0a-:\x0a<<<+-+-
artifact_prefix='./'; Test unit written to ./crash-d4d36d06d5d8987d8eefc4fa7ea7868479ed6ea7
Base64: KgEDJCQKLToKPDw8KgEDJCQKLToKPDw8Ky0rLQ==

the test input (I can also send you the file). Probably it's easier to read from above report (*\x01\x03$$\x0a-:\x0a<<<*\x01\x03$$\x0a-:\x0a<<<+-+-)

*��$$
-:
<<<*��$$
-:
<<<+-+-

tight loop and eats all RAM

The following input causes tree-sitter to tight loop and eat all the available RAM:

[(https://example.com/foo[bar]

parser still crashes and loops forever under fuzzing

Hi,

I thought I'd see how the parser did with the new TREE_SITTER_MARKDOWN_AVOID_CRASH trick against the fuzzer. Unfortunately, the fuzzer still finds crashes, and infinite loops.

Here are a couple of examples, I guess that there may be more.

[foo](*https://example.com
[foo](*https://example.com

result: parser loops and eats up all available memory.

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

result:

fatal runtime error: Rust cannot catch foreign exceptions
Aborted

Distinguish heading level tokens

The actual ATX heading markers (#, ##, etc.) are not queryable now, we should expose them as anonymous nodes so that it could be queried.

Similar to what I did in ikatyang/tree-sitter-yaml#9.

Question: How can I use this parser in C++

Unfortunately there is no information provided for using this parser with tree sitter in C++.
There is no Makefile or similar provided, using cargo build does not produce a library that I can consume from C++.
I tried following this tree-sitter tutorial for using the json parser and adjusting it for the markdown parser.
This means I added the parser.c from the tree-sitter-markdown project I first get the following error:

Compiler/Linker output with just parser.c (Meson)

FAILED: main
c++  -o main main.p/main.cpp.o main.p/tree-sitter-markdown_src_parser.c.o -Wl,--as-needed -Wl,--no-undefined '-Wl,- 
rpath,$ORIGIN/../tree-sitter' -Wl,-rpath-link,/home/max/Code/tree-sitter-test/builddir/../tree-sitter -Wl,--start-group 
/home/max/Code/tree-sitter-test/tree-sitter/libtree-sitter.so -Wl,--end-group
/usr/bin/ld: main.p/main.cpp.o: in function `main':
main.cpp:(.text+0x27): undefined reference to `tree_sitter_markdown()'
/usr/bin/ld: main.p/tree-sitter-markdown_src_parser.c.o:(.data.rel+0xb8): undefined reference to 
`tree_sitter_markdown_external_scanner_create'
/usr/bin/ld: main.p/tree-sitter-markdown_src_parser.c.o:(.data.rel+0xc0): undefined reference to 
`tree_sitter_markdown_external_scanner_destroy'
/usr/bin/ld: main.p/tree-sitter-markdown_src_parser.c.o:(.data.rel+0xc8): undefined reference to 
`tree_sitter_markdown_external_scanner_scan'
/usr/bin/ld: main.p/tree-sitter-markdown_src_parser.c.o:(.data.rel+0xd0): undefined reference to 
`tree_sitter_markdown_external_scanner_serialize'
/usr/bin/ld: main.p/tree-sitter-markdown_src_parser.c.o:(.data.rel+0xd8): undefined reference to 
`tree_sitter_markdown_external_scanner_deserialize'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

To try and fix this error I then added the scanner.cc to the compilation.
I then get the following error, which I cannot quite explain.

Compiler/Linker output with parser.c and scanner.cc(Meson)

FAILED: main 
c++  -o main main.p/main.cpp.o main.p/tree-sitter-markdown_src_parser.c.o main.p/tree-sitter-markdown_src_scanner.cc.o -Wl,--as-needed -Wl,--no-undefined '-Wl,-rpath,$ORIGIN/../tree-sitter' -Wl,-rpath-link,/home/max/Code/tree-sitter-test/builddir/../tree-sitter -Wl,--start-group /home/max/Code/tree-sitter-test/tree-sitter/libtree-sitter.so -Wl,--end-group
/usr/bin/ld: main.p/main.cpp.o: in function `main':
main.cpp:(.text+0x27): undefined reference to `tree_sitter_markdown()'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

What am I doing wrong?

Here are my project files, but they are basically just the tutorial and a

meson.build

project('ke_markdown', ['c', 'cpp'], default_options : ['cpp_std=c++17', 'buildtype=debugoptimized'] )

cpp = meson.get_compiler('cpp')

libdirs = []
incdirs = []

libdirs += meson.current_source_dir() + '/tree-sitter/'
incdirs += include_directories('tree-sitter/lib/src')
incdirs += include_directories('tree-sitter/lib/include')

treesitter_dep = cpp.find_library('tree-sitter', dirs: libdirs, required: true, header_include_directories : incdirs)

executable('main', ['main.cpp', 'tree-sitter-markdown/src/parser.c', 'tree-sitter-markdown/src/scanner.cc'], include_directories : incdirs, dependencies : [treesitter_dep])

main.cpp

#include <assert.h>
#include <string.h>
#include <stdio.h>
#include <tree_sitter/api.h>

// Declare the `tree_sitter_json` function, which is
// implemented by the `tree-sitter-json` library.
TSLanguage *tree_sitter_markdown();

int main() {
  // Create a parser.
  TSParser *parser = ts_parser_new();

  // Set the parser's language (JSON in this case).
  ts_parser_set_language(parser, tree_sitter_markdown());

  // Build a syntax tree based on source code stored in a string.
  const char *source_code = "# main";
  TSTree *tree = ts_parser_parse_string(
    parser,
    NULL,
    source_code,
    strlen(source_code)
  );

  // Get the root node of the syntax tree.
  TSNode root_node = ts_tree_root_node(tree);

  return 0;
}

Link definitions should take precedence over code spans

The following case will result in an assertion error, whose root cause is that link definitions should take precedence over code spans, while currently the code spans take precedence over link definitions.

[`]: a
[`]: b

support for front-matter

Hey @ikatyang,

Is it possible to support identification of front-matter block. I'd like to inject yaml to it like

(fenced_code_block
  (info_string) @language
  (code_fence_content) @content)

Currently I'm getting for

---
title: GitHub Flavored Markdown
author: GitHub
---

=

thematic_break [0, 0] - [0, 3]
setext_heading [1, 0] - [3, 3]
  heading_content [1, 0] - [2, 14]
    text [1, 0] - [1, 31]
    soft_line_break [1, 31] - [2, 0]
    text [2, 0] - [2, 14]
  setext_heading_underline [3, 0] - [3, 3]

Thanks

link_reference_definition not parsed when on last line of file

Steps to reproduce

Parse the markdown:

[ref]

[ref]: http://example.com

Bug 🐛

This results in the tree:

(document (paragraph (link (link_text (text)))) (paragraph (link (link_text (text))) (text) (uri_autolink (text))))

Notice how the link definition is not found

Expected

If I add a blank line the end of the file like so:

[ref]

[ref]: http://example.com

I instead see:

(document (paragraph (link (link_text (text)))) (link_reference_definition (link_label (text)) (link_destination (text))))

This seems to be the expected parsing

Crash when parsing with ts_parser_set_included_ranges

Hi,

First of all, thanks for creating this project.

I've been getting a crash via the assert on line 253 of scanner.cc when running the following code. Note that the crash doesn't happen if the ts_parser_set_included_ranges line is removed, but for larger files removing this line has a major impact on performance.

#include <string.h>
#include <stdio.h>
#include <tree_sitter/api.h>

TSLanguage *tree_sitter_markdown();

char* readFile() {
  FILE *f = fopen("/path/to/CHANGELOG.md", "rb");
  fseek(f, 0, SEEK_END);
  long fsize = ftell(f);
  fseek(f, 0, SEEK_SET);  /* same as rewind(f); */

  char *string = malloc(fsize + 1);
  fread(string, 1, fsize, f);
  fclose(f);

  string[fsize] = 0;
  return string;
}

int main() {
  const char *source_code = readFile();
  TSParser *parser = ts_parser_new();
  ts_parser_set_language(parser, tree_sitter_markdown());

  TSTree *tree = ts_parser_parse_string(parser, NULL, source_code, strlen(source_code));
  TSNode root_node = ts_tree_root_node(tree);
  char *string = ts_node_string(root_node);

  TSRange range = {
    .start_point = {
      .row = 915,
      .column = 0,
    },
    .end_point = {
      .row = 1033,
      .column = 38,
    },
    .start_byte = 35840,
    .end_byte = 40326
  };
  
  ts_parser_set_included_ranges(parser, &range, 1);
  tree = ts_parser_parse_string(parser, tree, source_code, strlen(source_code));
  
  free(string);
  ts_tree_delete(tree);
  ts_parser_delete(parser);

  return 0;
}

Using this file, or seemingly any larger CHANGELOG file.

If you need any additional information, please let me know. Thanks.

Avoid crash

Context: nvim-treesitter/nvim-treesitter#872 (comment)

Link-like should not be allowed to be used in the link definition label

Ref: #14 (comment)

[[]:)]:0

[[](]:0 ()
0 0

Assertion failed: (inl_dlm.sym() == SYM_EXT_AUT_LNK_BGN || inl_dlm.sym() == SYM_EXT_AUT_LNK_CTN), function transfer_to, file inline_delimiter.cc, line 240

So if I'm trying to parse the following content, it fails with the following error and exits from the script:

Assertion failed: (inl_dlm.sym() == SYM_EXT_AUT_LNK_BGN || inl_dlm.sym() == SYM_EXT_AUT_LNK_CTN), function transfer_to, file inline_delimiter.cc, line 240

The file content (https://raw.githubusercontent.com/dotnet/runtime/main/docs/area-owners.md):

# Pull Requests Tagging

If you need to tag folks on an issue or PR, you will generally want to tag the owners (not the lead) for [area](#areas) to which the change or issue is closest to. For areas which are large and can be operating system or architecture specific it's better to tag owners of [OS](#operating-systems) or [Architecture](#architectures).

## Repos

This list is for this **dotnet/runtime** repo. The **dotnet/aspnetcore** repo has [its own list](https://github.com/dotnet/aspnetcore/blob/main/docs/area-owners.md).

## Areas

Note: Editing this file doesn't update the mapping used by `@msftbot` for area-specific issue/PR notifications. That configuration is part of the [`fabricbot.json`](../.github/fabricbot.json) file, and many areas use GitHub teams for those notifications. If you're a community member interested in receiving area-specific issue/PR notifications, you won't appear in this table or be added to those GitHub teams, but you can create a PR that updates `fabricbot.json` to add yourself to those notifications. See [automation.md](infra/automation.md) for more information on the schema and tools used by FabricBot.

| Area                                           | Lead                 | Owners (area experts to tag in PRs and issues)      | Notes                                                                                                                                                                                                                                                                                     |
|------------------------------------------------|----------------------|-----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| area-AssemblyLoader-coreclr                    | @agocke              | @agocke @vitek-karas @vsadov                        |                                                                                                                                                                                                                                                                                           |
| area-AssemblyLoader-mono                       | @SamMonoRT           | @lambdageek                                         |                                                                                                                                                                                                                                                                                           |
| area-Build-mono                                | @steveisok           | @akoeplinger                                        |                                                                                                                                                                                                                                                                                           |
| area-Codeflow                                  | @dotnet/dnr-codeflow | @dotnet/dnr-codeflow                                | Used for automated PRs that ingest code from other repos                                                                                                                                                                                                                                  |
| area-Codegen-AOT-mono                          | @SamMonoRT           | @vargaz @kotlarmilos                                |                                                                                                                                                                                                                                                                                           |
| area-CodeGen-coreclr                           | @JulieLeeMSFT        | @BruceForstall @dotnet/jit-contrib                  |                                                                                                                                                                                                                                                                                           |
| area-Codegen-Interpreter-mono                  | @SamMonoRT           | @BrzVlad @kotlarmilos                               |                                                                                                                                                                                                                                                                                           |
| area-Codegen-Intrinsics-mono                   | @SamMonoRT           | @fanyang-mono                                       |                                                                                                                                                                                                                                                                                           |
| area-Codegen-JIT-mono                          | @SamMonoRT           | @vargaz                                             |                                                                                                                                                                                                                                                                                           |
| area-Codegen-LLVM-mono                         | @SamMonoRT           | @vargaz                                             |                                                                                                                                                                                                                                                                                           |
| area-Codegen-meta-mono                         | @SamMonoRT           | @vargaz                                             |                                                                                                                                                                                                                                                                                           |
| area-CoreLib-mono                              | @steveisok           | @vargaz                                             |                                                                                                                                                                                                                                                                                           |
| area-CrossGen/NGEN-coreclr                     | @mangod9             | @dotnet/crossgen-contrib                            |                                                                                                                                                                                                                                                                                           |
| area-crossgen2-coreclr                         | @mangod9             | @trylek @dotnet/crossgen-contrib                    |                                                                                                                                                                                                                                                                                           |
| area-Debugger-mono                             | @lewing              | @thaystg @radical                                   |                                                                                                                                                                                                                                                                                           |
| area-DependencyModel                           | @ericstj             | @dotnet/area-dependencymodel                        | Included:<ul><li>Microsoft.Extensions.DependencyModel</li></ul>                                                                                                                                                                                                                           |
| area-Diagnostics-coreclr                       | @tommcdon            | @tommcdon                                           |                                                                                                                                                                                                                                                                                           |
| area-Diagnostics-mono                          | @steveisok           | @lateralusX @mdh1418                                |                                                                                                                                                                                                                                                                                           |
| area-EnC-mono                                  | @SamMonoRT           | @lambdageek                                         | Hot Reload on WebAssembly, Android, iOS, etc                                                                                                                                                                                                                                              |
| area-ExceptionHandling-coreclr                 | @mangod9             | @janvorli                                           |                                                                                                                                                                                                                                                                                           |
| area-Extensions-Caching                        | @ericstj             | @dotnet/area-extensions-caching                     | Consultants: @mgravell, @sebastienros                                                                                                                                                                                                                                                     |
| area-Extensions-Configuration                  | @ericstj             | @dotnet/area-extensions-configuration               | Consultants: @eerhardt                                                                                                                                                                                                                                                                    |
| area-Extensions-DependencyInjection            | @ericstj             | @dotnet/area-extensions-dependencyinjection         | Consultants: @halter73, @benjaminpetit                                                                                                                                                                                                                                                    |
| area-Extensions-FileSystem                     | @jeffhandley         | @dotnet/area-extensions-filesystem                  |                                                                                                                                                                                                                                                                                           |
| area-Extensions-Hosting                        | @ericstj             | @dotnet/area-extensions-hosting                     | Consultants: @halter73, @tratcher                                                                                                                                                                                                                                                         |
| area-Extensions-HttpClientFactory              | @karelz              | @dotnet/ncl                                         |                                                                                                                                                                                                                                                                                           |
| area-Extensions-Logging                        | @ericstj             | @dotnet/area-extensions-logging                     | Consultants: @brennanconroy                                                                                                                                                                                                                                                               |
| area-Extensions-Options                        | @ericstj             | @dotnet/area-extensions-options                     | Consultants: @eerhardt, @brennanconroy                                                                                                                                                                                                                                                    |
| area-Extensions-Primitives                     | @ericstj             | @dotnet/area-extensions-primitives                  |                                                                                                                                                                                                                                                                                           |
| area-GC-coreclr                                | @mangod9             | @Maoni0                                             |                                                                                                                                                                                                                                                                                           |
| area-GC-mono                                   | @SamMonoRT           | @BrzVlad                                            |                                                                                                                                                                                                                                                                                           |
| area-Host                                      | @agocke              | @jeffschwMSFT @vitek-karas  @vsadov                 | Issues with dotnet.exe including bootstrapping, framework detection, hostfxr.dll and hostpolicy.dll                                                                                                                                                                                       |
| area-HostModel                                 | @agocke              | @vitek-karas                                        |                                                                                                                                                                                                                                                                                           |
| area-ILTools-coreclr                           | @JulieLeeMSFT        | @BruceForstall @dotnet/jit-contrib                  |                                                                                                                                                                                                                                                                                           |
| area-Infrastructure                            | @agocke              | @jeffschwMSFT @MichaelSimons                        |                                                                                                                                                                                                                                                                                           |
| area-Infrastructure-coreclr                    | @agocke              | @jeffschwMSFT @trylek                               |                                                                                                                                                                                                                                                                                           |
| area-Infrastructure-installer                  | @MichaelSimons       | @NikolaMilosavljevic                                |                                                                                                                                                                                                                                                                                           |
| area-Infrastructure-libraries                  | @ericstj             | @dotnet/area-infrastructure-libraries               | Covers:<ul><li>Packaging</li><li>Build and test infra for libraries in dotnet/runtime repo</li><li>VS integration</li></ul><br/>                                                                                                                                                          |
| area-Infrastructure-mono                       | @steveisok           | @directhex                                          |                                                                                                                                                                                                                                                                                           |
| area-Interop-coreclr                           | @agocke              | @AaronRobinsonMSFT @jkoritzinsky                    |                                                                                                                                                                                                                                                                                           |
| area-Interop-mono                              | @marek-safar         | @lambdageek                                         |                                                                                                                                                                                                                                                                                           |
| area-Meta                                      | @jeffhandley         | @dotnet/area-meta                                   | Cross-cutting concerns that span many or all areas, including project-wide code/test patterns and documentation.                                                                                                                                                                          |
| area-Microsoft.CSharp                          | @jaredpar            | @cston @333fred                                     | Archived component - limited churn/contributions (see [#27790](https://github.com/dotnet/runtime/issues/27790))                                                                                                                                                                           |
| area-Microsoft.VisualBasic                     | @jaredpar            | @cston @333fred                                     | Archived component - limited churn/contributions (see [#27790](https://github.com/dotnet/runtime/issues/27790))                                                                                                                                                                           |
| area-Microsoft.Win32                           | @ericstj             | @dotnet/area-microsoft-win32                        | Included:<ul><li>System.Windows.Extensions</li></ul>                                                                                                                                                                                                                                      |
| area-NativeAOT-coreclr                         | @agocke              | @dotnet/ilc-contrib                                 |                                                                                                                                                                                                                                                                                           |
| area-PAL-coreclr                               | @mangod9             | @janvorli                                           |                                                                                                                                                                                                                                                                                           |
| area-Performance-mono                          | @SamMonoRT           | @SamMonoRT                                          |                                                                                                                                                                                                                                                                                           |
| area-R2RDump-coreclr                           | @mangod9             | @trylek                                             |                                                                                                                                                                                                                                                                                           |
| area-ReadyToRun-coreclr                        | @mangod9             | @trylek                                             |                                                                                                                                                                                                                                                                                           |
| area-Serialization                             | @HongGit             | @StephenMolloy @HongGit                             | Packages:<ul><li>System.Runtime.Serialization.Xml</li><li>System.Runtime.Serialization.Json</li><li>System.Private.DataContractSerialization</li><li>System.Xml.XmlSerializer</li></ul> Excluded:<ul><li>System.Runtime.Serialization.Formatters</li></ul>                                |
| area-Setup                                     | @MichaelSimons       | @NikolaMilosavljevic                                | Distro-specific (Linux, Mac and Windows) setup packages and msi files                                                                                                                                                                                                                     |
| area-Single-File                               | @agocke              | @vitek-karas @vsadov                                |                                                                                                                                                                                                                                                                                           |
| area-Snap                                      | @MichaelSimons       | @NikolaMilosavljevic @leecow @MichaelSimons         |                                                                                                                                                                                                                                                                                           |
| area-System.Buffers                            | @jeffhandley         | @dotnet/area-system-buffers                         |                                                                                                                                                                                                                                                                                           |
| area-System.CodeDom                            | @ericstj             | @dotnet/area-system-codedom                         |                                                                                                                                                                                                                                                                                           |
| area-System.Collections                        | @jeffhandley         | @dotnet/area-system-collections                     | Excluded:<ul><li>System.Array -> System.Runtime</li></ul>                                                                                                                                                                                                                                 |
| area-System.ComponentModel                     | @ericstj             | @dotnet/area-system-componentmodel                  | Consultants: @dotnet/dotnet-winforms                                                                                                                                                                                                                                                      |
| area-System.ComponentModel.Composition         | @ericstj             | @dotnet/area-system-componentmodel-composition      |                                                                                                                                                                                                                                                                                           |
| area-System.ComponentModel.DataAnnotations     | @jeffhandley         | @dotnet/area-system-componentmodel-dataannotations  | Included:<ul><li>System.ComponentModel.Annotations</li></ul>                                                                                                                                                                                                                              |
| area-System.Composition                        | @ericstj             | @dotnet/area-system-composition                     |                                                                                                                                                                                                                                                                                           |
| area-System.Configuration                      | @ericstj             | @dotnet/area-system-configuration                   |                                                                                                                                                                                                                                                                                           |
| area-System.Console                            | @jeffhandley         | @dotnet/area-system-console                         |                                                                                                                                                                                                                                                                                           |
| area-System.Data                               | @ajcvickers          | @ajcvickers @davoudeshtehari @david-engel @roji     | <ul><li>Odbc, OleDb - @saurabh500</li></ul>                                                                                                                                                                                                                                               |
| area-System.Data.Odbc                          | @ajcvickers          | @ajcvickers @roji                                   |                                                                                                                                                                                                                                                                                           |
| area-System.Data.OleDB                         | @ajcvickers          | @ajcvickers @roji                                   |                                                                                                                                                                                                                                                                                           |
| area-System.Data.SqlClient                     | @David-Engel         | @davoudeshtehari @david-engel @jrahnama             | Archived component - limited churn/contributions (see https://devblogs.microsoft.com/dotnet/introducing-the-new-microsoftdatasqlclient/)                                                                                                                                                  |
| area-System.DateTime                           | @ericstj             | @dotnet/area-system-datetime                        | System namespace APIs related to dates and times, including DateOnly, DateTime, DateTimeKind, DateTimeOffset, DayOfWeek, TimeOnly, TimeSpan, TimeZone, and TimeZoneInfo                                                                                                                   |
| area-System.Diagnostics                        | @tommcdon            | @tommcdon                                           |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics-coreclr                | @tommcdon            | @tommcdon                                           |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics-mono                   | @lewing              | @thaystg @radical                                   |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics.Activity               | @tommcdon            | @dotnet/area-system-diagnostics-activity            |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics.EventLog               | @ericstj             | @dotnet/area-system-diagnostics-eventlog            |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics.Metric                 | @tommcdon            | @noahfalk                                           |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics.PerformanceCounter     | @ericstj             | @dotnet/area-system-diagnostics-performancecounter  |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics.Process                | @jeffhandley         | @dotnet/area-system-diagnostics-process             |                                                                                                                                                                                                                                                                                           |
| area-System.Diagnostics.Tracing                | @tommcdon            | @noahfalk @tommcdon @tarekgh                        | Included:<ul><li>System.Diagnostics.DiagnosticSource</li></ul>                                                                                                                                                                                                                            |
| area-System.Diagnostics.TraceSource            | @ericstj             | @dotnet/area-system-diagnostics-tracesource         |                                                                                                                                                                                                                                                                                           |
| area-System.DirectoryServices                  | @ericstj             | @dotnet/area-system-directoryservices               | Consultants: @BRDPM @grubioe @jay98014                                                                                                                                                                                                                                                    |
| area-System.Drawing                            | @ericstj             | @dotnet/area-system-drawing                         | Excluded:<ul><li>System.Drawing.Common -> winforms</li></ul>                                                                                                                                                                                                                              |
| area-System.Dynamic.Runtime                    | @jaredpar            | @cston @333fred                                     | Archived component - limited churn/contributions (see [#27790](https://github.com/dotnet/runtime/issues/27790))                                                                                                                                                                           |
| area-System.Formats.Asn1                       | @jeffhandley         | @dotnet/area-system-formats-asn1                    | Consultants: @bartonjs @vcsjones                                                                                                                                                                                                                                                          |
| area-System.Formats.Cbor                       | @jeffhandley         | @dotnet/area-system-formats-cbor                    | Consultants: @bartonjs @vcsjones                                                                                                                                                                                                                                                          |
| area-System.Formats.Tar                        | @jeffhandley         | @dotnet/area-system-formats-tar                     |                                                                                                                                                                                                                                                                                           |
| area-System.Globalization                      | @ericstj             | @dotnet/area-system-globalization                   |                                                                                                                                                                                                                                                                                           |
| area-System.IO                                 | @jeffhandley         | @dotnet/area-system-io                              |                                                                                                                                                                                                                                                                                           |
| area-System.IO.Compression                     | @jeffhandley         | @dotnet/area-system-io-compression                  | Included:<ul><li>System.Formats.Tar</li><li>System.IO.Packaging</li></ul>                                                                                                                                                                                                                 |
| area-System.IO.Hashing                         | @jeffhandley         | @dotnet/area-system-io-hashing                      | APIs within the System.IO.Hashing namespace, which align more with cryptography than with I/O                                                                                                                                                                                             |
| area-System.IO.Pipelines                       | @adityamandaleeka    | @davidfowl @halter73                                |                                                                                                                                                                                                                                                                                           |
| area-System.IO.Ports                           | @jeffhandley         | @dotnet/area-system-io-ports                        |                                                                                                                                                                                                                                                                                           |
| area-System.Linq                               | @jeffhandley         | @dotnet/area-system-linq                            |                                                                                                                                                                                                                                                                                           |
| area-System.Linq.Expressions                   | @jaredpar            | @cston @333fred                                     | Archived component - limited churn/contributions (see [#27790](https://github.com/dotnet/runtime/issues/27790))                                                                                                                                                                           |
| area-System.Linq.Parallel                      | @jeffhandley         | @dotnet/area-system-linq-parallel                   | Consultants: @stephentoub @kouvel                                                                                                                                                                                                                                                         |
| area-System.Management                         | @ericstj             | @dotnet/area-system-management                      | WMI                                                                                                                                                                                                                                                                                       |
| area-System.Memory                             | @jeffhandley         | @dotnet/area-system-memory                          |                                                                                                                                                                                                                                                                                           |
| area-System.Net                                | @karelz              | @dotnet/ncl                                         | Included:<ul><li>System.Uri</li></ul>                                                                                                                                                                                                                                                     |
| area-System.Net.Http                           | @karelz              | @dotnet/ncl                                         |                                                                                                                                                                                                                                                                                           |
| area-System.Net.Quic                           | @karelz              | @dotnet/ncl                                         |                                                                                                                                                                                                                                                                                           |
| area-System.Net.Security                       | @karelz              | @dotnet/ncl                                         |                                                                                                                                                                                                                                                                                           |
| area-System.Net.Sockets                        | @karelz              | @dotnet/ncl                                         |                                                                                                                                                                                                                                                                                           |
| area-System.Numerics                           | @jeffhandley         | @dotnet/area-system-numerics                        |                                                                                                                                                                                                                                                                                           |
| area-System.Numerics.Tensors                   | @jeffhandley         | @dotnet/area-system-numerics-tensors                |                                                                                                                                                                                                                                                                                           |
| area-System.Reflection                         | @ericstj             | @dotnet/area-system-reflection                      |                                                                                                                                                                                                                                                                                           |
| area-System.Reflection.Emit                    | @ericstj             | @dotnet/area-system-reflection-emit                 |                                                                                                                                                                                                                                                                                           |
| area-System.Reflection.Metadata                | @ericstj             | @dotnet/area-system-reflection-metadata             | Consultants: @tmat                                                                                                                                                                                                                                                                        |
| area-System.Resources                          | @ericstj             | @dotnet/area-system-resources                       |                                                                                                                                                                                                                                                                                           |
| area-System.Runtime                            | @jeffhandley         | @dotnet/area-system-runtime                         | Included:<ul><li>System.Runtime.Serialization.Formatters</li><li>System.Runtime.InteropServices.RuntimeInfo</li><li>System.Array</li></ul>Excluded:<ul><li>Path -> System.IO</li><li>StopWatch -> System.Diagnostics</li><li>Uri -> System.Net</li><li>WebUtility -> System.Net</li></ul> |
| area-System.Runtime.Caching                    | @HongGit             | @StephenMolloy @HongGit                             |                                                                                                                                                                                                                                                                                           |
| area-System.Runtime.CompilerServices           | @ericstj             | @dotnet/area-system-runtime-compilerservices        |                                                                                                                                                                                                                                                                                           |
| area-System.Runtime.InteropServices            | @agocke              | @AaronRobinsonMSFT @jkoritzinsky                    | Excluded:<ul><li>System.Runtime.InteropServices.RuntimeInfo</li></ul>                                                                                                                                                                                                                     |
| area-System.Runtime.InteropServices.JavaScript | @lewing              | @pavelsavara                                        |                                                                                                                                                                                                                                                                                           |
| area-System.Runtime.Intrinsics                 | @jeffhandley         | @dotnet/area-system-runtime-intrinsics              | Consultants: @echesakovMSFT @kunalspathak                                                                                                                                                                                                                                                 |
| area-System.Security                           | @jeffhandley         | @dotnet/area-system-security                        | Consultants: @bartonjs @GrabYourPitchforks @SteveHarter @vcsjones @wfurt                                                                                                                                                                                                                  |
| area-System.ServiceModel                       | @HongGit             | @HongGit @mconnew                                   | Repo: https://github.com/dotnet/WCF<br>Packages:<ul><li>System.ServiceModel.Primitives</li><li>System.ServiceModel.Http</li><li>System.ServiceModel.NetTcp</li><li>System.ServiceModel.Duplex</li><li>System.ServiceModel.Security</li></ul>                                              |
| area-System.ServiceModel.Syndication           | @HongGit             | @StephenMolloy @HongGit                             |                                                                                                                                                                                                                                                                                           |
| area-System.ServiceProcess                     | @ericstj             | @dotnet/area-system-serviceprocess                  |                                                                                                                                                                                                                                                                                           |
| area-System.Speech                             | @jeffhandley         | @ericstj @jeffhandley                               |                                                                                                                                                                                                                                                                                           |
| area-System.Text.Encoding                      | @jeffhandley         | @dotnet/area-system-text-encoding                   |                                                                                                                                                                                                                                                                                           |
| area-System.Text.Encodings.Web                 | @jeffhandley         | @dotnet/area-system-text-encodings-web              |                                                                                                                                                                                                                                                                                           |
| area-System.Text.Json                          | @jeffhandley         | @dotnet/area-system-text-json                       |                                                                                                                                                                                                                                                                                           |
| area-System.Text.RegularExpressions            | @ericstj             | @dotnet/area-system-text-regularexpressions         | Consultants: @stephentoub                                                                                                                                                                                                                                                                 |
| area-System.Threading                          | @mangod9             | @kouvel                                             |                                                                                                                                                                                                                                                                                           |
| area-System.Threading.Channels                 | @ericstj             | @dotnet/area-system-threading-channels              | Consultants: @stephentoub                                                                                                                                                                                                                                                                 |
| area-System.Threading.RateLimiting             | @rafikiassumani-msft | @BrennanConroy @halter73                            |                                                                                                                                                                                                                                                                                           |
| area-System.Threading.Tasks                    | @ericstj             | @dotnet/area-system-threading-tasks                 | Consultants: @stephentoub                                                                                                                                                                                                                                                                 |
| area-System.Transactions                       | @ajcvickers          | @roji                                               |                                                                                                                                                                                                                                                                                           |
| area-System.Xml                                | @jeffhandley         | @dotnet/area-system-xml                             |                                                                                                                                                                                                                                                                                           |
| area-TieredCompilation-coreclr                 | @mangod9             | @kouvel                                             |                                                                                                                                                                                                                                                                                           |
| area-Tools-ILLink                              | @agocke              | @dotnet/illink                                      |                                                                                                                                                                                                                                                                                           |
| area-Tools-ILVerification                      | @JulieLeeMSFT        | @BruceForstall @dotnet/jit-contrib                  |                                                                                                                                                                                                                                                                                           |
| area-Tracing-coreclr                           | @tommcdon            | @sywhang @josalem                                   |                                                                                                                                                                                                                                                                                           |
| area-Tracing-mono                              | @steveisok           | @lateralusX                                         |                                                                                                                                                                                                                                                                                           |
| area-TypeSystem-coreclr                        | @mangod9             | @davidwrighton @MichalStrehovsky @janvorli @mangod9 |                                                                                                                                                                                                                                                                                           |
| area-UWP                                       | @tommcdon            | @jashook                                            | UWP-specific issues including Microsoft.NETCore.UniversalWindowsPlatform and Microsoft.Net.UWPCoreRuntimeSdk                                                                                                                                                                              |
| area-VM-coreclr                                | @mangod9             | @mangod9                                            |                                                                                                                                                                                                                                                                                           |
| area-VM-meta-mono                              | @SamMonoRT           | @lambdageek                                         |                                                                                                                                                                                                                                                                                           |
| area-VM-reflection-mono                        | @SamMonoRT           | @lambdageek                                         | MonoVM-specific reflection and reflection-emit issues                                                                                                                                                                                                                                     |
| area-VM-threading-mono                         | @SamMonoRT           | @lambdageek                                         |                                                                                                                                                                                                                                                                                           |
| area-Workloads                                 | @lewing              | @dotnet/net-sdk-workload-contributors               |                                                                                                                                                                                                                                                                                           |

## Operating Systems

| Operating System | Lead          | Owners (area experts to tag in PRs and issues)          | Description  |
|------------------|---------------|---------------------------------------------------------|--------------|
| os-alpine        |               |                                                         |              |
| os-android       | @steveisok    | @akoeplinger                                            |              |
| os-freebsd       |               |                                                         |              |
| os-mac-os-x      |               |                                                         |              |
| os-maccatalyst   | @steveisok    | @kotlarmilos                                            |              |
| os-ios           | @steveisok    | @vargaz, @kotlarmilos                                   |              |
| os-tizen         | @gbalykov     | @hjleee, @wscho77, @clamp03, @JongHeonChoi, @t-mustafin |              |
| os-tvos          | @steveisok    | @vargaz, @kotlarmilos                                   |              |
| os-wasi          | @lewing       | @pavelsavara                                            |              |

## Architectures

| Architecture     | Lead          | Owners (area experts to tag in PRs and issues)        | Description  |
|------------------|---------------|-------------------------------------------------------|--------------|
| arch-wasm        | @lewing       | @lewing @BrzVlad                                      |              |

## Community Triagers

While not necessarily associated with a specific area, these community members have the power to assist with routing and labeling issues and pull requests, and are generally knowledgeable about how the repo works:

* @am11
* @filipnavara
* @huoyaoyuan
* @SingleAccretion
* @teo-tsirpanis
* @tmds
* @vcsjones

Also, I'll attach a crash report on my mac:
tree_sitter_error-stack-trace.txt

HTML Comments

I'm using tree-sitter-markdown with nvim-treesitter, and I'm not sure whether what I'm seeing is some artifact from the combination of these two programs or an issue with tree-sitter-markdown. But just in case it's an issue with the parser, here's something I've found.

I've noticed that when an html comment (<!-- comment -->) begins a line, it seems to be recognised correctly as a comment (and highlighted with a sort of greyed out color in neovim). However, when a comment is inline (This is a sentence<!-- comment --> with a comment.), then the comment shows up just like normal text. Finally, when the line starts with a comment and also contains inline comments (<!-- comment -->This is a sentence<!-- comment --> with a comment.), then all comments are displayed correctly again.

Rust dependencies are out-of-date

Current tree-sitter version is 0.20.9 but this crate depends on 0.19. Fix is cargo update (or update the version manually in Cargo.toml). An unsuspecting user would add the following to their Cargo.toml and get this error:

tree-sitter = "0.20.9"
tree-sitter-markdown = "0.7.1"

error[E0308]: mismatched types
   --> src/markdown.rs:81:29
    |
81  |         parser.set_language(tree_sitter_markdown::language()).unwrap();
    |                ------------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `tree_sitter::Language`, found a different struct `tree_sitter::Language`
    |                |
    |                arguments to this function are incorrect
    |
    = note: perhaps two different versions of crate `tree_sitter` are being used?
note: associated function defined here
   --> /home/qtfkwk/.cargo/registry/src/github.com-1ecc6299db9ec823/tree-sitter-0.20.9/binding_rust/lib.rs:357:12
    |
357 |     pub fn set_language(&mut self, language: Language) -> Result<(), LanguageError> {
    |            ^^^^^^^^^^^^

For more information about this error, try `rustc --explain E0308`.
error: could not compile `project` due to previous error

Also the Cargo.lock file is included, but this is a library not a binary crate, so afaik it should not be committed to the repository (?). Fix would be git rm Cargo.lock. If it's needed for some reason, it has outdated dependencies too and it will be fixed by cargo update ran above.

I added this repo as a submodule to a private project and it worked without issue with these updates.

---

An alternative fix is to simply use old/matching tree-sitter in the Cargo.toml:

tree-sitter = "0.19"
tree-sitter-markdown = "0.7.1"

Hard crash parsing certain markdown files

I have been using tree_sitter_languages to parse markdown. Some of my md files are causing a hard crash of the parser.parse() call:

Assertion failed: (i == length), function deserialize, file scanner.cc, line 79.
Abort trap: 6

I have isolated a sample which can trigger the crash. I binary searched the file to find a single offending line. Then, I gradually replaced all the characters with X until the crash went away.

Some notes:

• There is a single astrisk * remaining. The parser no longer crashes if you remove it.
• There are still underscores _ remaining. Turning them into Xs resolves the crash.
• Reducing the line length, or splitting the text onto multiple lines also seems to avoid the crash.

I have filed this issue with both of these projects, as I am not sure which is most likely to be able to resolve it:

• https://github.com/ikatyang/tree-sitter-markdown
• https://github.com/grantjenks/py-tree-sitter-languages

code = '''
XXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXX X XXXX X XXXXX *X XXXXXXXX XXXXXXX X XXXX X XXXXX XXXXXXX_XXXXX XXXXXXXXXXX X XXXX X XXXXX XXXXXXX_XXXXXXXXX XXXXX X XXXX X XXXXX XX_XXXXX XXXX X XXXX X XXXXX XXXXXX XXXXX X XXXX X XXXXX XXXXXXXXXX XXXXXXXXX X XXXX X XXXXX XXXXXXXX_XX_XXXXXXX XXXX X XXXX X XXXXX XX_XXXXXXXXX XXXX X XXXX X XXXXX XXX_XXXXXXXXX XXXXXXXXXXXXXXX X XXXX X XXXXX XXXXXX_XXXXXXXX XXXXXXXXXXXXXXXX X XXXX X XXXXX XXXXXX XXXXXXXXX X XXXX X XXXXX XXXXX_XXXXXX XXXXXXXXXX X XXXX X XXXXX XXXXXXX XXXXXXXXXXXXXXXXXXXX X XXXXX XXXXXXX_XXXXXXX_XXXXXXXX_XXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXX_XXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXX X XXXX X XXXXX XXXX_XXXXX_XXX_XXXX_XXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXXXX XXXXXXXXXXXXXXXX X XXXX X XXXXX XXXXXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXX_XXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXX X XXXX X XXXXX XXXX_XXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXX_XXXXXXX_XXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXX_XXXXXX_XXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXXXX_XXXXX_XXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXXXX_XXXXX_XXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXXXX_XXXXXX XXXXXXXXXXXXX X XXXX X XXXXX XXXX_XXXXXXXX XXXXXXXXXXX X XXXX X XXXXX XXXXXXXXXX XXXXXXXXX X XXXX X XXXXX XXXXXXXXXX XXXXXXXXX X XXXX X XXXXX XXXXX_XXXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXX_XXXXXXXXXXX XXXXXXXXXXXXXXX X XXXX X XXXXX XXXXXXXXXXXX XXXXXXXXXXXXXXXX X XXXX X XXXXX XXXXXXX_XXXXX_XXX_XXXXX XXX X XXXX X XXXXX XXXXXX_XXXXXX_XXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXX_XXXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXX_XXXX_XX_XXXXXXX XXXXXXXXXXXX X XXXX X XXXXX XXXXXXXX_XXXXXXX XXX X XXXXXXXXXXXX XXXX X XXXX X XXXXX XXXXXXXXX XXX X XXXXXXXXXXXX XXXX X XXXX X XXXXX XXXXXXXX XXX X XXX XXXXXX_XXXXXXXX XXXX X XXXXXX XXX_XXXX XXXXXXXXXXXX XXXXX X XXXX X XXXX
'''

import tree_sitter_languages

print(tree_sitter_languages.__version__) # 1.7.0

parser = tree_sitter_languages.get_parser('markdown')
parser.parse(bytes(code, "utf8"))

Code span should take precedence over html tags

If there are 2 (or more) html tags within code spans, the html tags take precedence over the code span.

Tree structure of the above sentence from TSPlayground:

paragraph [0, 0] - [1, 0]
  inline [0, 0] - [0, 39]
    html_tag [0, 10] - [0, 16]
    code_span [0, 16] - [0, 31]
      code_span_delimiter [0, 16] - [0, 17]
      code_span_delimiter [0, 30] - [0, 31]
    html_tag [0, 31] - [0, 38]

Add highlights.scm

Context: #19 (comment)

soft_line_break contain ">" when parsing blockquote

Currently, blockquote

> 1
> 2

is parsed as

document [0, 0] - [1, 3])
  block_quote [0, 0] - [1, 3])
    paragraph [0, 2] - [1, 3])
      text [0, 2] - [0, 3])
      soft_line_break [0, 3] - [1, 2])
      text [1, 2] - [1, 3])

which means soft_line_break is \n>. I think it's wrong to contain >.

Where are injections / aliases kept?

I noticed that glimmer is highlighted in markdown codefences, but the more limited subset of a language, hbs, is not:

where would I add an alias for hbs to use the glimmer highlighting?

Different Colors for different heading levels

When you have like

# Heading 1
Notes...
## Heading 2
More notes...
### Heading 3
Hello world

Is it possible to make each heading have different colors with this parser? I really like Markdown + this parser for taking notes, but it would be extremely useful if different levels of the markdown would have different colors for readability.