tree-sitter/tree-sitter-embedded-template

Tree-sitter grammar for embedded template languages like ERB, EJS

tree-sitter

Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited. Tree-sitter aims to be:

• General enough to parse any programming language
• Fast enough to parse on every keystroke in a text editor
• Robust enough to provide useful results even in the presence of syntax errors
• Dependency-free so that the runtime library (which is written in pure C) can be embedded in any application

Links

• Documentation
• Rust binding
• WASM binding
• Command-line interface

tree-sitter-embedded-template's Issues

Bug: RangeError: Incompatible language version.

Error:

RangeError: Incompatible language version. Compatible range: 13 - 13. Got: 14
/node_modules/tree-sitter/index.js:260

Steps to reproduce:

npm install -g tree-sitter
npm install -g tree-sitter-embedded-template

const Parser = require("tree-sitter")
const Et = require("tree-sitter-embedded-template")
const parser = new Parser()
parser.setLanguage(Et)

Versions:

npm list -g
[email protected]
[email protected]

How do I provide CODE / CONTENT grammars?

I've read through http://tree-sitter.github.io/tree-sitter/creating-parsers#external-scanners
And read through the source code of this repo, and then I started looking at http://tree-sitter.github.io/tree-sitter/using-parsers#multi-language-documents

But, I'm not entirely sure where the boundaries are between the parser (produced by tree-sitter generate) and the rust bindings I'm attempting to make.

Is there another resource that I might be missing?

Cheers!

Cannot get syntax highlighting

Hi, I installed the plugin but I can't have syntax highlighting.
I also typed these commands:
:TSInstall ejs
:TSBufEnable hightlight

My config repo : https://github.com/hamidreza4dev/dotfiles

Does this work with any file type? .yml.ejs?

I'm writing some .ejs on top of yml, and am wondering how to configure this package?

Atm, I setup

local parser_config = require("nvim-treesitter.parsers").get_parser_configs()

parser_config.ejs = {
  install_info = {
    url = "https://github.com/tree-sitter/tree-sitter-embedded-template",
    files = { "src/parser.c" },
    requires_generate_from_grammar = true,
  },
  filetype = "ejs",
}

but when I open an ejs file, no highlighting occurs.

Highlighting Not Working on GitHub.com

I was told this package was used for GitHub.com's syntax highlighting. Please close this if this isn't true.

I've added a .gitattributes linguist-language mapping for app's HTML files to EJS:
https://github.com/MetaMask/metamask-extension/blob/0495d5b395bab9f3ec954cdcac84aaf085fc7f98/.gitattributes#L24

But the HTML files aren't syntax highlighted as I'd expect. Example file that lacks EJS syntax highlighting:
https://github.com/MetaMask/metamask-extension/blob/0cd76e5b6a3fe4b491348ac0c348a27a82368a32/app/home.html#L6-L10

Parser never leaves comment mode when it encounters an embedded comment

As far as I am aware, ERB Syntax permits an embedded ruby comment in a non-comment erb tag

<%   # This line is not an ERB comment, but it does nothing %>

But, once the treesitter encounters a mid-embedding comment, it evaluates and highlights the rest of the embedded ruby as comments.
For example

<%= tag.div class: 'outer-div' do %>
    <%= tag.div class: 'inner-div' do %>
        <p>Text</p>
    <% end # .inner-div %>
<% end # .outer-div %>

<% value = 12 %>
<p> <%= value %> </p>

Even though this works just fine all the rest of the embedded ruby will be evaluated, from the fourth line:

<% end # .inner-div %>

until the end, all embedded ruby is highlighted as a comment.

As you may notice, the github syntax highlighter also exhibits this bug at present.

Fails to compile on node 12

Description

Fails to compile on node 12

Steps to Reproduce

1. Ensure you are running on node 12
2. Clone a copy of the repository if you don't have it
3. Run npm install

Expected behavior:
Compile successfully
Actual behavior:
The following errors are thrown

void Init(Handle<Object> exports, Handle<Object> module) {
     ^
../src/binding.cc:13:11: error: use of undeclared identifier 'Handle'
void Init(Handle<Object> exports, Handle<Object> module) {
          ^
../src/binding.cc:13:18: error: 'Object' does not refer to a value
void Init(Handle<Object> exports, Handle<Object> module) {
                 ^
/Users/musaibrahim/.atom/.node-gyp/.node-gyp/5.0.12/include/node/v8.h:3257:17: note: declared here
class V8_EXPORT Object : public Value {
                ^
../src/binding.cc:13:26: error: use of undeclared identifier 'exports'
void Init(Handle<Object> exports, Handle<Object> module) {
                         ^
../src/binding.cc:13:35: error: use of undeclared identifier 'Handle'
void Init(Handle<Object> exports, Handle<Object> module) {
                                  ^
../src/binding.cc:13:42: error: 'Object' does not refer to a value
void Init(Handle<Object> exports, Handle<Object> module) {
                                         ^
/Users/musaibrahim/.atom/.node-gyp/.node-gyp/5.0.12/include/node/v8.h:3257:17: note: declared here
class V8_EXPORT Object : public Value {
                ^
../src/binding.cc:13:50: error: use of undeclared identifier 'module'
void Init(Handle<Object> exports, Handle<Object> module) {
                                                 ^
../src/binding.cc:13:57: error: expected ';' after top level declarator
void Init(Handle<Object> exports, Handle<Object> module) {
                                                        ^
                                                        ;

Reproduces how often:

100%

Parsing fails on empty ERB comments

The following code, a comment block with no comment text, will fail to parse:

<%# %>

(template [0, 0] - [1, 0]
  (comment_directive [0, 0] - [1, 0]
    (comment [0, 3] - [1, 0])))
example.erb	0 ms	(MISSING "%>" [1, 0] - [1, 0])

<% with highlight true freeze nvim

Hello,
when enable highlight my nvim is freezing when typing <%

HTML nodes contain ERB code in text node.

Hello,

I'm not sure if it's a bug or if it's me not understanding the parser logic.
I try to parse an ERB file. I followed the provided example using C API, and converted it to use the Node equivalent.

const Parser = require("tree-sitter")
const ERB = require("tree-sitter-embedded-template")
const Ruby = require("tree-sitter-ruby")
const HTML = require("tree-sitter-html")

const parse = function parse(sourceCode) {
  const parser = new Parser()
  parser.setLanguage(ERB)
  const erbTree = parser.parse(sourceCode)
  const erbRootNode = erbTree.rootNode

  const rubyRanges = []
  const htmlRanges = []

  erbRootNode.children.forEach(node => {
    if (node.type === "content") {
      htmlRanges.push({
        startIndex: node.startIndex,
        endPosition: node.endPosition,
        startPosition: node.startPosition,
        endIndex: node.endIndex,
    })
    } else {
      const codeNode = node.namedChild(0)
      if(!codeNode) return
      rubyRanges.push({
        startIndex: codeNode.startIndex,
        endPosition: codeNode.endPosition,
        startPosition: codeNode.startPosition,
        endIndex: codeNode.endIndex,
      })
    }
  })

  parser.setLanguage(Ruby)
  const rubyTree = parser.parse(
    sourceCode,
    undefined,
    { includedRanges: rubyRanges }
  )
  const rubyRootNode = rubyTree.rootNode

  parser.setLanguage(HTML)
  const htmlTree = parser.parse(
    sourceCode,
    undefined,
    { includedRanges: htmlRanges }
  )
  const htmlRootNode = htmlTree.rootNode

  const erb_sexp = erbRootNode.toString()
  const html_sexp = htmlRootNode.toString()
  const ruby_sexp = rubyRootNode.toString()

  console.log(`ERB: ${erb_sexp}`)
  console.log(`HTML: ${html_sexp}`)
  console.log(`RUBY: ${ruby_sexp}`)
}

When I run the following code:

parse("<ul><%= person.name %></ul>")

It outputs:

• ERB: (template (content) (output_directive (code)) (content))
• HTML: (fragment (element (start_tag (tag_name)) (end_tag (tag_name))))
• RUBY: (program (call receiver: (identifier) method: (identifier)))

Ruby part is correctly excluded from the HTML.

When I run the following code (note the space before the ending tag):

parse("<ul><%= person.name %> </ul>")

It outputs:

• ERB: (template (content) (output_directive (code)) (content))
• HTML: (fragment (element (start_tag (tag_name)) (text) (end_tag (tag_name))))
• RUBY: (program (call receiver: (identifier) method: (identifier)))

There is a new (text) node due to space. I expect this text node, but I don't understand why it contains the text <%= person.name %> (ruby code + space) and not just (space).
As I specify the HTML range to parse, I was expecting a text node containing only a single space.

Is it:

• an issue related to the HTML parser?
• an issue related to the ranges detection?
• the expected behaviour?

I think I'm missing something, but I don't know what. Please do not hesitate to ask for more details if I'm not clear.

Thank you for your help ⛑️

Support for haml and slim template

Hi! As per the documentation ERB and EJS are supported. Asking if there is any plans in supporting Haml and/or Slim, or if that would fall under a different project altogether? Thanks!

I'm new to neovim, how do I install this plugin?

Sorry can't figure this out, how do I install, use and make sure it's always loaded? BTW I use Astronvim and followed the installation instructions here: https://astronvim.com/. The default plugin manager is lazy.nvim.

Thanks!

Help

Hi
I want to create an EJS parser because I couldn't find any. I want to create a parser because I want to create a plugin for Prettier to format the EJS code. The problem is that I don't know how to work with this package. I looked up at the street sitter docs and I literally didn't understand anything.
Thanks