ericsmekens/jsep

JavaScript Expression Parser

Eric Smekens

Hi, I'm Eric, a Node.js developer at Enexis 👨‍🔧⚡, working on the energy transition. I live in Dorst, a small village near Breda in the Netherlands. 🙌

My values, explained by my favorite proverbs

🌴 No man is an island.
☘ You will never plough a field if you only turn it over in your mind.
💖 Be kind, for everyone you meet is fighting a hard battle.

How I work

I love it to learn something new, get that elbow grease, and then sharing the problems and knowledge you've found. And after that, piece by piece, no matter how tiny that piece might be, make it more understandable, make it better.

Projects

Have a look in my repositories down below.

🚗 obd projects

Part of my graduation internship. It perfectly describes the way how I like to work. Learning obd and node.js at the same time, and making something that is shareable to the rest of the community. It is definitely not the best package out there, but it is still a place of shareable information.

✔ jsep

Used this as a dependency in my first job. Spent some free time on it, to learn more and try to extend it. Eventually became a collaborator, and now the maintainer of the package. Thanks to the original author as well for the great work.

Get in touch

LinkedIn

StackOverFlow

jsep's Issues

Numeric and literals

678none + "a" is not working.

678none <-- should be a literal.

Custom Operators

Allow custom binary and unary operators to be specified (e.g. add a binary operator ^ for power) and allow its precedence to be specified.

Proposed enhancement:

jsep.addBinaryOperator(name, precedence)
jsep.addUnaryOperator(name, precedence)
jsep.removeBinaryOperator(name)
jsep.removeUnaryOperator(name)

Publish new version to npmjs?

I've found jeep really useful for a project I'm working on--thanks for the package!

For this project, I haven't needed to handle CallExpressions, but I did as of today. However, I'm using the package from Typescript, and the typings that are included in the 0.3.2 release don't include typings for ICallExpression. It is on master, so I can simply install that, but I'm wondering if a release is (or can be) planned anytime soon?

Thanks.

ThisExpression as a property

Shouldn't this in test.this be considered as an Identifier instead of a ThisExpression?

function parameters invalid

Parsing the function with parameters,

functionName(args1 ,args2, ) is considered as valid but actually it should be invalid.
3rd argument is invalid, It should not be blank.

Wrong precedence of +

1+1^2 gives:

type: 'BinaryExpression'
    •operator: '^'
    •left:
        •type: 'BinaryExpression'
        •operator: '+'
        •left: ◦type: 'Literal'
            ◦value: 1
            ◦raw: '1'
        •right:
            ◦type: 'Literal'
            ◦value: 1
            ◦raw: '1'
    •right:
        •type: 'Literal'
        •value: 2
        •raw: '2'

It should actually parse it as 1+(1^2).

Unary expression with no arg should be invalid.

Currently, the parser considers this perfectly valid - x+++, x+++---- and so on. The problem seems to be that in gobbleToken, the argument object is not being checked for presence.
Here's an example...
For x+++, the parsed structure is -

• type: 'BinaryExpression'
• operator: '+'
• left:
  • type: 'Identifier'
  • name: 'x'
• right:
  • type: 'UnaryExpression'
  • operator: '+'
  • argument:
    • type: 'UnaryExpression'
    • operator: '+'
    • argument: false
    • prefix: true
  • prefix: true

Line breaks make jsep throw

Testcase: Just try parsing "\n1+1".
It reproduces on the website too.

Unable to download from BOWER

Getting this error:

ETIMEDOUT Request to https://bower.herokuapp.com/packages/jsep failed: ETIMEDOUT

why? I was able to install it a few weeks ago. Let me know if this can be resolved ASAP

Assing typescript to project

Hi,
There is no typescript definitions file associated with the repo.

Are there any plans to add one?

Thanks.

Unary operator precedence

Please add feature to adjust unary operator precedence so it will be possible to make following example works properly.
-x^2 and (-x)^2
gives

{
    type: "BinaryExpression",
    operator: "^",
    left: {
        type: "UnaryExpression",
        operator: "-",
        argument: { type: "Identifier", name: "x" },
        prefix: true
    },
    right: { type: "Literal", value: 2, raw: 2 }
}

I would like to add something like this:
jsep.addUnaryOp("-", 0);
So output of jsep() would be

{
    type: "UnaryExpression",
    operator: "-",
    argument: {
        type: "BinaryExpression",
        operator: "^",
        left: { type: "Identifier", name: "x" },
        right: { type: "Literal", value: 2, raw: 2 }
    },
    prefix: true
}

It would be only optional feature so IMO it will only improve library.

Identifiers that start with an alphanumeric operator are not parsed correctly

Turns out this was not introduced after #27 was merged, just exposed. It also happens with binary operators however, it's just harder to trigger, so in a sense it became a more serious bug after #27 was merged.

Testcase:

> jsep.addUnaryOp("not");
> jsep("notes")
< {type: "UnaryExpression", operator: "not", argument: {…}, prefix: true}
> jsep.addBinaryOp("and");
> jsep("a andb");
< {type: "BinaryExpression", operator: "and", left: {…}, right: {…}}

Newline, Return in expressions (not strings)

Would be nice if \r and \n were allowed as whitespace, tweak at line 132 I think.

Great package!
Mike

Allow postfix unary operators

It would be cool if unary operations could be at the end, not just the beginning.

How to add "-" as unary operator with lower precedence than "^"

Hi,

I am running into the issue that when interpret the expression "2^-1^2^3", the returned expression tree takes precedence as ((2^(-1))^2)^3. But the expectation is 2^(-((1^2)^3)). I realize that in this case the unary operator "-" takes higher precedence than "^".

Is there any way that I can assign "-" with precedence lower than "^"?

I have tried to call jsep.addUnaryOp('-'); but it doesn't seem to change the default precedence.

Backslash behaviour in strings should match esprima's output.

Currently it is not matchting the same behaviour as esprima.
Postponing it a little bit. Esprima displays another value when using backslashes.

Support object literals

An expression with object-literal is not supported. E.g.

{ object: 'Literal' }

How come?

Expose operators and precedences

There are many use cases where you want to inspect which operators are currently registered and/or look up their precedences.

E.g. Assume you want to add and as a synonym for &&. Currently one has to look at the code to find the precedence of && and use the same. And of course, that means it cannot be done dynamically, on runtime.
Or you may just need to look up whether an operator is registered. Currently the only way to do that is to try and parse a test expression.

This could be done in a number of ways:

1. Just expose unary_ops and binary_ops on jsep.
2. Add functions that accept an operator and return its precedence (we already have that for binary operators in binaryPrecedence(), we just need to expose it.

@EricSmekens Once I know how you'd rather proceed, I can implement it and send a PR.

And while we're at it, we could also expose isDecimalDigit, isIdentifierStart etc so they can be overridden, to create an even more extensible parser 😁

JSEP does not throw when binary operators are used as unary

Removing the unary operator "+" using jsep.removeUnaryOp("+"), but not removing "+" as a binary operator causes unary expressions such as +1 to be interpreted as binary expressions.

Multicharacter operations failure

“NOT” cannot be added as a unary operator.

jsep.addUnaryOp("NOT")
>>> JavaScript Expression Parser (JSEP) v0.3.0
>>> Object {type: "Compound", body: Array[2]}
jsep("(NOT x)")
>>> jsep.min.js:2 Uncaught Error: Unclosed ( at character 5(…)

This works with other operators:

jsep.addUnaryOp(“%")
>>> JavaScript Expression Parser (JSEP) v0.3.0
jsep("% x")
>>> Object {type: "UnaryExpression", operator: "%", argument: Object, prefix: true}

Implicit Binary Operators

Add a flag to allow implicit binary operators (e.g. 2pi means 2*pi). This flag could be optional

Preparing 0.3.1 release.

Need to define a tasklist to prepare 0.3.1 release.

• Add tests for new functionality and specific fixed use-cases.
• Update package.json to build jsep. (Deprecated and unsafe versions are being used (build-tools).)

Backslashes are not handled properly

A string literal containing "" should be parsed to "", but instead it gets parsed to "". This is true for any even number of backslashes you throw in there. The problem is in gobbleStringLiteral, in the switch statement: there should be a default case that does 'str += ch', but there isn't, so backslashes that aren't part of a common escape code get dropped on the floor.

In fact, a backslash followed by anything that isn't a common escape sequence should result in the character after the backslash being added to the string. For example, saying 'var str = "\J"' ends up with a string whose value is "J"

Parsing array expression after logical operator '||' failed.

When I ran this code:

jsep('a || [1, 2]');

then I got exception stack:

Error: Expected expression after || at character 5
    at throwError (/Users/mytharcher/work/packages/yust/node_modules/jsep/build/jsep.js:38:16)
    at gobbleBinaryExpression (/Users/mytharcher/work/packages/yust/node_modules/jsep/build/jsep.js:210:7)
    at gobbleExpression (/Users/mytharcher/work/packages/yust/node_modules/jsep/build/jsep.js:140:17)
    at jsep (/Users/mytharcher/work/packages/yust/node_modules/jsep/build/jsep.js:524:17)
    at Object.<anonymous> (/Users/mytharcher/work/packages/yust/test.js:3:28)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)

a || [1, 2]should be a legal JavaScript expression.

Add jsep to cdnjs

It would be very cool if you guys add jsep to cdnjs services.

regards
Heitor

Why not support regular expressions and new expression?

Exposing the eval code in the API or the doc

Hi there, jsep looks wonderful.

One thing that might be obvious to people familiar with esprima is that it is trivial to evaluate the generated AST:

var do_eval = function(node) {
  if(node.type === "BinaryExpression") {
    return binops[node.operator](do_eval(node.left), do_eval(node.right));
  } else if(node.type === "UnaryExpression") {
    return unops[node.operator](do_eval(node.argument));
  } else if(node.type === "Literal") {
    return node.value;
  }
};

I found it in the tests and it was not obvious to me.

I hope this helps
Many thanks!

Invalid expression parses successfully and incorrectly

The following invalid expression parses successfully even though it's not valid syntax:

jsep("myFunction('test' === 'test'")

As seen in the image below, it returns the incorrect result:

Ternary shorthands parsing

Are you planning to add support for ternary shorthands parsing?
I mean expressions like "a = b ? c : d"
Thanks!

A reverse of parse feature, nodes to expression

Is there any way instead of parse expression string to nodes, it does the reverse. Create an expression string from nodes.
I use jsep to parse the expression in my rule engine editor. But, to save the edited expression I need to convert the new nodes to an expression.
In Symfony Expression Language there are 2 ways, parse and "dump" (nodes to expression).

AST for expression with "anonymous" array indices are not translated correctly

It seems that expressions that contain array indices without identifiers
are not parsed correctly. In the example below the "+" is not detected as a binary-operator but as an unary-operator.

Such expressions are often used to express that some expression components are evaluated against an implicit root-object.

You can find an example for this in the second snippet here:
https://spring.io/blog/2013/12/04/what-s-new-in-spring-data-mongodb-1-4-m1

I want to be able to do the same in pure java script:
https://gist.github.com/thomasdarimont/b54a8e9fcc01ed59d465

As an example the following expression:

jsep("[0] + [1]")

Is translated to:

{
   "type":"Compound",
   "body":[
      {
         "type":"Array",
         "body":[
            {
               "type":"Literal",
               "value":0,
               "raw":"0"
            }
         ]
      },
      {
         "type":"UnaryExpression",
         "operator":"+",
         "argument":false,
         "prefix":true
      },
      {
         "type":"Array",
         "body":[
            {
               "type":"Literal",
               "value":1,
               "raw":"1"
            }
         ]
      }
   ]
}

It would be helpful if this were rendered as:

{
   "type":"BinaryExpression",
   "operator":"+",
   "left": {
         "type":"Array",
         "body":[
            {
               "type":"Literal",
               "value":0,
               "raw":"0"
            }
         ]
      },
   "right":{
         "type":"Array",
         "body":[
            {
               "type":"Literal",
               "value":1,
               "raw":"1"
            }
         ]
      }
}

Cannot add NOT as a unary operator

“NOT” cannot be added as a unary operator.

jsep.addUnaryOp("NOT")
>>> JavaScript Expression Parser (JSEP) v0.3.0
>>> Object {type: "Compound", body: Array[2]}
jsep("(NOT x)")
>>> jsep.min.js:2 Uncaught Error: Unclosed ( at character 5(…)

As a control, note this works for other operators:

jsep.addUnaryOp(“%")
>>> JavaScript Expression Parser (JSEP) v0.3.0
jsep("% x")
>>> Object {type: "UnaryExpression", operator: "%", argument: Object, prefix: true}

Incorrect precedence with 1^2^3-4^5

Hi,
Thank you for providing this impressive parser to help me handle most of the complex equation parsing.
I found an issue with the result when I parsed the equation 1^2^3-4^5. For this equation I was hoping to get a binary in the below structure (equivalent to (1^2^3)-(4^5)):

But actually it returns the binary tree object representing 1^2^(3-4)^5

Is this a bug with the library or is there anything I can config?
Please advise how I can overcome this issue.
Thank you ahead for your help.

The website is down

Getting ERR_NAME_NOT_RESOLVED in Chrome

Automated build/test

I've set up a very simple automated build/test on a fork of jsep.
I think it was worth my time, and will improve the project.

https://travis-ci.org/EricSmekens/jsep
https://circleci.com/gh/EricSmekens/jsep

Going to make a PR soon, so we can see it also posts statuses back.

(You can only do this on your own projects as maintainer, not collabs, so when we decide, soney has to activate it.).

publish version 0.3.1 to npm

Arrow functions?

Awesome project, I'm looking forward to using this in a library I'm working on! So glad I don't need to bite the bullet and include the 30KB of Acorn!

Are there any plans to support arrow functions, or are they out of scope? I would argue something like array.filter(a => a > 2) is still a simple expression, but YMMV. :)

In any case, fantastic work and I'm so glad this seems to still be maintained! 😃

Starting an expression with parenthesis gives an unclosed error, how to fix?

(point_granularity > 2 and eq_site_limit == 0) and (point_latitude < 29)

This expression gives an error because the beginning of the expression is a parenthesis, is there a workaround for this, is this intended? Any advice to get it to parse the first expression in this line as a compound like it usually would?

Add repo as lib in package.json

I need help about how to add this repo as lib in package.json, i'm not clear in the path to add this lib
example:

"devDependencies": {
"jsep":"git+https://github.com/soney/jsep.git"
}
this show error

[1,2,3][4] returns Compound instead of MemberExpression

When using [1,2,3] the parser responds with an ArrayExpression. When using Identifier[4] the parser reacts with a MemberExpression. When using [1,2,3][4] the parser assumes a Compound with two ArrayExpressions in the Body. Shouldn't it assume a MemberExpression instead? This becomes problematic when having [1,2,3][4][5].

The order of execution of arithmetic operations

1 * 2 + 3 != 1 * (2 + 3)

type: 'op'
subtype: 'binary'
op: '*'
operands:
    0:
        type: 'constant'
        text: '1'
        value: 1
    1:
        type: 'op'
        subtype: 'binary'
        op: '+'
        operands:
            0:
                type: 'constant'
                text: '2'
                value: 2
            1:
                type: 'constant'
                text: '3'
                value: 3

Unary operators cannot look like identifiers

I need "not" as a unary operator, but jsep's gobbleToken interprets this as an identifier, before it has a chance to interpret it as a unary operator.

This is fixed in my fork here https://github.com/alexwarren/jsep which also includes @canac's fix from #22.

Set parser to vanilla state

Hi! I'd want to make use of jsep as a simple expression parser for one of my libraries. However, jsep by default includes support for all javascript operators (like === ) which I don't plan to support. Could you please add a quick way to get a vanilla parser from jsep? (something like a reset method)

MemberExpression cannot differentiate between property vs dot access

Given these two expressions they both evaluate to the same result nodes which they are obviously very different.

>>> JSON.stringify(jsep("one.[test]"), null, 4);
{
    "type": "MemberExpression",
    "computed": true,
    "object": {
        "type": "Identifier",
        "name": "one"
    },
    "property": {
        "type": "Identifier",
        "name": "test"
    }
} 

>>> JSON.stringify(jsep("one.test"), null, 4);
{
    "type": "MemberExpression",
    "computed": false,
    "object": {
        "type": "Identifier",
        "name": "one"
    },
    "property": {
        "type": "Identifier",  // Should be a literal
        "name": "test"
    }
} 

The dot access method should be the same as quoted property access with the property being a literal and not an identifier. Also some note of which access method was used would be nice

>>> JSON.stringify(jsep("one['test']"), null, 4);
"{
    "type": "MemberExpression",
    "computed": true,
    "object": {
        "type": "Identifier",
        "name": "one"
    },
    "property": {
        "type": "Literal",
        "value": "test",
        "raw": "'test'"
    }
}"

Support unitary operator with multiple letters

jsep("x && !y") generates the correct AST:

{ type: 'LogicalExpression',
  operator: '&&',
  left: { type: 'Identifier', name: 'x' },
  right: 
   { type: 'UnaryExpression',
     operator: '!',
     argument: { type: 'Identifier', name: 'y' },
     prefix: true } }

If I add the new operators AND and NOT, the unitary operator is not correctly parsed

jsep.addBinaryOp("AND", 3);
jsep.addUnaryOp('NOT', 1);
jsep("x AND NOT y")

result in

{ type: 'Compound',
  body: 
   [ { type: 'BinaryExpression',
       operator: 'AND',
       left: { type: 'Identifier', name: 'x' },
       right: { type: 'Identifier', name: 'NOT' } },
     { type: 'Identifier', name: 'y' } ] }

Is there a way to support this?

(1, 1) produces error: Unclosed ( at character 6

Given that 1, 1 by itself is parsed as a Compound, it seems a little surprising that wrapping it with parentheses produces an error, what do you think @EricSmekens?

foo(bar is not marked invalid

Sorry for maybe stupid issue.
Why does "foo(bar" expression not marked invalid for unclosed bracket?

feature request: Support Unicode identifiers

According to the ECMA specification, Identifiers can also contain Unicode characters

Could there be a more token type as `Accessor` (or other) than `Identifier`?

Sometimes I need to identify whether a identifier is a global variable which could access its closure or not. For example code below:

a[b.c]

In this expression, identifier a and b are variable start which indicate some value in closure, but c is just an alias of a string literal. I think they should be a little difference.

I found it can be implemented by only one line addition at line 466:

node = gobbleIdentifier();
if (node.type != THIS_EXP) node.type = ACCESSOR; // could be defined in header and could be other word

I could send a PR if you like this feature.

Support ternary operator