Giter Site home page Giter Site logo

micromatch / braces Goto Github PK

View Code? Open in Web Editor NEW
216.0 11.0 62.0 396 KB

Faster brace expansion for node.js. Besides being faster, braces is not subject to DoS attacks like minimatch, is more accurate, and has more complete support for Bash 4.3.

Home Page: https://github.com/jonschlinkert

License: MIT License

JavaScript 100.00%
glob-pattern expansion braces minimatch multimatch brace-patterns brace-expansion glob-matcher regular-expression glob

braces's Introduction

braces Donate NPM version NPM monthly downloads NPM total downloads

Bash-like brace expansion, implemented in JavaScript. Safer than other brace expansion libs, with complete support for the Bash 4.3 braces specification, without sacrificing speed.

Please consider following this project's author, Jon Schlinkert, and consider starring the project to show your ❤️ and support.

Install

Install with npm:

$ npm install --save braces

v3.0.0 Released!!

See the changelog for details.

Why use braces?

Brace patterns make globs more powerful by adding the ability to match specific ranges and sequences of characters.

  • Accurate - complete support for the Bash 4.3 Brace Expansion specification (passes all of the Bash braces tests)
  • fast and performant - Starts fast, runs fast and scales well as patterns increase in complexity.
  • Organized code base - The parser and compiler are easy to maintain and update when edge cases crop up.
  • Well-tested - Thousands of test assertions, and passes all of the Bash, minimatch, and brace-expansion unit tests (as of the date this was written).
  • Safer - You shouldn't have to worry about users defining aggressive or malicious brace patterns that can break your application. Braces takes measures to prevent malicious regex that can be used for DDoS attacks (see catastrophic backtracking).
  • Supports lists - (aka "sets") a/{b,c}/d => ['a/b/d', 'a/c/d']
  • Supports sequences - (aka "ranges") {01..03} => ['01', '02', '03']
  • Supports steps - (aka "increments") {2..10..2} => ['2', '4', '6', '8', '10']
  • Supports escaping - To prevent evaluation of special characters.

Usage

The main export is a function that takes one or more brace patterns and options.

const braces = require('braces');
// braces(patterns[, options]);

console.log(braces(['{01..05}', '{a..e}']));
//=> ['(0[1-5])', '([a-e])']

console.log(braces(['{01..05}', '{a..e}'], { expand: true }));
//=> ['01', '02', '03', '04', '05', 'a', 'b', 'c', 'd', 'e']

Brace Expansion vs. Compilation

By default, brace patterns are compiled into strings that are optimized for creating regular expressions and matching.

Compiled

console.log(braces('a/{x,y,z}/b'));
//=> ['a/(x|y|z)/b']
console.log(braces(['a/{01..20}/b', 'a/{1..5}/b']));
//=> [ 'a/(0[1-9]|1[0-9]|20)/b', 'a/([1-5])/b' ]

Expanded

Enable brace expansion by setting the expand option to true, or by using braces.expand() (returns an array similar to what you'd expect from Bash, or echo {1..5}, or minimatch):

console.log(braces('a/{x,y,z}/b', { expand: true }));
//=> ['a/x/b', 'a/y/b', 'a/z/b']

console.log(braces.expand('{01..10}'));
//=> ['01','02','03','04','05','06','07','08','09','10']

Lists

Expand lists (like Bash "sets"):

console.log(braces('a/{foo,bar,baz}/*.js'));
//=> ['a/(foo|bar|baz)/*.js']

console.log(braces.expand('a/{foo,bar,baz}/*.js'));
//=> ['a/foo/*.js', 'a/bar/*.js', 'a/baz/*.js']

Sequences

Expand ranges of characters (like Bash "sequences"):

console.log(braces.expand('{1..3}')); // ['1', '2', '3']
console.log(braces.expand('a/{1..3}/b')); // ['a/1/b', 'a/2/b', 'a/3/b']
console.log(braces('{a..c}', { expand: true })); // ['a', 'b', 'c']
console.log(braces('foo/{a..c}', { expand: true })); // ['foo/a', 'foo/b', 'foo/c']

// supports zero-padded ranges
console.log(braces('a/{01..03}/b')); //=> ['a/(0[1-3])/b']
console.log(braces('a/{001..300}/b')); //=> ['a/(0{2}[1-9]|0[1-9][0-9]|[12][0-9]{2}|300)/b']

See fill-range for all available range-expansion options.

Steppped ranges

Steps, or increments, may be used with ranges:

console.log(braces.expand('{2..10..2}'));
//=> ['2', '4', '6', '8', '10']

console.log(braces('{2..10..2}'));
//=> ['(2|4|6|8|10)']

When the .optimize method is used, or options.optimize is set to true, sequences are passed to to-regex-range for expansion.

Nesting

Brace patterns may be nested. The results of each expanded string are not sorted, and left to right order is preserved.

"Expanded" braces

console.log(braces.expand('a{b,c,/{x,y}}/e'));
//=> ['ab/e', 'ac/e', 'a/x/e', 'a/y/e']

console.log(braces.expand('a/{x,{1..5},y}/c'));
//=> ['a/x/c', 'a/1/c', 'a/2/c', 'a/3/c', 'a/4/c', 'a/5/c', 'a/y/c']

"Optimized" braces

console.log(braces('a{b,c,/{x,y}}/e'));
//=> ['a(b|c|/(x|y))/e']

console.log(braces('a/{x,{1..5},y}/c'));
//=> ['a/(x|([1-5])|y)/c']

Escaping

Escaping braces

A brace pattern will not be expanded or evaluted if either the opening or closing brace is escaped:

console.log(braces.expand('a\\{d,c,b}e'));
//=> ['a{d,c,b}e']

console.log(braces.expand('a{d,c,b\\}e'));
//=> ['a{d,c,b}e']

Escaping commas

Commas inside braces may also be escaped:

console.log(braces.expand('a{b\\,c}d'));
//=> ['a{b,c}d']

console.log(braces.expand('a{d\\,c,b}e'));
//=> ['ad,ce', 'abe']

Single items

Following bash conventions, a brace pattern is also not expanded when it contains a single character:

console.log(braces.expand('a{b}c'));
//=> ['a{b}c']

Options

options.maxLength

Type: Number

Default: 10,000

Description: Limit the length of the input string. Useful when the input string is generated or your application allows users to pass a string, et cetera.

console.log(braces('a/{b,c}/d', { maxLength: 3 })); //=> throws an error

options.expand

Type: Boolean

Default: undefined

Description: Generate an "expanded" brace pattern (alternatively you can use the braces.expand() method, which does the same thing).

console.log(braces('a/{b,c}/d', { expand: true }));
//=> [ 'a/b/d', 'a/c/d' ]

options.nodupes

Type: Boolean

Default: undefined

Description: Remove duplicates from the returned array.

options.rangeLimit

Type: Number

Default: 1000

Description: To prevent malicious patterns from being passed by users, an error is thrown when braces.expand() is used or options.expand is true and the generated range will exceed the rangeLimit.

You can customize options.rangeLimit or set it to Inifinity to disable this altogether.

Examples

// pattern exceeds the "rangeLimit", so it's optimized automatically
console.log(braces.expand('{1..1000}'));
//=> ['([1-9]|[1-9][0-9]{1,2}|1000)']

// pattern does not exceed "rangeLimit", so it's NOT optimized
console.log(braces.expand('{1..100}'));
//=> ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '60', '61', '62', '63', '64', '65', '66', '67', '68', '69', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91', '92', '93', '94', '95', '96', '97', '98', '99', '100']

options.transform

Type: Function

Default: undefined

Description: Customize range expansion.

Example: Transforming non-numeric values

const alpha = braces.expand('x/{a..e}/y', {
  transform(value, index) {
    // When non-numeric values are passed, "value" is a character code.
    return 'foo/' + String.fromCharCode(value) + '-' + index;
  },
});
console.log(alpha);
//=> [ 'x/foo/a-0/y', 'x/foo/b-1/y', 'x/foo/c-2/y', 'x/foo/d-3/y', 'x/foo/e-4/y' ]

Example: Transforming numeric values

const numeric = braces.expand('{1..5}', {
  transform(value) {
    // when numeric values are passed, "value" is a number
    return 'foo/' + value * 2;
  },
});
console.log(numeric);
//=> [ 'foo/2', 'foo/4', 'foo/6', 'foo/8', 'foo/10' ]

options.quantifiers

Type: Boolean

Default: undefined

Description: In regular expressions, quanitifiers can be used to specify how many times a token can be repeated. For example, a{1,3} will match the letter a one to three times.

Unfortunately, regex quantifiers happen to share the same syntax as Bash lists

The quantifiers option tells braces to detect when regex quantifiers are defined in the given pattern, and not to try to expand them as lists.

Examples

const braces = require('braces');
console.log(braces('a/b{1,3}/{x,y,z}'));
//=> [ 'a/b(1|3)/(x|y|z)' ]
console.log(braces('a/b{1,3}/{x,y,z}', { quantifiers: true }));
//=> [ 'a/b{1,3}/(x|y|z)' ]
console.log(braces('a/b{1,3}/{x,y,z}', { quantifiers: true, expand: true }));
//=> [ 'a/b{1,3}/x', 'a/b{1,3}/y', 'a/b{1,3}/z' ]

options.keepEscaping

Type: Boolean

Default: undefined

Description: Do not strip backslashes that were used for escaping from the result.

What is "brace expansion"?

Brace expansion is a type of parameter expansion that was made popular by unix shells for generating lists of strings, as well as regex-like matching when used alongside wildcards (globs).

In addition to "expansion", braces are also used for matching. In other words:

More about brace expansion (click to expand)

There are two main types of brace expansion:

  1. lists: which are defined using comma-separated values inside curly braces: {a,b,c}
  2. sequences: which are defined using a starting value and an ending value, separated by two dots: a{1..3}b. Optionally, a third argument may be passed to define a "step" or increment to use: a{1..100..10}b. These are also sometimes referred to as "ranges".

Here are some example brace patterns to illustrate how they work:

Sets

{a,b,c}       => a b c
{a,b,c}{1,2}  => a1 a2 b1 b2 c1 c2

Sequences

{1..9}        => 1 2 3 4 5 6 7 8 9
{4..-4}       => 4 3 2 1 0 -1 -2 -3 -4
{1..20..3}    => 1 4 7 10 13 16 19
{a..j}        => a b c d e f g h i j
{j..a}        => j i h g f e d c b a
{a..z..3}     => a d g j m p s v y

Combination

Sets and sequences can be mixed together or used along with any other strings.

{a,b,c}{1..3}   => a1 a2 a3 b1 b2 b3 c1 c2 c3
foo/{a,b,c}/bar => foo/a/bar foo/b/bar foo/c/bar

The fact that braces can be "expanded" from relatively simple patterns makes them ideal for quickly generating test fixtures, file paths, and similar use cases.

Brace matching

In addition to expansion, brace patterns are also useful for performing regular-expression-like matching.

For example, the pattern foo/{1..3}/bar would match any of following strings:

foo/1/bar
foo/2/bar
foo/3/bar

But not:

baz/1/qux
baz/2/qux
baz/3/qux

Braces can also be combined with glob patterns to perform more advanced wildcard matching. For example, the pattern */{1..3}/* would match any of following strings:

foo/1/bar
foo/2/bar
foo/3/bar
baz/1/qux
baz/2/qux
baz/3/qux

Brace matching pitfalls

Although brace patterns offer a user-friendly way of matching ranges or sets of strings, there are also some major disadvantages and potential risks you should be aware of.

tldr

"brace bombs"

  • brace expansion can eat up a huge amount of processing resources
  • as brace patterns increase linearly in size, the system resources required to expand the pattern increase exponentially
  • users can accidentally (or intentially) exhaust your system's resources resulting in the equivalent of a DoS attack (bonus: no programming knowledge is required!)

For a more detailed explanation with examples, see the geometric complexity section.

The solution

Jump to the performance section to see how Braces solves this problem in comparison to other libraries.

Geometric complexity

At minimum, brace patterns with sets limited to two elements have quadradic or O(n^2) complexity. But the complexity of the algorithm increases exponentially as the number of sets, and elements per set, increases, which is O(n^c).

For example, the following sets demonstrate quadratic (O(n^2)) complexity:

{1,2}{3,4}      => (2X2)    => 13 14 23 24
{1,2}{3,4}{5,6} => (2X2X2)  => 135 136 145 146 235 236 245 246

But add an element to a set, and we get a n-fold Cartesian product with O(n^c) complexity:

{1,2,3}{4,5,6}{7,8,9} => (3X3X3) => 147 148 149 157 158 159 167 168 169 247 248
                                    249 257 258 259 267 268 269 347 348 349 357
                                    358 359 367 368 369

Now, imagine how this complexity grows given that each element is a n-tuple:

{1..100}{1..100}         => (100X100)     => 10,000 elements (38.4 kB)
{1..100}{1..100}{1..100} => (100X100X100) => 1,000,000 elements (5.76 MB)

Although these examples are clearly contrived, they demonstrate how brace patterns can quickly grow out of control.

More information

Interested in learning more about brace expansion?

Performance

Braces is not only screaming fast, it's also more accurate the other brace expansion libraries.

Better algorithms

Fortunately there is a solution to the "brace bomb" problem: don't expand brace patterns into an array when they're used for matching.

Instead, convert the pattern into an optimized regular expression. This is easier said than done, and braces is the only library that does this currently.

The proof is in the numbers

Minimatch gets exponentially slower as patterns increase in complexity, braces does not. The following results were generated using braces() and minimatch.braceExpand(), respectively.

Pattern braces [minimatch][]
{1..9007199254740991}[^1] 298 B (5ms 459μs) N/A (freezes)
{1..1000000000000000} 41 B (1ms 15μs) N/A (freezes)
{1..100000000000000} 40 B (890μs) N/A (freezes)
{1..10000000000000} 39 B (2ms 49μs) N/A (freezes)
{1..1000000000000} 38 B (608μs) N/A (freezes)
{1..100000000000} 37 B (397μs) N/A (freezes)
{1..10000000000} 35 B (983μs) N/A (freezes)
{1..1000000000} 34 B (798μs) N/A (freezes)
{1..100000000} 33 B (733μs) N/A (freezes)
{1..10000000} 32 B (5ms 632μs) 78.89 MB (16s 388ms 569μs)
{1..1000000} 31 B (1ms 381μs) 6.89 MB (1s 496ms 887μs)
{1..100000} 30 B (950μs) 588.89 kB (146ms 921μs)
{1..10000} 29 B (1ms 114μs) 48.89 kB (14ms 187μs)
{1..1000} 28 B (760μs) 3.89 kB (1ms 453μs)
{1..100} 22 B (345μs) 291 B (196μs)
{1..10} 10 B (533μs) 20 B (37μs)
{1..3} 7 B (190μs) 5 B (27μs)

Faster algorithms

When you need expansion, braces is still much faster.

(the following results were generated using braces.expand() and minimatch.braceExpand(), respectively)

Pattern braces [minimatch][]
{1..10000000} 78.89 MB (2s 698ms 642μs) 78.89 MB (18s 601ms 974μs)
{1..1000000} 6.89 MB (458ms 576μs) 6.89 MB (1s 491ms 621μs)
{1..100000} 588.89 kB (20ms 728μs) 588.89 kB (156ms 919μs)
{1..10000} 48.89 kB (2ms 202μs) 48.89 kB (13ms 641μs)
{1..1000} 3.89 kB (1ms 796μs) 3.89 kB (1ms 958μs)
{1..100} 291 B (424μs) 291 B (211μs)
{1..10} 20 B (487μs) 20 B (72μs)
{1..3} 5 B (166μs) 5 B (27μs)

If you'd like to run these comparisons yourself, see test/support/generate.js.

Benchmarks

Running benchmarks

Install dev dependencies:

npm i -d && npm benchmark

Latest results

Braces is more accurate, without sacrificing performance.

● expand - range (expanded)
     braces x 53,167 ops/sec ±0.12% (102 runs sampled)
  minimatch x 11,378 ops/sec ±0.10% (102 runs sampled)
● expand - range (optimized for regex)
     braces x 373,442 ops/sec ±0.04% (100 runs sampled)
  minimatch x 3,262 ops/sec ±0.18% (100 runs sampled)
● expand - nested ranges (expanded)
     braces x 33,921 ops/sec ±0.09% (99 runs sampled)
  minimatch x 10,855 ops/sec ±0.28% (100 runs sampled)
● expand - nested ranges (optimized for regex)
     braces x 287,479 ops/sec ±0.52% (98 runs sampled)
  minimatch x 3,219 ops/sec ±0.28% (101 runs sampled)
● expand - set (expanded)
     braces x 238,243 ops/sec ±0.19% (97 runs sampled)
  minimatch x 538,268 ops/sec ±0.31% (96 runs sampled)
● expand - set (optimized for regex)
     braces x 321,844 ops/sec ±0.10% (97 runs sampled)
  minimatch x 140,600 ops/sec ±0.15% (100 runs sampled)
● expand - nested sets (expanded)
     braces x 165,371 ops/sec ±0.42% (96 runs sampled)
  minimatch x 337,720 ops/sec ±0.28% (100 runs sampled)
● expand - nested sets (optimized for regex)
     braces x 242,948 ops/sec ±0.12% (99 runs sampled)
  minimatch x 87,403 ops/sec ±0.79% (96 runs sampled)

About

Contributing

Pull requests and stars are always welcome. For bugs and feature requests, please create an issue.

Running Tests

Running and reviewing unit tests is a great way to get familiarized with a library and its API. You can install dependencies and run tests with the following command:

$ npm install && npm test
Building docs

(This project's readme.md is generated by verb, please don't edit the readme directly. Any changes to the readme must be made in the .verb.md readme template.)

To generate the readme, run the following command:

$ npm install -g verbose/verb#dev verb-generate-readme && verb

Contributors

Commits Contributor
197 jonschlinkert
4 doowb
1 es128
1 eush77
1 hemanth
1 wtgtybhertgeghgtwtg

Author

Jon Schlinkert

License

Copyright © 2019, Jon Schlinkert. Released under the MIT License.


This file was generated by verb-generate-readme, v0.8.0, on April 08, 2019.

braces's People

Contributors

aaronmoat avatar coderaiser avatar doowb avatar es128 avatar eush77 avatar hemanth avatar jonschlinkert avatar koopero avatar paulmillr avatar vemoo avatar wtgtybhertgeghgtwtg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

braces's Issues

Expand-braces test cases failed when upgrading braces from v1.8.5 to v2.3.2

First, Travis log from micromatch/expand-braces#2

 1) given an array of patterns with braces
       braces
         should work with one value:
      AssertionError: expected [ 'a{b}c', 'a/b/c{d}e' ] to equal [ 'abc', 'a/b/cde' ] (at '0', A has 'a{b}c' and B has 'abc')
      + expected - actual
       [
      -  "a{b}c"
      -  "a/b/c{d}e"
      +  "abc"
      +  "a/b/cde"
       ]
      
      at Proxy.fail (node_modules/should/lib/assertion.js:228:17)
      at Proxy.prop.(anonymous function) (node_modules/should/lib/assertion.js:69:14)
      at Context.<anonymous> (test.js:32:45)
  2) given an array of patterns with braces
       braces
         should work with nested non-sets:
      AssertionError: expected [ '{a-b}', '{a-c}', '{a-d}', 'a' ] to equal [ 'a-b', 'a-c', 'a-d', 'a' ] (at '0', A has '{a-b}' and B has 'a-b')
      + expected - actual
       [
      -  "{a-b}"
      -  "{a-c}"
      -  "{a-d}"
      +  "a-b"
      +  "a-c"
      +  "a-d"
         "a"
       ]
      
      at Proxy.fail (node_modules/should/lib/assertion.js:228:17)
      at Proxy.prop.(anonymous function) (node_modules/should/lib/assertion.js:69:14)
      at Context.<anonymous> (test.js:36:57)
  3) given an array of patterns with braces
       braces
         should expand sets:
      AssertionError: expected [ 'a/x/c{d}e', 'a/y/c{d}e', 'a/b/c/x', 'a/b/c/y' ] to equal [ 'a/x/cde', 'a/y/cde', 'a/b/c/x', 'a/b/c/y' ] (at '0', A has 'a/x/c{d}e' and B has 'a/x/cde')
      + expected - actual
       [
      -  "a/x/c{d}e"
      -  "a/y/c{d}e"
      +  "a/x/cde"
      +  "a/y/cde"
         "a/b/c/x"
         "a/b/c/y"
       ]
      
      at Proxy.fail (node_modules/should/lib/assertion.js:228:17)
      at Proxy.prop.(anonymous function) (node_modules/should/lib/assertion.js:69:14)
      at Context.<anonymous> (test.js:44:55)
  4) given an array of patterns with braces
       braces
         should throw an error when imbalanced braces are found.:
     AssertionError: expected [Function] to throw exception
      at Proxy.fail (node_modules/should/lib/assertion.js:228:17)
      at Proxy.prop.(anonymous function) (node_modules/should/lib/assertion.js:69:14)
      at Context.<anonymous> (test.js:50:22)
  5) given an array of patterns with braces
       should expand a combination of nested sets and ranges.
         should expand sets:
      AssertionError: expected [
  'a/x/c{d}e',
  'a/1/c{d}e',
  'a/2/c{d}e',
  'a/3/c{d}e',
  'a/4/c{d}e',
  'a/5/c{d}e',
  'a/y/c{d}e'
] to equal [
  'a/x/cde',
  'a/1/cde',
  'a/y/cde',
  'a/2/cde',
  'a/3/cde',
  'a/4/cde',
  'a/5/cde'
] (at '0', A has 'a/x/c{d}e' and B has 'a/x/cde')
      + expected - actual
       [
      -  "a/x/c{d}e"
      -  "a/1/c{d}e"
      -  "a/2/c{d}e"
      -  "a/3/c{d}e"
      -  "a/4/c{d}e"
      -  "a/5/c{d}e"
      -  "a/y/c{d}e"
      +  "a/x/cde"
      +  "a/1/cde"
      +  "a/y/cde"
      +  "a/2/cde"
      +  "a/3/cde"
      +  "a/4/cde"
      +  "a/5/cde"
       ]
      
      at Proxy.fail (node_modules/should/lib/assertion.js:228:17)
      at Proxy.prop.(anonymous function) (node_modules/should/lib/assertion.js:69:14)
      at Context.<anonymous> (test.js:85:47)

I think for braces expand with single item,
in v 1.8.5, default is

braces('a{b}c');
//=> ['abc']

in v2.3.2, the default

console.log(braces.expand('a{b}c'));
//=> ['a{b}c']

That's the reason why the cases failed

but @jonschlinkert said that this is unrelated, so could you elaborate a bit? thanks

question

Snyk says this vulnerability has been fixed....

Zero-padding not supported in compiled output

It appears that zero-padded numeric sequences are advertised, but not properly supported.

// Example from README.md
console.log(braces('a/{001..300}/b')); //=> ['a/(0{2}[1-9]|0[1-9][0-9]|[12][0-9]{2}|300)/b']

// Actual output is:
[ 'a/(0{0,2}[1-9]|0?[1-9][0-9]|[12][0-9]{2}|300)/b' ]

Will follow up with PR with failing test cases.

Please advice whether the fix for this is in this module, or upstream in to-regex-range.

As always, thanks for you hard work on this module and so many others.

please update snapdragon dependency to 0.11.0

I'd like to use snapdragon 0.11.0 in debian but when using that version, lots of tests are failing with errors.

823) range expansion should expand negative ranges using steps::
     TypeError: Cannot read property 'length' of undefined
      at Compiler.mapVisit (node_modules/snapdragon/lib/compiler.js:228:31)
      at Compiler.<anonymous> (lib/compilers.js:55:19)
      at Compiler.visit (node_modules/snapdragon/lib/compiler.js:200:18)
      at Compiler.mapVisit (node_modules/snapdragon/lib/compiler.js:231:23)
      at Compiler.compile (node_modules/snapdragon/lib/compiler.js:263:12)
      at Snapdragon.compile (node_modules/snapdragon/index.js:131:24)
      at Braces.compile (lib/braces.js:87:29)
      at Braces.expand (lib/braces.js:97:15)
      at create (index.js:142:15)
      at memoize (index.js:293:13)
      at Function.braces.create (index.js:161:10)
      at braces.expand (index.js:82:17)
      at equal (test/regression-1.8.5.js:12:16)
      at Context.<anonymous> (test/regression-1.8.5.js:436:5)

fails test on fresh checkout and npm i

$ npm t

> [email protected] test /Users/isaacs/dev/js/minimatch/braces
> mocha -R spec



  braces
    brace expansion
      ✓ should work with no braces
      ✓ should work with one value
      ✓ should work with nested non-sets
      ✓ should work with commas.
      ✓ should expand sets
      ✓ should expand multiple sets
      ✓ should expand nested sets
      ✓ should expand with globs.
      ✓ should throw an error when imbalanced braces are found.
    range expansion
      ✓ should expand numerical ranges
      ✓ should honor padding
      ✓ should expand alphabetical ranges
      1) should use a custom function for expansions.
    complex
      ✓ should expand a complex combination of ranges and sets:
      ✓ should expand a combination of nested sets and ranges:


  14 passing (13ms)
  1 failing

  1) braces range expansion should use a custom function for expansions.:

      AssertionError: expected [ '\u000097', '\u000098', '\u000099', '\u0000100', '\u0000101' ] to equal [ 'a0', 'b1', 'c2', 'd3', 'e4' ] (at '0', A has '\u000097' and B has 'a0')
      + expected - actual

       [
      +  "a0"
      +  "b1"
      +  "c2"
      +  "d3"
      +  "e4"
      -  "\u000097"
      -  "\u000098"
      -  "\u000099"
      -  "\u0000100"
      -  "\u0000101"
       ]

      at Assertion.fail (/Users/isaacs/dev/js/minimatch/braces/node_modules/should/lib/assertion.js:113:17)
      at Assertion.prop.(anonymous function) [as eql] (/Users/isaacs/dev/js/minimatch/braces/node_modules/should/lib/assertion.js:39:14)
      at Context.<anonymous> (/Users/isaacs/dev/js/minimatch/braces/test.js:87:20)
      at callFn (/Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runnable.js:250:21)
      at Test.Runnable.run (/Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runnable.js:243:7)
      at Runner.runTest (/Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runner.js:373:10)
      at /Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runner.js:451:12
      at next (/Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runner.js:298:14)
      at /Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runner.js:308:7
      at next (/Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runner.js:246:23)
      at Object._onImmediate (/Users/isaacs/dev/js/minimatch/braces/node_modules/mocha/lib/runner.js:275:5)
      at processImmediate [as _immediateCallback] (timers.js:330:15)



npm ERR! Test failed.  See above for more details.

quotes

Brace patterns within quoted sections should not be expanded.

$ echo "{a,b}{{a,b},a,b}"{x,y}
{a,b}{{a,b},a,b}x {a,b}{{a,b},a,b}y

$ node -p 'require("./")("\"{a,b}{{a,b},a,b}\"{x,y}")'
[ '"aa"x',
  '"ab"x',
  '"ba"x',
  '"bb"x',
  '"aa"y',
  '"ab"y',
  '"ba"y',
  '"bb"y' ]

CWE-400 | Uncontrolled resource consumption

Ran latest scan and landed with this issue - CWE-400 | Uncontrolled resource consumption

The NPM package "braces" fails to limit the number of characters it can handle, which could lead to Memory Exhaustion. In "lib/parse.js," if a malicious user sends "imbalanced braces" as input, the parsing will enter a loop, which will cause the program to start allocating heap memory without freeing it at any moment of the loop. Eventually, the JavaScript heap limit is reached, and the program will crash.

braces/lib/parse.js on line 308 - do {

Dot character not working as it should (escape + expansion)

Input

{1\\.2}

Output

{1\\.2}

Expected

{1.2}

I wanted to escape the dot to get this to work:

{{a,b,c},foo.bar/{a,b,c}}/**/*.baz

Output

[ 'c', 'foo.bar/c' ]

Expected

[ 'a/**/*.baz',
  'foo.bar/a/**/*.baz',
  'foo.bar/b/**/*.baz',
  'foo.bar/c/**/*.baz',
  'b/**/*.baz',
  'c/**/*.baz' ]

But why should I escape the dot? As far as I know, it doesn't carry any particular meaning in glob patterns...

Fails to expand Windows paths with brackets

> const p2 = 'C:\\Users\\foo\\Workspace\\project\\routes\\[locale]'
> braces.expand(p2)
Error: no parsers registered for: "]"
    at parse (C:\Users\foo\Workspace\project\node_modules\snapdragon\lib\parser.js:475:15)

Braces may be vulnerable to DoS attack through snapdragon

I see this dependency chain in my package-lock.json
http-proxy-middleware > braces > snapdragon > source-map-resolve > decode-uri-component

decode-uri-component has a security vulnerability CVE-2022-38900 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-38900

decode-uri-component is used by the deprecated source-map-resolve which will not fix the problem. lydell/source-map-resolve#36

snapdragon uses source-map-resolve and hasn't addressed the issue: here-be/snapdragon#31

Therefore I'm posting here. Is it possible to remove snapdragon as a dependency of braces?

Update split-string

Please update split-string from 3.0.2 to 6.0.0

according to its changelog, the keepQuotes option has been removed

updating this dependency breaks most tests so I guess that you need tweak some code, but looking at lib/utils.js it's not obvious to me what should be changed

Request: Remove or make optional result property on output array

Currently, after doing an expansion, you attach the result object returned from the compiler on the array object. This then gets put into the cache object which takes up a significant amount of memory, particularly when the cache grows large, as it creates a new compiler for each call which is cached, leading to a ~5/6mb overhead every time.

This result property does not appear to be documented behaviour and after a quick github code search, I couldn't find any references to it. Could you either remove this property, make it an optional addition, or suggest a third option to help mitigate the memory usage?

I am currently using chokidar to watch media files for changes, on a folder with ~50k items in it. This leads to the cache object holding ~300mb of memory at ~60% of my total application's memory usage.

Example program to try to demonstrate the issue:

var chokidar = require('chokidar');
var braces = require('braces');

console.log("Test starting");
chokidar.watch('some/folder/with/many/files/*', {}).on('ready', (event, path) => {
  console.log("chokidar ready");

  if (global.gc) { global.gc(); }
  let memoryUsage = process.memoryUsage();
  console.log(`Before clear: ${memoryUsage.heapUsed}/${memoryUsage.heapTotal}`);

  braces.clearCache();

  if (global.gc) { global.gc(); }
  memoryUsage = process.memoryUsage();
  console.log(`After clear: ${memoryUsage.heapUsed}/${memoryUsage.heapTotal}`);

  process.exit();
});

Running the above using node --expose-gc bracesTest.js on my testing file system gives the following output:

Test starting
chokidar ready
Before clear: 446778640/585011200
After clear: 133833408/241065984

My testing system is created using symlinked media files into a single folder with the following script:

#!/bin/bash

for i in {0..50000} 
do
	ln -s ../MediaFile.mp4 clip_$i.mp4
done

Screenshot of the memory snapshot that led me to the cache:
image

Screenshot of the memory snapshot after adding a .slice() call to the memoize method when adding an item to the cache:
image

Braces don't expand if they start with the range operator

Example:

braces.expand('{..a,b}/c')

Expected result:

[ '..ac', 'bc' ]

Actual result:

[ '{..a,b}c' ]

Note that the current behaviour is differing from the bash implementation:

tim@EK-22-LT-04:~/virtuoso-test-data$ echo {..a,b}c
..ac bc

Note that it does work if the .. comes anywhere except the first parameter to the brace, so

'{b,..a}c'
{b,z,..a}c

do each give

[ 'bc', '..ac' ]
[ 'bc', 'zc', '..ac' ]

changelog

I totally meant to do this before publishing 2.0 and forgot.

Braces with a single value are not expanded

Hello,

I'm not sure if this is a feature or not, but braces containing only a single value do not get expanded properly.

Two values work fine:

❯ node -e "console.log(require('braces')('{x,y}', { expand:true }))"
[ 'x', 'y' ]

However, only a single value doesn't:

❯ node -e "console.log(require('braces')('{x}', { expand:true }))" 
[ '{x}' ]

I would expect it to instead return:

❯ node -e "console.log(require('braces')('{x}', { expand:true }))" 
[ 'x' ]

Should this be using `Array.prototype.flat()` instead of `flatten` from /lib/utils

Just tested performance and seems Array.flat has better performance until a certain point, but then your implementation pulls ahead.

Array size = 1,000, depth = 5
% node testFlatPerf.js
Custom Flatten Time: 0.14 ms
Built-in Flatten Time: 0.12 ms

size = 10,000, depth = 5
% node testFlatPerf.js
Custom Flatten Time: 2.35 ms
Built-in Flatten Time: 1.02 ms

-- Performance change over somewhere in here --

size = 100,000, depth = 5
% node testFlatPerf.js
Custom Flatten Time: 14.81 ms
Built-in Flatten Time: 79.13 ms

size = 1,000,0000, depth = 5
% node testFlatPerf.js
Custom Flatten Time: 127.74 ms
Built-in Flatten Time: 768.20 ms

size = 100,000, depth = 1000
% node testFlatPerf.js
Custom Flatten Time: 4.22 ms
Built-in Flatten Time: 9.88 ms

Things come apart when I really ramped up the depth for both
size = 100,000, depth = 10,000
node testFlatPerf.js
/Users/mike/Desktop/testFlatPerf.js:18
if (Array.isArray(ele)) {
^

Here's the code I used to test

const { performance } = require('perf_hooks');

// Function to generate a large nested array for testing
const generateNestedArray = (size, depth) => {
  let array = Array(size).fill(1);
  for (let i = 0; i < depth; i++) {
    array = [array];
  }
  return array;
};

// Custom recursive flatten function
const customFlatten = (...args) => {
  const result = [];
  const flat = arr => {
    for (let i = 0; i < arr.length; i++) {
      const ele = arr[i];
      if (Array.isArray(ele)) {
        flat(ele);
        continue;
      }
      if (ele !== undefined) {
        result.push(ele);
      }
    }
    return result;
  };
  flat(args);
  return result;
};

// Built-in flat function
const builtInFlatten = (...args) => args.flat(Infinity);

// Test arrays
const testArray = generateNestedArray(1000, 5);

// Benchmark customFlatten
let start = performance.now();
customFlatten(testArray);
let customFlattenTime = performance.now() - start;

console.log(`Custom Flatten Time: ${customFlattenTime.toFixed(2)} ms`);

// Benchmark builtInFlatten
start = performance.now();
builtInFlatten(testArray);
let builtInFlattenTime = performance.now() - start;

console.log(`Built-in Flatten Time: ${builtInFlattenTime.toFixed(2)} ms`);

3.0.1 has .DS_Store inside

node_modules/braces
node_modules/braces/LICENSE
node_modules/braces/CHANGELOG.md
node_modules/braces/index.js
node_modules/braces/README.md
node_modules/braces/package.json
node_modules/braces/lib
node_modules/braces/lib/constants.js
node_modules/braces/lib/stringify.js
node_modules/braces/lib/.DS_Store
node_modules/braces/lib/parse.js
node_modules/braces/lib/expand.js
node_modules/braces/lib/utils.js
node_modules/braces/lib/compile.js

braces Uncontrolled resource consumption

braces Uncontrolled resource consumption
VULNERABILITY
CWE-400OPEN THIS LINK IN A NEW TAB
CVE-2024-4068OPEN THIS LINK IN A NEW TAB
CVSS 7.5OPEN THIS LINK IN A NEW TAB HIGH
SNYK-JS-BRACES-6838727OPEN THIS LINK IN A NEW TAB
SCORE
169
Introduced through
@ckeditor/[email protected]
Exploit maturity
PROOF OF CONCEPT
Show less detail
Detailed paths
Introduced through: [email protected] › @ckeditor/[email protected][email protected][email protected][email protected][email protected][email protected]
Fix: No remediation path available.
Security information
Factors contributing to the scoring:
Snyk: CVSS 7.5 - High Severity

NVD: Not available. NVD has not yet published its analysis.

documentation: "keepEscaping" is missing

I'm not sure if it's a braces or micromatch issue, but

braces("C:/Program Files \\(x86\\)")

returns

[ "C:/Program Files (x86)" ]

losing the escape characters, making

micromatch.parse("C:/Program Files \\(x86\\)")

return the tokens

[
	{
		"type": "bos",
		"value": "",
		"output": ""
	},
	{
		"type": "text",
		"value": "C:"
	},
	{
		"type": "slash",
		"value": "/",
		"output": "\\/"
	},
	{
		"type": "text",
		"value": "Program Files "
	},
	{
		"type": "paren",
		"value": "("
	},
	{
		"type": "text",
		"value": "x86"
	},
	{
		"type": "paren",
		"value": ")",
		"output": ")"
	}
]

vs what I expected and what

picomatch.parse("C:/Program Files \\(x86\\)")

returns

[
	{
		"type": "bos",
		"value": "",
		"output": ""
	},
	{
		"type": "text",
		"value": "C:"
	},
	{
		"type": "slash",
		"value": "/",
		"output": "\\/"
	},
	{
		"type": "text",
		"value": "Program Files \\(x86\\)"
	}
]

Here's some sample code: https://repl.it/repls/SharpStrangeChemistry

High Severity Vulnerability

After updating to the latest version got this report after running the command npm audit.
❯ npm audit

npm audit report

braces <3.0.3
Severity: high
Uncontrolled resource consumption in braces - GHSA-grv7-fg5c-xmjg
fix available via npm audit fix
node_modules/braces

1 high severity vulnerability

To address all issues, run:
npm audit fix

need escape `\` backslash or `{`

We need something like this

writeFixture('fixture.js', 'benchmark/fixtures/a/{b,{c,d},e/d}/*.\{js,md,txt\}');
//=> [
  "benchmark/fixtures/a/b/*.{js,md,txt}",
  "benchmark/fixtures/a/c/*.{js,md,txt}",
  "benchmark/fixtures/a/e/d/*.{js,md,txt}",
  "benchmark/fixtures/a/d/*.{js,md,txt}"
]

then we can pass the string to globbing lib, eg micromatch.

With single slash it removes it, with two \\, not works. The result is

//=> [
  "benchmark/fixtures/a/b/*.\\{js,md,txt\\}",
  "benchmark/fixtures/a/c/*.\\{js,md,txt\\}",
  "benchmark/fixtures/a/e/d/*.\\{js,md,txt\\}",
  "benchmark/fixtures/a/d/*.\\{js,md,txt\\}"
]

Braces is marked as DoS vulnerable via Memory Exhaustion by Blackduck

With recent scan, we found that braces npm package is marked as DoS vulnerable with high security risk according to BDSA-2024-2474 from Black Duck Security Advisory. As of now, no solution or workaround is available in the Blackduck report. Could someone from maintenance side to confirm this and provide a way to solve it?

npm run dev

When I am trying to run command npm run dev

It is giving below error :

[email protected] dev D:\angularpro\MEAN-STACK-PROJECT-01\backend
babel-watch sever.js

Error: Cannot find module 'D:\angularpro\MEAN-STACK-PROJECT-01\backend\sever.js'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
at Function.Module._load (internal/modules/cjs/loader.js:562:25)
at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
at process.on (D:\angularpro\MEAN-STACK-PROJECT-01\backend\node_modules\babel-watch\runner.js:110:21)
at process.emit (events.js:198:13)
at emit (internal/child_process.js:832:12)
at process._tickCallback (internal/process/next_tick.js:63:19)

When I have checked its showing two vunarabilities 👎
=== npm audit security report ===

                             Manual Review                                  
         Some vulnerabilities require your attention to resolve             
                                                                            
      Visit https://go.npm.me/audit-guide for additional guidance           

Low Regular Expression Denial of Service

Package braces

Patched in >=2.3.1

Dependency of babel-cli [dev]

Path babel-cli > chokidar > anymatch > micromatch > braces

More info https://npmjs.com/advisories/786

Low Regular Expression Denial of Service

Package braces

Patched in >=2.3.1

Dependency of babel-watch [dev]

Path babel-watch > chokidar > anymatch > micromatch > braces

More info https://npmjs.com/advisories/786

found 2 low severity vulnerabilities in 6339 scanned packages
2 vulnerabilities require manual review. See the full report for details.

I hae update all the packages 2 times.Update (Chokidar braces npm also babel also)
still it is giving me same error
When I am trying to run command
npm update

Package.json:

{
"name": "backend",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1",
"dev": "babel-watch sever.js"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"@babel/core": "^7.4.5",
"@babel/preset-env": "^7.4.5",
"babel-cli": "^6.26.0",
"babel-preset-env": "^1.7.0",
"babel-watch": "^7.0.0"
},
"dependencies": {
"anymatch": "^3.0.2",
"babel-core": "^6.26.3",
"braces": "^2.3.2",
"chokidar": "^3.0.1",
"cors": "^2.8.5",
"express": "^4.17.1",
"micromatch": "^4.0.2",
"mongoose": "^5.5.15"
}
}

Memory exhaustion issue in version 3.0.2

Vulnerability Details:

Vulnerability ID: CVE-2024-4068

Vulnerability Source: NVD

CWEs: CWE-1050

Inspector Score: None

Exploit Prediction Scoring System (EPSS): 0.00045

Related Vulnerabilities: None

A security vulnerability (CVE-2024-4068) has been identified in the braces NPM package, version 3.0.2 and below. The package fails to limit the number of characters it can handle, which could lead to memory exhaustion. In lib/parse.js, if a malicious user sends "imbalanced braces" as input, the parsing will enter a loop, causing the program to allocate heap memory continuously without freeing it. Eventually, this will lead to the JavaScript heap limit being reached and the program crashing.

nested bracket mistake

$ node -p 'require("./")("{a,b}{{a,b},a,b}")'
[ 'aa', 'ab', 'ba', 'bb' ]

$ echo {a,b}{{a,b},a,b}
aa ab aa ab ba bb ba bb

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.