Nongreedy rules

[microsoft]

Nongreedy rules

[ZihanChen-MSFT]

I think it works like rep which gives you all results and rep_sc gives you the "most greedy" result. Do you have a more specific requirement?

[peterbud]

Consider the following input:

Dear customer, here is the statement of your account as of 01.01.2023. more text to discard...

Account statement EUR

The following transaction were recorded... even more text to discard

01.12.2022 Transfer 100
03.12.2022 Transfer 300

Sincerely, more text to discard

The first requirement to be able to consume any token in a non-greedy way, like at the beginning of the input.
I have not seen any function which would match any token, I have created a function which does that - I have called as any - maybe there are better way to do that?

Then I have tried the following lexar/parser:

export enum TokenKind {
  DMY,
  Number,
  Header,
  Other,
  Space,
}

export const lexer = buildLexer([
  [true, /^\d{2}\.\d{2}\.\d{4}/g, TokenKind.DMY],
  [true, /^(\d*\.?)*(,\d*)?( *-)?/g, TokenKind.Number],
  [true, /^account statement/gi, TokenKind.Header],

  [true, /^\S+/g, TokenKind.Other], // all other words
  [false, /^\s+/g, TokenKind.Space], // ignore whitespace
])

const STATEMENT = rule<TokenKind, unknown>()
STATEMENT.setPattern(
  seq(
    rep(any()),
    tok(TokenKind.Header),
    rep(any()),
    rep(seq(tok(TokenKind.DMY), str("Transfer"), tok(TokenKind.Number))),
  ),
)

The problem is that rep() is greedy in a sense that it will match all tokens, although what I'm looking for is to match until it finds the first Header token. Then again I'd need to consume all tokens until I find the first date token which marks the beginning of the transactions.

What would be your suggestion for this problem?

[ZihanChen-MSFT]

I feel like this is overusing, that's why there is no any. I would just split the text into lines first and match each line and keep all succeeded ones. And I think regular expression is simpler for this particular input.