ark0f/logos-iter

Extensions for Logos lexer

logos-iter's Issues

`PeekableLexer::span` changes after a peek

Thanks for creating this crate! It has been very useful.

I have a minor bug, best understood with an example-

let mut l = Token::lexer("1+2").peekable_lexer();
assert_eq!(l.next(), Some(Token::I32Literal(1)));
assert_eq!(l.span(), 0..1);
assert_eq!(l.peek(), Some(&Token::Plus));
// since the lexer hasn't progressed, its span should be the same
assert_eq!(l.span(), 0..1);

Currently, the last assert fails and span gives 1..2 instead.

I guess the fix would be to use prev_span here instead of span-

logos-iter/src/peekable.rs

Line 238 in 8240fc7

Some(Peeked { span, .. }) => span.clone(),

Would be happy to raise a PR if needed.