mcdostone/nom_locate

A special input type for nom to locate tokens

nom_locate

A special input type for nom to locate tokens

Documentation

The documentation of the crate is available here.

How to use it

The crate provide the LocatedSpan struct that encapsulates the data. Look at the below example and the explanations:

extern crate nom;
extern crate nom_locate;

use nom::bytes::complete::{tag, take_until};
use nom::IResult;
use nom_locate::{position, LocatedSpan};

type Span<'a> = LocatedSpan<&'a str>;
struct Token<'a> {
    pub position: Span<'a>,
    pub _foo: &'a str,
    pub _bar: &'a str,
}

fn parse_foobar(s: Span) -> IResult<Span, Token> {
    let (s, _) = take_until("foo")(s)?;
    let (s, pos) = position(s)?;
    let (s, foo) = tag("foo")(s)?;
    let (s, bar) = tag("bar")(s)?;

    Ok((
        s,
        Token {
            position: pos,
            _foo: foo.fragment(),
            _bar: bar.fragment(),
        },
    ))
}

fn main() {
    let input = Span::new("Lorem ipsum \n foobar");
    let output = parse_foobar(input);
    let position = output.unwrap().1.position;
    assert_eq!(position, unsafe {
        Span::new_from_raw_offset(
            14, // offset
            2,  // line
            "", // fragment
            (), // extra
        )
    });
    assert_eq!(position.get_column(), 2);
}

Import

Import nom and nom_locate.

extern crate nom;
extern crate nom_locate;

use nom::bytes::complete::{tag, take_until};
use nom::IResult;
use nom_locate::{position, LocatedSpan};

Also you'd probably create type alias for convenience so you don't have to specify the fragment type every time:

type Span<'a> = LocatedSpan<&'a str>;

Define the output structure

The output structure of your parser may contain the position as a Span (which provides the index, line and column information to locate your token).

struct Token<'a> {
    pub position: Span<'a>,
    pub _foo: &'a str,
    pub _bar: &'a str,
}

Create the parser

The parser has to accept a Span as an input. You may use position() in your nom parser, in order to capture the location of your token:

fn parse_foobar(s: Span) -> IResult<Span, Token> {
    let (s, _) = take_until("foo")(s)?;
    let (s, pos) = position(s)?;
    let (s, foo) = tag("foo")(s)?;
    let (s, bar) = tag("bar")(s)?;

    Ok((
        s,
        Token {
            position: pos,
            _foo: foo.fragment(),
            _bar: bar.fragment(),
        },
    ))
}

Call the parser

The parser returns a nom::IResult<Token, _> (hence the unwrap().1). The position property contains the offset, line and column.

fn main() {
    let input = Span::new("Lorem ipsum \n foobar");
    let output = parse_foobar(input);
    let position = output.unwrap().1.position;
    assert_eq!(position, unsafe {
        Span::new_from_raw_offset(
            14, // offset
            2,  // line
            "", // fragment
            (), // extra
        )
    });
    assert_eq!(position.get_column(), 2);
}