A simple library for tokenizing text with additional functionalities. (NOT MAINTAINED ANYMORE)
SBLex is a lexer that is capable to tokenize text with only a few lines of code, it includes as well other extra functions.
- Dependent Tokens
- Custom Errors
- Custom functions when a token is found
- Custom capture groups
- Support of multiple regular expressions
- Ignore patterns
- Documentation (Please use a IDE that supports autocomplete for docs)
You can easily install SBLex through pip like this:
pip install SBLex
If you get an environment error, you can also try using the --user option:
pip install SBLex --user
Starting using SBLex is as easy as importing the SBLex and declaring the lexer, here you have a working example lexing the world Hello World!:
from SBLex import lex
lexer = lex.lex()
lexer.add("Hello World", r'Hello World\!')
lexer.evaluate("Hello World!")
"""
Returns:
>>> [token(type: 'Hello World', value: 'Hello World!', line: 0)]
"""
Here is another example using a custom capturing_group.
from SBLex import lex
lexer = lex.lex()
# Try to lex: The weather today is {weather}
lexer.add(
"The weather today is",
r'The weather today is ([a-zA-Z]+)',
capturing_group=1
)
lexer.evaluate("The weather today is rainy")
"""
Returns:
>>> [token(type: 'The weather today is', value: 'rainy', line: 0)]
"""
SBLex is compatible with Python2 & Python3.
SBLex is under the MIT License.
SBLex was written by KadMuffin.