facebook/pyre2

Python wrapper for RE2

pyre2

Summary

pyre2 is a Python extension that wraps Google's RE2 regular expression library. It implements many of the features of Python's built-in re module with compatible interfaces.

New Features

• Regexp objects have a fullmatch method that works like match, but anchors the match at both the start and the end.
• Regexp objects have test_search, test_match, and test_fullmatch methods that work like search, match, and fullmatch, but only return True or False to indicate whether the match was successful. These methods should be faster than the full versions, especially for patterns with capturing groups.

Missing Features

• No substitution methods.
• No flags.
• No split, findall, or finditer.
• No top-level convenience functions like search and match. (Just use compile.)
• No compile cache. (If you care enough about performance to use RE2, you probably care enough to cache your own patterns.)
• No lastindex or lastgroup on Match objects.

Current Status

pyre2 has only received basic testing, and I am by no means a Python extension expert, so it is quite possible that it contains bugs. I'd guess the most likely are reference leaks in error cases.

RE2 doesn't build with fPIC, so I had to build it with

make CFLAGS='-fPIC -c -Wall -Wno-sign-compare -O3 -g -I.'

I also had to add it to my compiler search path when building the module with a command like

env CPPFLAGS='-I/path/to/re2' LDFLAGS='-L/path/to/re2/obj' ./setup.py build

Contact

You can file bug reports on GitHub, or email the author: David Reiss <[email protected]>.

License

See the LICENSE file for more information.

pyre2's Issues

Match misplace by 2

When the ’ (RIGHT SINGLE QUOTATION MARK ; HTML &RSQUO; HEX 0x2019 ) is in the string to search the span is displace by 2, for example:

string = "it’s your car"
regex = re2.compile('you')
match = regex.search(string)
pather_match = string[match.start():match.end()]
print(pather_match)

'ur c'

Display the dynamic members in dir() function with __dict__ member

As you known, regexp and match object always create some dynamic members with attr_dict, which can't be display in the dir() function result, we should add a static member dict point to the dict

Please refer to the commit, thanks

flier@7caeb37

cannot build on Mac

@localhost pyre2 % make CFLAGS='-fPIC -c -Wall -Wno-sign-compare -O3 -g -I.'
python3 setup.py build
running build
running build_py
running build_ext
building '_re2' extension
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -fPIC -c -Wall -Wno-sign-compare -O3 -g -I. -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c _re2.cc -o build/temp.macosx-10.15-x86_64-3.8/_re2.o -std=c++11
_re2.cc:211:3: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8/object.h:85:5: note: expanded from macro 'PyObject_HEAD_INIT'
1, type },
^~~~~~~
_re2.cc:255:3: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8/object.h:85:5: note: expanded from macro 'PyObject_HEAD_INIT'
1, type },
^~~~~~~
_re2.cc:299:3: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8/object.h:85:5: note: expanded from macro 'PyObject_HEAD_INIT'
1, type },
^~~~~~~
3 warnings generated.
clang++ -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -c -Wall -Wno-sign-compare -O3 -g -I. build/temp.macosx-10.15-x86_64-3.8/_re2.o -L/usr/local/lib -L/usr/local/opt/[email protected]/lib -L/usr/local/opt/sqlite/lib -lre2 -o build/lib.macosx-10.15-x86_64-3.8/_re2.cpython-38-darwin.so
clang: warning: build/temp.macosx-10.15-x86_64-3.8/_re2.o: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lre2: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-bundle' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-undefined dynamic_lookup' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-fPIC' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-O3' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-g' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I .' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/lib' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/opt/[email protected]/lib' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/opt/sqlite/lib' [-Wunused-command-line-argument]
python3 setup.py build_ext -i
running build_ext
building '_re2' extension
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -fPIC -c -Wall -Wno-sign-compare -O3 -g -I. -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c _re2.cc -o build/temp.macosx-10.15-x86_64-3.8/_re2.o -std=c++11
_re2.cc:211:3: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8/object.h:85:5: note: expanded from macro 'PyObject_HEAD_INIT'
1, type },
^~~~~~~
_re2.cc:255:3: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8/object.h:85:5: note: expanded from macro 'PyObject_HEAD_INIT'
1, type },
^~~~~~~
_re2.cc:299:3: warning: suggest braces around initialization of subobject [-Wmissing-braces]
PyObject_HEAD_INIT(NULL)
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/include/python3.8/object.h:85:5: note: expanded from macro 'PyObject_HEAD_INIT'
1, type },
^~~~~~~
3 warnings generated.
clang++ -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -c -Wall -Wno-sign-compare -O3 -g -I. build/temp.macosx-10.15-x86_64-3.8/_re2.o -L/usr/local/lib -L/usr/local/opt/[email protected]/lib -L/usr/local/opt/sqlite/lib -lre2 -o build/lib.macosx-10.15-x86_64-3.8/_re2.cpython-38-darwin.so
clang: warning: build/temp.macosx-10.15-x86_64-3.8/_re2.o: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lre2: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-bundle' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-undefined dynamic_lookup' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-fPIC' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-O3' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-g' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I .' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/lib' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/opt/[email protected]/lib' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/opt/sqlite/lib' [-Wunused-command-line-argument]
error: can't copy 'build/lib.macosx-10.15-x86_64-3.8/_re2.cpython-38-darwin.so': doesn't exist or not a regular file
make: *** [build] Error 1
@localhost pyre2 %

Cannot build on newer versions of absl

After June 2022, ABSL no longer supports c++11, so it cannot build anymore.

fb-re2 compiles and installs on macos but using gives segmentation fault

While I suspect there is some specific local environment issue that produces this awful result, I think that we need to prevent it from building/installing a broken package.

pip install fb-re2
python -c "import re2; re2.compile('foo')"
[1]    99748 segmentation fault  python -c "import re2; re2.compile('foo')"

I was able to reproduce the error with both python3.6 and pyton3.7 installed using pyenv.

No license included in Python sdist

See PR #17 that should fix this issue. No license included is an issue for distribution packaging.