Giter Site home page Giter Site logo

hanickadot / compile-time-regular-expressions Goto Github PK

View Code? Open in Web Editor NEW
3.2K 67.0 174.0 2.86 MB

Compile Time Regular Expression in C++

Home Page: https://twitter.com/hankadusikova

License: Apache License 2.0

Makefile 0.07% C++ 99.79% Python 0.03% CMake 0.07% JavaScript 0.04%
regular-expression cpp17 cpp20 header-only constexpr compile-time awesome template-udl pcre

compile-time-regular-expressions's Introduction

Compile time regular expressions v3

Build Status

Fast compile-time regular expressions with support for matching/searching/capturing during compile-time or runtime.

You can use the single header version from directory single-header. This header can be regenerated with make single-header. If you are using cmake, you can add this directory as subdirectory and link to target ctre.

More info at compile-time.re

What this library can do

ctre::match<"REGEX">(subject); // C++20
"REGEX"_ctre.match(subject); // C++17 + N3599 extension
  • Matching
  • Searching (search or starts_with)
  • Capturing content (named captures are supported too)
  • Back-Reference (\g{N} syntax, and \1...\9 syntax too)
  • Multiline support (with multi_) functions
  • Unicode properties and UTF-8 support

The library is implementing most of the PCRE syntax with a few exceptions:

  • callouts
  • comments
  • conditional patterns
  • control characters (\cX)
  • match point reset (\K)
  • named characters
  • octal numbers
  • options / modes
  • subroutines
  • unicode grapheme cluster (\X)

More documentation on pcre.org.

Unknown character escape behaviour

Not all escaped characters are automatically inserted as self, behaviour of the library is escaped characters are with special meaning, unknown escaped character is a syntax error.

Explicitly allowed character escapes which insert only the character are:

\-\"\<\>

Basic API

This is approximated API specification from a user perspective (omitting constexpr and noexcept which are everywhere, and using C++20 syntax even the API is C++17 compatible):

// look if whole input matches the regex:
template <fixed_string regex> auto ctre::match(auto Range &&) -> regex_results;
template <fixed_string regex> auto ctre::match(auto First &&, auto Last &&) -> regex_results;

// look if input contains match somewhere inside of itself:
template <fixed_string regex> auto ctre::search(auto Range &&) -> regex_results;
template <fixed_string regex> auto ctre::search(auto First &&, auto Last &&) -> regex_results;

// check if input starts with match (but doesn't need to match everything):
template <fixed_string regex> auto ctre::starts_with(auto Range &&) -> regex_results;
template <fixed_string regex> auto ctre::starts_with(auto First &&, auto Last &&) -> regex_results;

// result type is deconstructible into a structured bindings
template <...> struct regex_results {
	operator bool() const; // if it's a match
	auto to_view() const -> std::string_view; // also view()
	auto to_string() const -> std::string; // also str()
	operator std::string_view() const; // also supports all char variants
	explicit operator std::string() const;
	
	// also size(), begin(), end(), data()
	
	size_t count() const; // number of captures 
	template <size_t Id> const captured_content & get() const; // provide specific capture, whole regex_results is implicit capture 0
};

Range outputing API

// search for regex in input and return each occurence, ignoring rest:
template <fixed_string regex> auto ctre::range(auto Range &&) -> range of regex_result;
template <fixed_string regex> auto ctre::range(auto First &&, auto Last &&) -> range of regex_result;

// return range of each match, stopping at something which can't be matched
template <fixed_string regex> auto ctre::tokenize(auto Range &&) -> range of regex_result;
template <fixed_string regex> auto ctre::tokenize(auto First &&, auto Last &&) -> range of regex_result;

// return parts of the input splited by the regex, returning it as part of content of the implicit zero capture (other captures are not changed, you can use it to access how the values were splitted):
template <fixed_string regex> auto ctre::split(auto Range &&) -> regex_result;
template <fixed_string regex> auto ctre::split(auto First &&, auto Last &&) -> range of regex_result;

Functors

All the functions (ctre::match, ctre::search, ctre::starts_with, ctre::range, ctre::tokenize, ctre::split) are functors and can be used without parenthesis:

auto matcher = ctre::match<"regex">;
if (matcher(input)) ...

Possible subjects (inputs)

  • std::string-like objects (std::string_view or your own string if it's providing begin/end functions with forward iterators)
  • pairs of forward iterators

Unicode support

To enable you need to include:

  • <ctre-unicode.hpp>
  • or <ctre.hpp> and <unicode-db.hpp>

Otherwise you will get missing symbols if you try to use the unicode support without enabling it.

Supported compilers

  • clang 7.0+ (template UDL, C++17 syntax)
  • xcode clang 10.0+ (template UDL, C++17 syntax)
  • clang 12.0+ (C++17 syntax, C++20 cNTTP syntax)
  • gcc 8.0+ (template UDL, C++17 syntax)
  • gcc 9.0+ (C++17 & C++20 cNTTP syntax)
  • MSVC 14.29+ (Visual Studio 16.11+) (C++20)

Template UDL syntax

The compiler must support extension N3599, for example as GNU extension in gcc (not in GCC 9.1+) and clang.

constexpr auto match(std::string_view sv) noexcept {
    using namespace ctre::literals;
    return "h.*"_ctre.match(sv);
}

If you need extension N3599 in GCC 9.1+, you can't use -pedantic. Also, you need to define macro CTRE_ENABLE_LITERALS.

C++17 syntax

You can provide a pattern as a constexpr ctll::fixed_string variable.

static constexpr auto pattern = ctll::fixed_string{ "h.*" };

constexpr auto match(std::string_view sv) noexcept {
    return ctre::match<pattern>(sv);
}

(this is tested in MSVC 15.8.8)

C++20 syntax

Currently, the only compiler which supports cNTTP syntax ctre::match<PATTERN>(subject) is GCC 9+.

constexpr auto match(std::string_view sv) noexcept {
    return ctre::match<"h.*">(sv);
}

Examples

Extracting number from input

std::optional<std::string_view> extract_number(std::string_view s) noexcept {
    if (auto m = ctre::match<"[a-z]+([0-9]+)">(s)) {
        return m.get<1>().to_view();
    } else {
        return std::nullopt;
    }
}

link to compiler explorer

Extracting values from date

struct date { std::string_view year; std::string_view month; std::string_view day; };

std::optional<date> extract_date(std::string_view s) noexcept {
    using namespace ctre::literals;
    if (auto [whole, year, month, day] = ctre::match<"(\\d{4})/(\\d{1,2})/(\\d{1,2})">(s); whole) {
        return date{year, month, day};
    } else {
        return std::nullopt;
    }
}

//static_assert(extract_date("2018/08/27"sv).has_value());
//static_assert((*extract_date("2018/08/27"sv)).year == "2018"sv);
//static_assert((*extract_date("2018/08/27"sv)).month == "08"sv);
//static_assert((*extract_date("2018/08/27"sv)).day == "27"sv);

link to compiler explorer

Using captures

auto result = ctre::match<"(?<year>\\d{4})/(?<month>\\d{1,2})/(?<day>\\d{1,2})">(s);
return date{result.get<"year">(), result.get<"month">, result.get<"day">};

// or in C++ emulation, but the object must have a linkage
static constexpr ctll::fixed_string year = "year";
static constexpr ctll::fixed_string month = "month";
static constexpr ctll::fixed_string day = "day";
return date{result.get<year>(), result.get<month>, result.get<day>};

// or use numbered access
// capture 0 is the whole match
return date{result.get<1>(), result.get<2>, result.get<3>};

Lexer

enum class type {
    unknown, identifier, number
};

struct lex_item {
    type t;
    std::string_view c;
};

std::optional<lex_item> lexer(std::string_view v) noexcept {
    if (auto [m,id,num] = ctre::match<"([a-z]+)|([0-9]+)">(v); m) {
        if (id) {
            return lex_item{type::identifier, id};
        } else if (num) {
            return lex_item{type::number, num};
        }
    }
    return std::nullopt;
}

link to compiler explorer

Range over input

This support is preliminary, probably the API will be changed.

auto input = "123,456,768"sv;

for (auto match: ctre::range<"([0-9]+),?">(input)) {
    std::cout << std::string_view{match.get<0>()} << "\n";
}

Unicode

#include <ctre-unicode.hpp>
#include <iostream>
// needed if you want to output to the terminal
std::string_view cast_from_unicode(std::u8string_view input) noexcept {
    return std::string_view(reinterpret_cast<const char *>(input.data()), input.size());
}
int main()
{
    using namespace std::literals;
    std::u8string_view original = u8"Tu es un génie"sv;

    for (auto match : ctre::range<"\\p{Letter}+">(original))
        std::cout << cast_from_unicode(match) << std::endl;
    return 0;
}

link to compiler explorer

Installing ctre using vcpkg

You can download and install ctre using the vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install ctre

The ctre port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

Running tests (for developers)

Just run make in root of this project.

compile-time-regular-expressions's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

compile-time-regular-expressions's Issues

Compile error with gcc 9.1 in "pedantic" mode

Including ctre.hpp gives me the following errors when compiling with gcc 9.1 in "pedantic" mode:

[  0%] Building CXX object lib/CMakeFiles/sdblib.dir/validate.cpp.o
In file included from /home/wolfram/sdb/lib/validate.cpp:10:
/home/wolfram/sdb/include/ctre.hpp:3652:124: error: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3652 | template <typename CharT, CharT... charpack> CTRE_FLATTEN constexpr CTRE_FORCE_INLINE auto operator""_ctre() noexcept {
      |                                                                                                                            ^       
/home/wolfram/sdb/include/ctre.hpp:3670:124: error: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3670 | template <typename CharT, CharT... charpack> CTRE_FLATTEN constexpr CTRE_FORCE_INLINE auto operator""_ctre_id() noexcept {
      |                                                                                                                            ^       
/home/wolfram/sdb/include/ctre.hpp:3684:93: error: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3684 | template <typename CharT, CharT... charpack> CTRE_FLATTEN constexpr inline auto operator""_ctre_test() noexcept {
      |                                                                                             ^~~~~~~~
/home/wolfram/sdb/include/ctre.hpp:3694:93: error: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3694 | template <typename CharT, CharT... charpack> CTRE_FLATTEN constexpr inline auto operator""_ctre_gen() noexcept {
      |                                                                                             ^~~~~~~~
/home/wolfram/sdb/include/ctre.hpp:3706:124: error: ISO C++ did not adopt string literal operator templates taking an argument pack of characters [-Wpedantic]
 3706 | template <typename CharT, CharT... charpack> CTRE_FLATTEN constexpr CTRE_FORCE_INLINE auto operator""_ctre_syntax() noexcept {
      |                                                                                                                            ^~~     
lib/CMakeFiles/sdblib.dir/build.make:446: recipe for target 'lib/CMakeFiles/sdblib.dir/validate.cpp.o' failed
make[2]: *** [lib/CMakeFiles/sdblib.dir/validate.cpp.o] Error 1
CMakeFiles/Makefile2:85: recipe for target 'lib/CMakeFiles/sdblib.dir/all' failed
make[1]: *** [lib/CMakeFiles/sdblib.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

The line in question (validate.cpp, line 10) is:

#include <ctre.hpp>

Compiler version:

$ gcc --version
gcc (GCC) 9.1.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiler flags:

-O3
-funsigned-char
-pedantic -pedantic-errors -Wall -Wextra -Werror
-Wno-missing-field-initializers -Wno-unused-result -Wno-deprecated-declarations

Using the latest version of ctre.hpp, downloaded from https://raw.githubusercontent.com/hanickadot/compile-time-regular-expressions/master/single-header/ctre.hpp just half an hour ago.

It works when I remove -pedantic -pedantic-errors from the compiler flags.

Incorrect use of __has_cpp_attribute

There are snippets like below spilled over the code.

#if __has_cpp_attribute(__cpp_nontype_template_parameter_class)

But __has_cpp_attribute is for checking c++ attributes, e.g. [[noreturn]].

Instead, what you need is:

#if __cpp_nontype_template_parameter_class

These are macros, it evaluates to false if not defined.

Links to talks

It'd be nice if the README provided links to the talks you've given on this library, and maybe other existing resources on it aswell. Is the CppCon 2018 one the most up-to-date?

[License] Re-license under BSL-1.0?

ctre is currently licensed under the MIT license.

Perhaps you want to reconsider this and re-license it under Boost license (BSL-1.0)?

Naturally when ctre is part of the C++ standard, this question becomes less relevant ;)

My experience is that the Boost license is sometimes (often?) preferred as that does not require that the license appear with executables or other binary uses of the library, see this note by Vinnie Falco on reddit. As a result I changed the nonstd-lite projects to use the Boost license instead of MIT's.

Wrong result with match

I believe the result in the following repo is wrong:

I'm expecting that the match is true. Verified with other implementations: https://regex101.com/r/Ln5iH3/1

Please disregard. A full match is required with the match() function. search() does what I want.

constexpr with function arguments over NTTPs

It seems that the current implementation uses NTTPs, which means that using complex regexes become troublesome or impossible due to resource consumption. With GCC9+ supporting std::is_constant_evaluated(), a pure constexpr implementation that takes the regex as a function argument should be possible. That shouldn't consume as much resources as the NTTP implementation, and error messages should improve if exceptions were to be thrown at the point of error. After all, the current implementation simply static_asserts at a fixed position, giving no clue as to what might have gone wrong.

Underscore symbol pattern doesn't compile

Clang 5.0

Regular expression with single underscore _ does not compile.
But something like [_] compiles.

#include <ctre/include/ctre.hpp>

int main(int, char**)
{
  using namespace ctre::literals;
  "_"_ctre.match("_");
  return 0;
}
In file included from ctre_test/main.cpp:1:
In file included from ctre/include/ctre.hpp:4:
ctre/include/ctre/literals.hpp:55:2: error: static_assert failed due to requirement 'tmp()' "Regular Expression contains syntax error."
        static_assert(tmp(), "Regular Expression contains syntax error.");
        ^             ~~~~~
main.cpp:6:6: note: in instantiation of function template specialization 'ctre::literals::operator""_ctre<char, '_'>' requested here
  "_"_ctre.match("_");
     ^

warning C4244: 'initializing': conversion from 'int64_t' to 'size_t', possible loss of data

When the arch is 32-bit (size_t = 32 bits), I get the following warning:
warning C4244: 'initializing': conversion from 'int64_t' to 'size_t', possible loss of data

When using something like that:
const auto results = ctre::re().search(begin, end);

The pattern does not seem to matter. But it is triggered with this one:
LR"(^([-dlsDLS][-rwxstlRWXSTL]{9}[+]?)\s*(\d+)?\s+(\S+)?\s+(.*?)?\s+(\d+|\d+.\d+\S)\s+([A-Za-z][a-z][a-z])\s+(\d{1,2})\s+((\d{4,4})|(\d{1,2}):.)\s)"

Idea: make it possible to obtain std::partial_ordering from a match

Right now there is an implicit conversion from a match (more explicitly from a captured_content::storage as I can see). I wonder if it was possible to convert a match to std::partial_ordering to obtain more information from a non-match. The semantics of the returned values would be:

  • std::less/std::greater : Not a match and the subject orders lexicographically before/after all possible strings that match the regular expression.
  • std::equivalent : It's a match.
  • std::unordered : All else.

Use case: searching matching strings in a lexicographically ordered collection of strings efficiently. Right now there is no interface for std::map and std::set to allow this, but other maps and sets could offer searching with a partial_ordering predicate.

Some examples for return values:

constexpr auto re = ctre::match<"^abc[0-46-9]+$">;
static_assert(re("abc1"sv).to_partial_ordering() == std::equivalent);
static_assert(re("bac1"sv).to_partial_ordering() == std::greater);
static_assert(re("aaa1"sv).to_partial_ordering() == std::less);
static_assert(re("abc5"sv).to_partial_ordering() == std::unordered);

Unnecessary escaping of otherwise special characters in square brackets

According to http://www.pcre.org/current/doc/html/pcre2pattern.html#SEC9, between [ and ]:

  • You don't need to escape ^ for it to be considered literally if its not after the [, where it has a special meaning.
  • Similarly, you need to escape - for it to considered literally even if it's in a place where it unambiguously can't be used to define a range.

See example: https://godbolt.org/z/8v6cu7. It seems that you also need to escape |, even though it has no special meaning anywhere between square brackets.

The only metacharacters that are recognized in character classes are backslash, hyphen (only where it can be interpreted as specifying a range), circumflex (only at the start), opening square bracket (only when it can be interpreted as introducing a POSIX class name, or for a special compatibility feature - see the next two sections), and the terminating closing square bracket. However, escaping other non-alphanumeric characters does no harm.

Stack overflow on large subject (~80k characters)

I have the following target:

auto re = "a+b"_ctre

The subject is 80000 characters of the character 'a' passed in standard input then stored in an std::string. I ran it on Linux with 8192 KiB stack size, compiled with both g++ and clang++ with -O3. I get a stack overflow.

The full source of the test program:

#include <ctre.hpp>
#include <iostream>
#include <sstream>


int main() {
	using namespace ctre::literals;

	constexpr auto re_ctre = "a+b"_ctre;

	std::stringstream buf;
	buf << std::cin.rdbuf();

	const auto str = buf.str();

	volatile auto ctre_match = bool(re_ctre.match(str));

	return 0;
}

For certain patterns it's impossible to match in constant space, for these patterns it's guaranteed to have a stack overflow with large enough input.
I think it should be possible to match against the above specific pattern using constant space though.
I wonder how a functional style metaprogramming heavy library can avoid this. I think range-v3 can easily have the same problem.

Headers should not include <iostream>

Including iostream introduces dependecies to the global stream objects, that when used remove unit-testability from code. If i/o functions are implemented include and instead, or only when only declarations of such functions are made.

MSVC: compilation error with latest commits

With one of the latest commits (sorry I couldn't identify which commit exactly, but it works with https://github.com/hanickadot/compile-time-regular-expressions/tree/b9c35abf0c69ed4049e508458f6a6c563f902566), compilation with MSVC (15.9.5) is broken again:


static constexpr auto Pattern = ctll::basic_fixed_string{ LR"(^(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\d+)\s+(\d+)\s*$)" };

_Success_(return != FALSE) BOOL CType1::Parse(_In_reads_(end - begin) const wchar_t* begin, _In_ const wchar_t* end)
{
	const auto results = ctre::re<Pattern>().match(begin, end);
}
..

Error:

1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(96): error C2975: 'ctll::parser': invalid template argument for 'input', expected compile-time constant expression
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(25): note: see declaration of 'ctll::parser'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(144): note: see reference to function template instantiation 'auto ctll::parser<ctre::pcre,Parser::Pattern,ctre::pcre_actions,false>::move<0,ctll::term<94>,ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,Subject>(ctll::pop_input,Terminal,Stack,Subject) noexcept' being compiled
1>        with
1>        [
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>,
1>            Terminal=ctll::term<94>,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(100): note: see reference to function template instantiation 'auto ctll::parser<ctre::pcre,Parser::Pattern,ctre::pcre_actions,false>::decide<0,ctll::list<ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,Subject>(Stack,Subject) noexcept' being compiled
1>        with
1>        [
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>,
1>            Stack=ctll::list<ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(144): note: see reference to function template instantiation 'auto ctll::parser<ctre::pcre,Parser::Pattern,ctre::pcre_actions,false>::move<0,ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2,ctll::term<94>,ctll::list<>,Subject>(ctll::list<ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,Terminal,Stack,Subject) noexcept' being compiled
1>        with
1>        [
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>,
1>            Terminal=ctll::term<94>,
1>            Stack=ctll::list<>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(148): note: see reference to function template instantiation 'auto ctll::parser<ctre::pcre,Parser::Pattern,ctre::pcre_actions,false>::decide<0,ctll::list<ctre::pcre::_start>,Subject>(Stack,Subject) noexcept' being compiled
1>        with
1>        [
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>,
1>            Stack=ctll::list<ctre::pcre::_start>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(157): note: see reference to function template instantiation 'auto ctll::parser<ctre::pcre,Parser::Pattern,ctre::pcre_actions,false>::trampoline_decide<Subject,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49>(Subject,std::integer_sequence<unsigned int,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49>) noexcept' being compiled
1>        with
1>        [
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\myproject\parser\gxsenterpriseparser.cpp(41): note: see reference to class template instantiation 'ctll::basic_fixed_string<wchar_t,51>' being compiled
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(160): note: see reference to function template instantiation 'auto ctll::parser<ctre::pcre,Parser::Pattern,ctre::pcre_actions,false>::trampoline_decide<ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>>(Subject) noexcept' being compiled
1>        with
1>        [
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\functions.hpp(40): note: see reference to alias template instantiation 'ctll::parser<ctre::pcre,Parser::Pattern,ctre::pcre_actions,false>::output<ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>>' being compiled
1>c:\projects\myproject\parser\gxsenterpriseparser.cpp(47): note: see reference to function template instantiation 'auto ctre::re<& Parser::Pattern>(void) noexcept' being compiled
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(81): error C2109: subscript requires array or pointer type
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(144): note: see reference to function template instantiation 'auto ctll::parser<Grammar,$S15,ActionSelector,false>::get_previous_term<1>(void) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(50): note: see reference to function template instantiation 'auto ctll::parser<Grammar,$S15,ActionSelector,false>::decide<1,Stack,Subject>(Stack,Subject) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(43): note: while compiling class template member function 'auto ctll::parser<Grammar,0,ActionSelector,false>::results<1,Stack,Subject,ctll::decision::undecided>::operator +(ctll::placeholder) noexcept const'
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(96): note: see reference to class template instantiation 'ctll::parser<Grammar,0,ActionSelector,false>::results<1,Stack,Subject,ctll::decision::undecided>' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(144): note: see reference to function template instantiation 'ctll::parser<Grammar,0,ActionSelector,false>::results<1,Stack,Subject,ctll::decision::undecided> ctll::parser<Grammar,Parser::Pattern,ActionSelector,false>::move<0,ctll::term<94>,ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,Subject>(ctll::pop_input,Terminal,Stack,Subject) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>,
1>            Terminal=ctll::term<94>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(100): note: see reference to function template instantiation 'ctll::parser<Grammar,0,ActionSelector,false>::results<1,Stack,Subject,ctll::decision::undecided> ctll::parser<Grammar,Parser::Pattern,ActionSelector,false>::decide<0,ctll::list<ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,Subject>(ctll::list<ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,Subject) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(144): note: see reference to function template instantiation 'ctll::parser<Grammar,0,ActionSelector,false>::results<1,Stack,Subject,ctll::decision::undecided> ctll::parser<Grammar,Parser::Pattern,ActionSelector,false>::move<0,ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2,ctll::term<94>,ctll::list<>,Subject>(ctll::list<ctll::anything,ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,Terminal,ctll::list<>,Subject) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>,
1>            Terminal=ctll::term<94>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(148): note: see reference to function template instantiation 'ctll::parser<Grammar,0,ActionSelector,false>::results<1,Stack,Subject,ctll::decision::undecided> ctll::parser<Grammar,Parser::Pattern,ActionSelector,false>::decide<0,ctll::list<ctre::pcre::_start>,Subject>(ctll::list<ctre::pcre::_start>,Subject) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::push_assert_begin,ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(81): error C3536: 'value': cannot be used before it is initialized
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(74): error C2131: expression did not evaluate to a constant
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(74): note: a non-constant (sub-)expression was encountered
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(81): error C2971: 'ctll::term': template parameter 'v': 'value': a variable with non-static storage duration cannot be used as a non-type argument
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\grammars.hpp(7): note: see declaration of 'ctll::term'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(73): note: see declaration of 'value'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(81): error C3487: 'ctll::epsilon': all return expressions must deduce to the same type: previously it was 'ctll::term<0>'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(81): error C2440: 'return': cannot convert from 'ctll::epsilon' to 'ctll::term<0>'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(81): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(66): error C2109: subscript requires array or pointer type
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(144): note: see reference to function template instantiation 'auto ctll::parser<Grammar,$S15,ActionSelector,false>::get_current_term<1>(void) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(144): note: see reference to function template instantiation 'auto ctll::parser<Grammar,$S15,ActionSelector,false>::decide<1,ctll::list<ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,ctre::pcre_context<ctll::list<ctre::assert_begin>,ctre::pcre_parameters<0>>>(Stack,Subject) noexcept' being compiled
1>        with
1>        [
1>            Grammar=ctre::pcre,
1>            ActionSelector=ctre::pcre_actions,
1>            Stack=ctll::list<ctre::pcre::repeat,ctre::pcre::string2,ctre::pcre::content2>,
1>            Subject=ctre::pcre_context<ctll::list<ctre::assert_begin>,ctre::pcre_parameters<0>>
1>        ]
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(66): error C3536: 'value': cannot be used before it is initialized
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(57): error C2131: expression did not evaluate to a constant
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(57): note: a non-constant (sub-)expression was encountered
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(66): error C2971: 'ctll::term': template parameter 'v': 'value': a variable with non-static storage duration cannot be used as a non-type argument
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\grammars.hpp(7): note: see declaration of 'ctll::term'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(56): note: see declaration of 'value'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(66): error C3487: 'ctll::epsilon': all return expressions must deduce to the same type: previously it was 'ctll::term<0>'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(66): error C2440: 'return': cannot convert from 'ctll::epsilon' to 'ctll::term<0>'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctll\parser.hpp(66): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\functions.hpp(43): error C2672: 'regular_expression': no matching overloaded function found
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\functions.hpp(43): error C2783: 'ctre::regular_expression<RE> ctre::regular_expression(void) noexcept': could not deduce template argument for 'RE'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\wrapper.hpp(34): note: see declaration of 'ctre::regular_expression'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\functions.hpp(43): error C2780: 'ctre::regular_expression<RE> ctre::regular_expression(RE) noexcept': expects 1 arguments - 0 provided
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\wrapper.hpp(35): note: see declaration of 'ctre::regular_expression'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\functions.hpp(43): error C2641: cannot deduce template argument for 'ctre::regular_expression'
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\functions.hpp(43): error C2512: 'ctre::regular_expression': no appropriate default constructor available
1>c:\projects\libraries\ctre\compile-time-regular-expressions\include\ctre\functions.hpp(43): note: No constructor could take the source type, or constructor overload resolution was ambiguous

Inconsistencies with `ctll::push<T...>`

Both ctre/pcre.hpp and ctre/simple.hpp use ctll::push<T...> from ctll/grammars.hpp but neither files includes it. Given that those two files are generated, and the generator isn't available (cf #37), I can't fix it.

Furthermore, the uses of ctll::push<T...> should be consistent with the implementation of the overload set of the move() functions in ctll/parser.hpp. In that overload set, the resolution is done with ctll::list<T...> instead of the alias ctll::push<T...> witch is surprising since the parsing is using the return type of the functions declared in ctre/pcre.hpp/ctre/simple.hpp:

auto current_term = get_current_term<Pos>();
auto rule = decltype(grammar.rule(top_symbol,current_term))();
return move<Pos>(rule, current_term, stack, previous_subject);

gcc 7.2.0 compilation error

$ make
g++ -std=c++1z -Wno-gnu-string-literal-operator-template -O3 -Iinclude -c math.cpp -o math.o
math.cpp: In function ‘int main()’:
math.cpp:211:2: error: non-constant condition for static assertion
static_assert("2*(3+2)+32"_expr == 42);
^~~~~~~~~~~~~
In file included from include/static-decider.hpp:4:0,
from math.cpp:1:
math.cpp:211:16: in constexpr expansion of ‘operator""_expr<char, '2', '*', '(', '3', '+', '2', ')', '+', '3', '2'>()’

... and the rest of errors

Put version number into ctre.hpp

Please add the library's version number to the single-header file (ctre.hpp) so that users can check which version they already have and whether they need to update.

Also, please supply a ctre::version() function (or whatever name you prefer) to make the version number accessible to application programs.

MSVC: linker stuck for hours

With the latest optimization, the MSVC linker (64-bit version, compiling 32-bit code) didn't finish after more than one hour (using LTCG). The memory usage is at 2-4GB, CPU usage is constantly around 70-80%. Before, the linker took a couple of minutes for the same code.

Does setting CTRE_DISABLE_GREEDY_OPT help in this case?

Warning for tuple_size and tuple_element with -Wall on clang++/g++ on Linux

I get these warnings on g++ 8.2.1 and clang++ 7.0.1 when compiling on Linux with -Wall set:

ctre/ctre/include/ctre/return_type.hpp:229:34: warning: 'tuple_size' defined as a class template here but previously declared as a struct template [-Wmismatched-tags]
        template <typename... Captures> class tuple_size<ctre::regex_results<Captures...>> : public std::integral_constant<size_t, ctre::regex_results<Captures...>::size()> { };
                                        ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/array:350:5: note: did you mean class here?
    struct tuple_size;
    ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/utility:84:5: note: did you mean class here?
    struct tuple_size;
    ^

...

ctre/ctre/include/ctre/return_type.hpp:231:44: warning: 'tuple_element' defined as a class template here but previously declared as a struct template [-Wmismatched-tags]
        template <size_t N, typename... Captures> class tuple_element<N, ctre::regex_results<Captures...>> {
                                                  ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/array:359:5: note: did you mean class here?
    struct tuple_element;
    ^
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1/../../../../include/c++/8.2.1/utility:110:5: note: did you mean class here?
    struct tuple_element;
    ^

It looks like a previous iteration of that file uses struct - is this because your compiler (MSVC?) uses class for those forward declarations instead? Do you know of any way to cleanly get rid of those warnings apart from editing the file itself?

Relicensing to Apache License v2.0 (with LLVM extension)

I decided to relicense CTRE to Apache 2.0 with LLVM extension.

More Syntax errors

msvc branch again

static constexpr inline auto pattern = ctll::basic_fixed_string{ "[+]" };

gives "Regular Expression contains syntax error."

All other than ']', '\' and '-' should be acceptable as normal characters?

Syntax Error

The following gives a compile time error "Regular Expression contains syntax error."

static constexpr inline auto pattern = ctll::basic_fixed_string{ "[A-Z_a-z]" };

(This is using the msvc branch)

Bracket expression with leading or trailing hyphen-minus, '[-xyz]' fails to compile.

In POSIX regex when the hyphen-minus, '-', is first or last in a bracket expression it is translated as a literal and not a range separator. This fails on CTRE though a workaround is to use hex "\x2D" instead of "-"

Example:

#include <string_view>
#include "ctre.hpp"

using namespace std::string_view_literals;
using namespace ctre::literals;
static_assert("^[\\x2D /]+?$"_ctre.match("/- "sv)); // Workaround using hex
static_assert("^[ /\\x2D]+?$"_ctre.match("/- "sv)); // Workaround using hex
static_assert("^[- /]+?$"_ctre.match("/- "sv)); // Compilation failure
static_assert("^[ /-]+?$"_ctre.match("/- "sv)); // Compilation failure

View on Compiler Explorer using gcc8.2/clang7.0.0 with CTRE trunk (v2 gives the same error).

Doesn't compile with gcc 9

Hello,

I don't know if it's a gcc9-related bug, or a bug in ctre (current master, 91fce35):

➜  make
In file included from include/ctre.hpp:4,
                 from tests/range.cpp:1:
include/ctre/literals.hpp:54:1: error: literal operator template ‘constexpr auto ctre::literals::operator""_ctre()’ has invalid parameter list;  Expected non-type template parameter pack <char...>   or single non-type parameter of class type
   54 | }
      | ^
include/ctre/literals.hpp:81:1: error: literal operator template ‘constexpr auto ctre::test_literals::operator""_ctre_test()’ has invalid parameter list;  Expected non-type template parameter pack <char...>   or single non-type parameter of class type
   81 | }
      | ^
include/ctre/literals.hpp:93:1: error: literal operator template ‘constexpr auto ctre::test_literals::operator""_ctre_gen()’ has invalid parameter list;  Expected non-type template parameter pack <char...>   or single non-type parameter of class type
   93 | }
      | ^
include/ctre/literals.hpp:104:1: error: literal operator template ‘constexpr auto ctre::test_literals::operator""_ctre_syntax()’ has invalid parameter list;  Expected non-type template parameter pack <char...>   or single non-type parameter of class type
  104 | }
      | ^
cc1plus: warning: unrecognized command line option ‘-Wno-gnu-string-literal-operator-template’
make: *** [Makefile:31: tests/range.o] Error 1

My g++ is really recent (3 days old, from trunk).

➜  g++ --version
g++ (GCC) 9.0.1 20190126 (experimental)
Copyright © 2019 Free Software Foundation, Inc.

The last commit that I can compile is 7eff180 (with a lot of warnings).


On previous commits (like e121f1e), but more recent than 7eff180 I had this error:

tests/matching2.cpp:192:42: error: unable to find string literal operator ‘operator""sv’ with ‘const char8_t [4]’, ‘long unsigned int’ arguments
  192 | static_assert(CTRE_CREATE("^abc$").match(u8"abc"sv));

Travis build is failing with Conan 1.8

 >> Calling 'conan create'
+-----------------+----------------------------------------------------+
| Configuration   | value                                              |
|-----------------+----------------------------------------------------|
| build_modes     |                                                    |
| name            | CTRE                                               |
| profile_name    | /tmp/tmp0KQ5bzconan_package_tools_profiles/profile |
| version         | master                                             |
| user            | ctre                                               |
| channel         | testing                                            |
+-----------------+----------------------------------------------------+
Exporting package recipe
CTRE/master@ctre/testing: Repo origin deduced by 'auto': https://github.com/hanickadot/compile-time-regular-expressions.git
CTRE/master@ctre/testing: Revision deduced by 'auto': 0c7e82f1198862f9b26d5a631137cb6145a48843
Traceback (most recent call last):
  File "/usr/local/bin/run_create_in_docker", line 11, in <module>
    load_entry_point('conan-package-tools==0.19.4', 'console_scripts', 'run_create_in_docker')()
  File "/usr/local/lib/python2.7/dist-packages/cpt/run_in_docker.py", line 45, in run
    runner.run()
  File "/usr/local/lib/python2.7/dist-packages/cpt/runner.py", line 69, in run
    test_folder=self._test_folder)
  File "/usr/local/lib/python2.7/dist-packages/conans/client/conan_api.py", line 84, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/conans/client/conan_api.py", line 349, in create
    self._client_cache, self._plugin_manager)
  File "/usr/local/lib/python2.7/dist-packages/conans/client/cmd/export.py", line 59, in cmd_export
    keep_source)
  File "/usr/local/lib/python2.7/dist-packages/conans/client/cmd/export.py", line 142, in _export_conanfile
    output, paths, conan_ref)
  File "/usr/local/lib/python2.7/dist-packages/conans/client/cmd/export.py", line 96, in _capture_export_scm_data
    scm_data)
  File "/usr/local/lib/python2.7/dist-packages/conans/client/cmd/export.py", line 103, in _replace_scm_data_in_conanfile
    tree = ast.parse(content)
  File "/usr/lib/python2.7/ast.py", line 37, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 0
SyntaxError: encoding declaration in Unicode string
Traceback (most recent call last):
  File ".conan/build.py", line 24, in <module>
    builder.run()
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/cpt/packager.py", line 386, in run
    self.run_builds(base_profile_name=base_profile_name)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/cpt/packager.py", line 486, in run_builds
    docker_entry_script=self.docker_entry_script)
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/cpt/runner.py", line 190, in run
    raise Exception("Error building: %s" % command)
Exception: Error building: sudo -E docker run --rm -v "/home/travis/build/hanickadot/compile-time-regular-expressions:/home/conan/project" -e CONAN_GCC_VERSIONS="8" -e CONAN_DOCKER_IMAGE="lasote/conangcc8" -e CONAN_LOGIN_USERNAME="hanickadot" -e CONAN_PASSWORD="[secure]" -e CONAN_CHANNEL="testing" -e CONAN_UPLOAD="https://api.bintray.com/conan/hanickadot/ctre@True@upload_repo" -e CONAN_REFERENCE="CTRE/master@ctre/testing" -e CPT_PROFILE="@@include(default)@@@@[settings]@@arch=x86@@build_type=Release@@compiler=gcc@@compiler.version=8@@[options]@@@@[env]@@@@[build_requires]@@@@" -e CONAN_USERNAME="ctre" -e CONAN_TEMP_TEST_FOLDER="1" -e CPT_UPLOAD_RETRY="3" -e CPT_TEST_FOLDER=".conan/test_package" -e PIP_DISABLE_PIP_VERSION_CHECK="1"  lasote/conangcc8 /bin/sh -c " cd project &&  run_create_in_docker "

MSVC: first.hpp: error C2664: cannot convert argument 2 from 'ctll::list' to 'ctll::list<>

One of the latest commits caused this regression with MSVC 16.1 Preview 2:

29>C:\Projects\Libraries\ctre\compile-time-regular-expressions\include\ctre\first.hpp(244,9): error C2664: 'auto ctre::first<>(ctll::list<>,ctll::list<>) noexcept': cannot convert argument 2 from 'ctll::list' to 'ctll::list<>'
29>C:\Projects\Libraries\ctre\compile-time-regular-expressions\include\ctre\first.hpp(244,9): error C2664: with
29>C:\Projects\Libraries\ctre\compile-time-regular-expressions\include\ctre\first.hpp(244,9): error C2664: [
29>C:\Projects\Libraries\ctre\compile-time-regular-expressions\include\ctre\first.hpp(244,9): error C2664: A=ctre::negative_setctre::space_chars
29>C:\Projects\Libraries\ctre\compile-time-regular-expressions\include\ctre\first.hpp(244,9): error C2664: ]

`desatomat` tool isn't available. Therefore some files cannot be regenerated.

Hello,
I noticed that the include/ctre/pcre.hpp rule uses /www/root/desatomat/console/desatomat.php witch isn't available on this repo, and I didn't found information about it either. The best I found was your desatomat2 repo, witch is unfortunately empty.

It also look like include/ctre/simple.hpp is missing a construction rule in the Makefile, since their is a comment telling that it was generated with desatomat.

Missing common.hpp

Hey there, I just noticed that tests/benchmark-exec/common.hpp is not commited.

I think it was to be added in this commit: 4e3c264

Non-standard capturing group index order

Given this code:

static constexpr auto pattern = ctll::basic_fixed_string{ "((\\d)(\\d))" };
void Test()
{
	auto m = ctre::re<pattern>().match("12");
	auto a1v = m.get<1>().to_view();
	auto a2v = m.get<2>().to_view();
	auto a3v = m.get<3>().to_view();
}

I'm expecting that a1v="12", a2v="1", a3v="2", however the values are:
a1v="1"
a2v="2"
a3v="12"

I do have an idea why it is this way, but is this something that should be changed?

MSVC: error C2247: 'ctll::fixed_string<156>::size' not accessible

I get the following error with the latest update:
error C2247: 'ctll::fixed_string<156>::size' not accessible because 'ctll::basic_fixed_string<wchar_t,156>' uses 'private' to inherit from 'ctll::fixed_string<156>'

In the call to trampoline

	template <typename Subject = empty_subject> static constexpr auto trampoline_decide(Subject subject = {}) noexcept {
		// there will be no recursion, just sequence long as the input
		return trampoline_decide(subject, std::make_index_sequence<input.size()>());
	}

The code to reproduce looks like this:
static constexpr auto Pattern = ctll::basic_fixed_string{ LR"(^\s*(\d+)\s+:(\S):$(\S+?)$(\S+?)$(\S+))" };

But it happens with every basic_fixed_string or fixed_string call.
Visual Studio 2019 RC1

I didn't find any instructions on how to build this Library on MSVC.

Hi hanickadot, thanks for this library at first.

We want to build this lib on MSVC, but I didn't find any instructions. Could you tell me where I can get it? Or can you provide the build steps for me? If you need any other question please let me know. Thank you very much.

Our environment: VS 2017 + Windows Server 2016

Comprehensive list of required changes to make the lib C++14 compatible

Hi,

I think we talked about this some weeks ago on twitter, but I would love if you could share the conflictive points you're aware of. I want to decide whether to bump my projects to C++17 or keep C++14 support, and I need your library to write a command line arg parser. I'm not sure whether updating all my toolchains or accepting the challenge of porting CTRE to C++14 will be more challenging.

wchar_t based pattern parsing problem

The library fails to compile with MSVC 16.0 (Visual Studio 2019 Preview 1).

error C2338: Regular Expression contains syntax error.
find_captures.hpp(11): error C2672: 'find_captures': no matching overloaded function found
etc

Syntax error for both [>] and [<]

Nither >, nor < can be placed inside character range:

#include <ctre.hpp>

using namespace ctre::literals;
constexpr auto mini1 = "[>]"_ctre;
constexpr auto mini2 = "[<]"_ctre;

See also Compiler Explorer and Regex 101.

My setup:

$ git -C external/ctre describe --always
v2.3-46-g7a975b0
$ c++ --version
c++ (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ctre::captured_content<0, void>::storage<const char *> can be passed to std::string constructor

I was trying to get a copy of the match into a std::vector<std::string> and figured out a few way that compiled, but not all of them worked as expected.

  • vector_of_string.push_back( std::string{ match.get<0>().to_view() } ); works but is very verbose and creates a needless copy.
  • vector_of_string.push_back( match.get<0>() ); doesn't compile, which is perfectly fine.
  • vector_of_string.emplace_back( match.get<0>().to_view() ); works without extra copies, but is still verbose.
  • vector_of_string.emplace_back( match.get<0>() ); compiles, but matches an unexpected explicit std::string constructor.

Here's an example that demonstrates the unexpected constructor matching:

#include <iostream>
#include "include/ctll.hpp"
#include "include/ctre.hpp"

static constexpr auto pattern = ctll::basic_fixed_string{ ".*" };

int main(void) {
	std::string str{ "a\ta\nc\td" };
	std::string str2 = std::string{ ctre::re< pattern >().match( str ).get< 0 >() };
	std::cout<< ( static_cast< int >( str2[ 0 ] ) == 1 );
}

Looking at the cppreference, line 10 of the sample code matches contructor 10 in the following list of std::string constructors: https://en.cppreference.com/w/cpp/string/basic_string/basic_string

I can think of three possible solutions:

  • Writing operator std::string and making ctre::captured_content<0, void>::storage<const char *> actually convertible to std::string. Probably marked explicit, so users keep in mind that the operator introduces a copy.
  • Banning this, I assume accidentally, allowed conversion and doing nothing more. Makes extracting a std::string verbose, but maybe copies should be that verbose.
  • Banning the conversion and implementing ctre::captured_content<0, void>::storage<const char *>::to_string() that would work similar to to_view().

static_assert gives error C2131: expression did not evaluate to a constant

msvc branch, VS 15.8.9:

static constexpr inline auto pattern34 = ctll::basic_fixed_string{ "[0-9]{1,8}" };

static_assert(ctre::re().match("123"));

1>h:\source\ben\scratchpad\main.cpp(1469): error C2131: expression did not evaluate to a constant
1>h:\source\ben\compile-time-regular-expressions-master\include\ctre\return_type.hpp(54): note: failure was caused by a read of an uninitialized symbol
1>h:\source\ben\compile-time-regular-expressions-master\include\ctre\return_type.hpp(54): note: see usage of 'ctre::captured_content<0,void>::storage::_matched'
1> with
1> [
1> Iterator=const char *
1> ]

Build fail on MSVC latest and g++ 7.3

MSVC 16.0.0 fails with in a ICE something like,

1>test_ctre.cpp
1>common\compile-time-regular-expressions\include\ctre\wrapper.hpp(31): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
1>common\compile-time-regular-expressions\include\ctre\wrapper.hpp(70): note: see reference to class template instantiation 'ctre::RangeLikeType' being compiled
1> with
1> [
1> Range=std::string &
1> ]
1>test_ctre\test_ctre.cpp(12): note: see reference to function template instantiation 'auto ctre::regex_match_t<ctre::sequence<B,ctre::character<32>,A,ctre::character<33>>>::operator ()std::string&(std::string &) noexcept const' being compiled
1> with
1> [
1> B=ctre::capture<1,ctre::plus<ctre::setctre::word_chars>>,
1> A=ctre::capture<2,ctre::plus<ctre::setctre::word_chars>>
1> ]
1>common\compile-time-regular-expressions\include\ctre\wrapper.hpp(31): fatal error C1001: An internal error has occurred in the compiler.
1>(compiler file 'd:\agent_work\3\s\src\vctools\Compiler\CxxFE\sl\p1\c\symbols.c', line 24466)

Code compiles with a CTRE version from around Feb 15. I've reported the ICE to Microsoft.

Also I tried g++ 7.3.0 on Ubuntu 18.04. Everything fails to build here too, but in a different way. This could be my finger trouble. Anyway 'make' in the ctre directory will do it, but I've tried a simple test file and get the same result.

paul@fasolt:~/compile-time-regular-expressions$ make
time g++ -std=c++17 -Iinclude -O3 -pedantic -Wall -Wextra -MMD -c tests/range.cpp -o tests/range.o
In file included from include/ctre/../ctll/parser.hpp:4:0,
from include/ctre/../ctll.hpp:4,
from include/ctre/literals.hpp:4,
from include/ctre.hpp:4,
from tests/range.cpp:1:
include/ctre/../ctll/fixed_string.hpp:176:62: error: parameter ‘’ includes reference to array of unknown bound ‘const T []’
template constexpr fixed_string(const T (&)[]) noexcept {
^~~~~~~~
Command exited with non-zero status 1
0.71user 0.07system 0:01.60elapsed 49%CPU (0avgtext+0avgdata 90368maxresident)k
35344inputs+8outputs (132major+19660minor)pagefaults 0swaps
Makefile:31: recipe for target 'tests/range.o' failed
make: *** [tests/range.o] Error 1

This is gcc 7.3.0.

Multiline regex support

First of all, thanks for this library.

I did manage to get CTRE to work for a test sample, but just like std::regex CTRE doesn't seem to support multiline regex patterns, where ^ and $ also match line separators.

Example where this would be useful:

#include <cassert>
#include <iostream>
#include <vector>
#include "include/ctll.hpp"
#include "include/ctre.hpp"

static constexpr auto pattern = ctll::basic_fixed_string{ "^(\\w)\\t(\\w)(?:$|\\n|\\r\\n)" };

int main(void) {
	std::string str{ "a\ta\nc\td" };
	auto start = str.begin();
	auto end = str.end();
	std::vector<std::string> first;
	std::vector<std::string> second;
	while ( auto match = ctre::re<pattern>().search_2( start, end ) ) {
		start = match.get_end_position();
		first.emplace_back( match.get<1>().to_view() );
		second.emplace_back( match.get<2>().to_view() );
	}
	assert( (first == std::vector<std::string>{ "a", "c" }) );
	assert( (second == std::vector<std::string>{ "a", "d" }) );
	return 0;
}

The above example works currently, but with multiline regex the last, non-capturing, group could be just $. This kind of multiline regex works currently in Boost.Regex and is supposed to work in C++17 std::regex, but none of the three most common STL implementations has multiline available.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.