boostorg/locale

Boost.Locale

Boost.Locale

Part of the Boost C++ Libraries.

Boost.Locale is a library that provides high quality localization facilities in a C++ way. It was originally designed a part of CppCMS - a C++ Web Framework project and then contributed to Boost.

Boost.Locale gives powerful tools for development of cross-platform localized software - the software that talks to users in their language.

Provided Features:

• Correct case conversion, case folding and normalization.
• Collation (sorting), including support for 4 Unicode collation levels.
• Date, time, timezone and calendar manipulations, formatting and parsing, including transparent support for calendars other than Gregorian.
• Boundary analysis for characters, words, sentences and line-breaks.
• Number formatting, spelling and parsing.
• Monetary formatting and parsing.
• Powerful message formatting (string translation) including support for plural forms, using GNU catalogs.
• Character set conversion.
• Transparent support for 8-bit character sets like Latin1
• Support for char and wchar_t
• Experimental support for C++11 char16_t and char32_t strings and streams.

Boost.Locale enhances and unifies the standard library's API the way it becomes useful and convenient for development of cross-platform and "cross-culture" software.

In order to achieve this goal Boost.Locale uses the-state-of-the-art Unicode and Localization library: ICU - International Components for Unicode.

Boost.Locale creates the natural glue between the C++ locales framework, iostreams, and the powerful ICU library.

Boost.Locale provides non-ICU based localization support as well. It is based on the operating system native API or on the standard C++ library support. Sacrificing some less important features, Boost.Locale becomes less powerful but lighter and easier to deploy and use library.

License

Distributed under the Boost Software License, Version 1.0.

Properties

• C++11
• Formatted with clang-format, see tools/format_sources.sh

Build Status

Branch	GH Actions	Appveyor	Drone	codecov.io	Deps	Docs	Tests
master
develop

Directories

Name	Purpose
doc	Documentation
examples	Examples
include	Headers
src	Source files
test	Unit tests

More information

• Ask questions
• Report bugs: Be sure to mention Boost version, platform and compiler you're using. A small compilable code sample to reproduce the problem is always good as well.
• Submit your patches as pull requests against develop branch. Note that by submitting patches you agree to license your modifications under the Boost Software License, Version 1.0.
• Discussions about the library are held on the Boost developers mailing list. Be sure to read the discussion policy before posting and add the [locale] tag at the beginning of the subject line.

locale's Issues

Boost-locale compiled with vcpkg won't pass program linking

When I try to use boost::locale::conv::from_utf he never works well!

• When I compile code like this with a version of x86-windows boost-locale installed by vcpkg (no icu backend installed) I get this linking problem: unresolved external command

  std::string u8ToGbk(const std::u8string &U8Str) {
      return boost::locale::conv::from_utf(U8Str, "GBK");    
  }

  

  But it links successfully when I use the following code, which seems to solve my problem

  std::string u8ToGbk(const std::string &U8Str) {
      return boost::locale::conv::from_utf(U8Str, "GBK");
  }

  msvc 17.7.0 c++20

• But the situation is even worse when I'm using the version of x86-windows-static installed by vcpkg, and I can barely use the function whether it's using u8 or std::string

  std::string u8ToGbk(const std::string &U8Str) {
    return boost::locale::conv::from_utf(U8Str, "GBK");
  }

  

  msvc 17.7.0 c++20

• Here's my example on compoler explorer (https://godbolt.org/z/M51oMzYec), which seems to be consistent with the problem I'm having

UTF-8 encoding not detected correctly during boundary detection using ICU 3.6+

ICU supports direct, in-place boundary detection for a c-strings encoded in UTF-8. boost::locale::boundary::impl_icu::do_map< char >() takes advantage of this when this check succeeds: https://github.com/boostorg/locale/blob/develop/src/icu/boundary.cpp#L142

This check only succeeds when the encoding is literally "UTF-8" and fails for similar strings, such as "utf-8". I submit that this is a bug, and this check should pass when the encoding is "utf-8" or any other string value meaning but not exactly equal to "UTF-8".

Moreover, when generating a locale, boost::locale::util::locale_data::parse_from_encoding() normalizes the encoding string to lowercase: https://github.com/boostorg/locale/blob/develop/src/util/locale_data.cpp#L86

This means that generating locale "en_US.UTF-8" results in an encoding value of "utf-8", which will be affected by this issue. This value is not exposed to the user, so there is not an easily accessible workaround.

Failing tests

With ./b2 toolset=gcc-10 libs/locale/test cxxstd=11 -j4 variant=debug I see many failing tests with recent ICU

One of those is "USD 1,345.00" instead of "USD1,345.00", other look more confusing.
See https://github.com/Flamefire/locale/runs/3748013166?check_suite_focus=true or https://github.com/Flamefire/locale/actions in general

@artyom-beilis As you wrote the tests I'm not sure if the tests are wrong or the results.

icu only build on windows fails

Compiling boost::locale with only icu enabled fails with a test_winapi_formatting.cpp link error.

Build configuration:

boost.locale.icu=on boost.locale.iconv=off boost.locale.winapi=off boost.locale.std=off boost.locale.posix=off

Error:

test_winapi_formatting.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) unsigned int __cdecl boost::locale::i\
mpl
_win::locale_to_lcid(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (__imp_?loca\
le_
to_lcid@impl_win@locale@boost@@YAIAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "void\
 __
cdecl test_main(int,char * *)" (?test_main@@YAXHPEAPEAD@Z)
bin.v2\libs\locale\test\test_winapi_formatting.test\msvc-14.1\debug\threadapi-win32\threading-multi\test_winapi_formatting.exe : fa\
tal
error LNK1120: 1 unresolved externals

Platform:

• Windows 10 version 22H2
• Windows Python, for the ICU build
• ICU 71.1
• Boost 1.82.0
• Visual Studio 2017 Community Edition version 15.9.55

Note: Compiling with only icu enabled does work on MacOS and Linux.

The following Cygwin bash script duplicates the error:

#! /bin/bash -x

# Download icu and boost, build icu, build boost --with-locale and
# the buile boost::locale test with only icu enabled, as follows:
#
#  boost.locale.icu=on boost.locale.iconv=off boost.locale.winapi=off boost.locale.std=off boost.locale.posix=off
#
# The boost:locale test build will fail a link error:
#
# test_winapi_formatting.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) unsigned int __cdecl boost::locale::impl
# _win::locale_to_lcid(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (__imp_?locale_
# to_lcid@impl_win@locale@boost@@YAIAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "void __
# cdecl test_main(int,char * *)" (?test_main@@YAXHPEAPEAD@Z)
# bin.v2\libs\locale\test\test_winapi_formatting.test\msvc-14.1\debug\threadapi-win32\threading-multi\test_winapi_formatting.exe : fatal
#  error LNK1120: 1 unresolved externals
#
#
# Test Platform:
#   Windows 10 version 22H2
#   Visual Studio 2017 Community Edition version 15.9.55
#   Cygwin to run this script

set -e

# Windows path to local Visual Studio environment setup batch file.
WIN_VCVARSALL_BAT="c:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvarsx86_amd64.bat"

# Don't raise ICU version beyond 71.1 until
# https://github.com/boostorg/locale/issues/176 is resolved.
ICU_VER="71.1"
ICU_RELEASE=release-$(echo $ICU_VER | sed s/\\./-/)
ICU4C_SRC=icu4c-$(echo $ICU_VER | sed s/\\./_/)-src

BOOST_VER="1.82.0"
BOOST_SRC=boost_$(echo $BOOST_VER | sed s/\\./_/g)

# setup error trap
function ErrorTrap {
    echo FAILED
}
trap ErrorTrap INT TERM EXIT

DIR=boost_locale_windows_icu_only.$(date "+%m-%d-%y_%H-%M-%S").d
mkdir $DIR
cd $DIR

# setup log
exec > >(tee boost_locale_windows_icu_only.log)
exec 2> >(tee -a boost_locale_windows_icu_only.log)

# Run windows build command by writing a temporary windows bat file
# and executing it using cmd.exe.  The visual studio environment is
# configured (vcvarsall.bat) before the build command.
function wincmd {
    local cmd="$*"
    local batfile=$(mktemp wincmd_tmp_XXXXXX.bat)

    cat <<__EOF > "$batfile"
call "$WIN_VCVARSALL_BAT"
$cmd
__EOF

    cat $batfile

    cmd.exe /c "$batfile"
}

# get icu
wget --quiet --no-check-certificate "https://github.com/unicode-org/icu/releases/download/${ICU_RELEASE}/${ICU4C_SRC}.tgz"
tar xvzf "${ICU4C_SRC}".tgz
ICU_PATH=${PWD}/icu

# get boost
wget --quiet --no-check-certificate "https://boostorg.jfrog.io/artifactory/main/release/"${BOOST_VER}"/source/"${BOOST_SRC}".tar.gz"
tar xvzf "${BOOST_SRC}".tar.gz

(
    cd icu/source
    wincmd msbuild allinone\\allinone.sln /p:Configuration=Debug /p:Platform=x64
    wincmd msbuild allinone\\allinone.sln /p:Configuration=Debug /p:Platform=x64 /target:makedata
)

(
    cd ${BOOST_SRC}

    wincmd bootstrap.bat

    wincmd b2 debug -j4 \
          -sICU_PATH=$(cygpath -w "${ICU_PATH}") \
          boost.locale.icu=on boost.locale.iconv=off boost.locale.winapi=off boost.locale.std=off boost.locale.posix=off \
          --with-locale \
          toolset="msvc-14.1" \
          architecture=x86 \
          address-model=64

    # Fails with test_winapi_formatting.cpp link error.
    # Note: cxxflags="\wd4702" works around https://github.com/boostorg/locale/issues/180
    unset TZ;
    wincmd b2 debug -j4 \
          -sICU_PATH=$(cygpath -w "${ICU_PATH}") \
          boost.locale.icu=on boost.locale.iconv=off boost.locale.winapi=off boost.locale.std=off boost.locale.posix=off \
          --with-locale \
          toolset="msvc-14.1" \
          architecture=x86 \
          address-model=64 \
          cxxflags="\wd4702" \
          libs/locale/test
)

# clear error trap
trap - INT TERM EXIT

echo PASSED

Undefined behaviour when getting address in do_basic_transform

in do_basic_transform https://github.com/boostorg/locale/blob/develop/src/icu/collator.cpp#L90
line
int len = collate->getSortKey(str,&tmp[0],tmp.size());

seems wrong as there is no way to make sure tmp is not a 0 sized vector which leads &tmp[0] to be undefined behaviour.
Moreover, on my fedora 30, a vector assertion fails __builtin_expect(__n < this->size(), true) and leads the program to crash.

A fix could be to use tmp.data() instead (if this function is really supposed to handle 0 lenght entry).

error C2220: warning treated as error, warning C4702: unreachable code

On Windows, the boost::locale test_std_convert.cpp test will fail to compile due to an unreachable code warning:

 ...\boost_1_82_0\libs\locale\test\test_std_formatting.c
 pp(152) : error C2220: warning treated as error - no 'object' file generated
 ...\boost_1_82_0\libs\locale\test\test_std_formatting.c
 pp(152) : warning C4702: unreachable code

Test Platform:

• Windows 10 version 22H2
• Windows Python, for the ICI build
• Visual Studio 2017 Community Edition version 15.9.55
• ICU 71.1
• Boost 1.82.0
• boost.locale.icu=on boost.locale.winapi=on

Compiling the boost::locale tests with cxxflags="\wd4702" works around the problem.

Here is a Cygwin bash script that downloads icu and boost, compiles both, and will as with the unreachable code warning noted above.

#! /bin/bash -x

# Download icu and boost, build icu, build boost --with-locale, and 
# build the boost::locale tests.
#
# The boost::locale test_std_formatting.cpp file, and other test files, will fail to
# compile due to an unreachable code warning:
#
# ...\boost_1_82_0\libs\locale\test\test_std_formatting.c
# pp(152) : error C2220: warning treated as error - no 'object' file generated
# ...\boost_1_82_0\libs\locale\test\test_std_formatting.c
# pp(152) : warning C4702: unreachable code
#
# Compiling the boost::locale tests with cxxflags="\wd4702" works around the
# problem by disabling the unreachable code warning.
#
# Test Platform:
#   Windows 10 version 22H2
#   Visual Studio 2017 Community Edition version 15.9.55
#   Cygwin to run this script

set -e

# Windows path to local Visual Studio environment setup batch file.
WIN_VCVARSALL_BAT="c:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvarsx86_amd64.bat"

# Don't raise ICU version beyond 71.1 until
# https://github.com/boostorg/locale/issues/176 is resolved.
ICU_VER="71.1"
ICU_RELEASE=release-$(echo $ICU_VER | sed s/\\./-/)
ICU4C_SRC=icu4c-$(echo $ICU_VER | sed s/\\./_/)-src

BOOST_VER="1.82.0"
BOOST_SRC=boost_$(echo $BOOST_VER | sed s/\\./_/g)

# setup error trap
function ErrorTrap {
    echo FAILED
}
trap ErrorTrap INT TERM EXIT

DIR=boost_locale_windows_unreachable_warning.$(date "+%m-%d-%y_%H-%M-%S").d
mkdir $DIR
cd $DIR

# setup log
exec > >(tee boost_locale_windows_unreachable_warning.log)
exec 2> >(tee -a boost_locale_windows_unreachable_warning.log)

# Run windows build command by writing a temporary windows bat file
# and executing it using cmd.exe.  The visual studio environment is
# configured (vcvarsall.bat) before the build command.
function wincmd {
    local cmd="$*"
    local batfile=$(mktemp wincmd_tmp_XXXXXX.bat)

    cat <<__EOF > "$batfile"
call "$WIN_VCVARSALL_BAT"
$cmd
__EOF

    cat $batfile

    cmd.exe /c "$batfile"
}

# get icu
wget --quiet --no-check-certificate "https://github.com/unicode-org/icu/releases/download/${ICU_RELEASE}/${ICU4C_SRC}.tgz"
tar xvzf "${ICU4C_SRC}".tgz
ICU_PATH=${PWD}/icu

# get boost
wget --quiet --no-check-certificate "https://boostorg.jfrog.io/artifactory/main/release/"${BOOST_VER}"/source/"${BOOST_SRC}".tar.gz"
tar xvzf "${BOOST_SRC}".tar.gz

(
    cd icu/source
    wincmd msbuild allinone\\allinone.sln /p:Configuration=Debug /p:Platform=x64
    wincmd msbuild allinone\\allinone.sln /p:Configuration=Debug /p:Platform=x64 /target:makedata
)

(
    cd ${BOOST_SRC}

    wincmd bootstrap.bat

    wincmd b2 debug -j4 \
          -sICU_PATH=$(cygpath -w "${ICU_PATH}") \
          boost.locale.icu=on boost.locale.iconv=off boost.locale.winapi=on boost.locale.std=off boost.locale.posix=off \
          --with-locale \
          toolset="msvc-14.1" \
          architecture=x86 \
          address-model=64

    # This fails with "error C2220: warning treated as error" do to
    # "warning C4702: unreachable code" when compiling
    # test_std_formatting.cpp. Compiling with cxxflags="\wd4702" works
    # around the problem.
    unset TZ;
    wincmd b2 debug -j4 \
          -sICU_PATH=$(cygpath -w "${ICU_PATH}") \
          boost.locale.icu=on boost.locale.iconv=off boost.locale.winapi=on boost.locale.std=off boost.locale.posix=off \
          --with-locale \
          toolset="msvc-14.1" \
          architecture=x86 \
          address-model=64 \
          libs/locale/test
)

# clear error trap
trap - INT TERM EXIT

echo PASSED

GCC suggest-override warnings

Complete list of warnings when Boost 1.72 is built with GCC 7.5 with -Wsuggest-override added to cxxflags. Duplicated warnings from same location are omitted:

./boost/locale/collator.hpp:164:21: warning: ‘int boost::locale::collator::do_compare(const char_type*, const char_type*, const char_type*, const char_type*) const [with CharType = char; boost::locale::collator::char_type = char]’ can be marked override [-Wsuggest-override]
./boost/locale/collator.hpp:173:29: warning: ‘boost::locale::collator::string_type boost::locale::collator::do_transform(const char_type*, const char_type*) const [with CharType = char; boost::locale::collator::string_type = std::__cxx11::basic_string; boost::locale::collator::char_type = char]’ can be marked override [-Wsuggest-override]
./boost/locale/collator.hpp:181:22: warning: ‘long int boost::locale::collator::do_hash(const char_type*, const char_type*) const [with CharType = char; boost::locale::collator::char_type = char]’ can be marked override [-Wsuggest-override]
./boost/locale/generic_codecvt.hpp:471:39: warning: ‘std::codecvt_base::result boost::locale::generic_codecvt::do_unshift(mbstate_t&, char*, char*, char*&) const [with CharType = wchar_t; CodecvtImpl = boost::locale::utf8_codecvt; mbstate_t = __mbstate_t]’ can be marked override [-Wsuggest-override]
./boost/locale/generic_codecvt.hpp:476:17: warning: ‘int boost::locale::generic_codecvt::do_encoding() const [with CharType = wchar_t; CodecvtImpl = boost::locale::utf8_codecvt]’ can be marked override [-Wsuggest-override]
./boost/locale/generic_codecvt.hpp:480:17: warning: ‘int boost::locale::generic_codecvt::do_max_length() const [with CharType = wchar_t; CodecvtImpl = boost::locale::utf8_codecvt]’ can be marked override [-Wsuggest-override]
./boost/locale/generic_codecvt.hpp:484:18: warning: ‘bool boost::locale::generic_codecvt::do_always_noconv() const [with CharType = wchar_t; CodecvtImpl = boost::locale::utf8_codecvt]’ can be marked override [-Wsuggest-override]
./boost/locale/generic_codecvt.hpp:490:5: warning: ‘int boost::locale::generic_codecvt::do_length(mbstate_t&, const char*, const char*, size_t) const [with CharType = wchar_t; CodecvtImpl = boost::locale::utf8_codecvt; mbstate_t = __mbstate_t; size_t = long unsigned int]’ can be marked override [-Wsuggest-override]
./boost/locale/generic_codecvt.hpp:524:5: warning: ‘std::codecvt_base::result boost::locale::generic_codecvt::do_in(mbstate_t&, const char*, const char*, const char*&, boost::locale::generic_codecvt::uchar*, boost::locale::generic_codecvt::uchar*, boost::locale::generic_codecvt::uchar*&) const [with CharType = wchar_t; CodecvtImpl = boost::locale::utf8_codecvt; mbstate_t = __mbstate_t; boost::locale::generic_codecvt::uchar = wchar_t]’ can be marked override [-Wsuggest-override]
./boost/locale/generic_codecvt.hpp:591:5: warning: ‘std::codecvt_base::result boost::locale::generic_codecvt::do_out(mbstate_t&, const uchar*, const uchar*, const uchar*&, char*, char*, char*&) const [with CharType = wchar_t; CodecvtImpl = boost::locale::utf8_codecvt; mbstate_t = __mbstate_t; boost::locale::generic_codecvt::uchar = wchar_t]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/iconv_codepage.ipp:150:18: warning: ‘bool boost::locale::conv::impl::iconv_from_utf::open(const char*, boost::locale::conv::method_type) [with CharType = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/iconv_codepage.ipp:155:25: warning: ‘std::__cxx11::string boost::locale::conv::impl::iconv_from_utf::convert(const char_type*, const char_type*) [with CharType = char; std::__cxx11::string = std::__cxx11::basic_string; boost::locale::conv::impl::iconv_from_utf::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/iconv_codepage.ipp:167:18: warning: ‘virtual bool boost::locale::conv::impl::iconv_between::open(const char*, const char*, boost::locale::conv::method_type)’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/iconv_codepage.ipp:171:25: warning: ‘virtual std::__cxx11::string boost::locale::conv::impl::iconv_between::convert(const char*, const char*)’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/iconv_codepage.ipp:189:18: warning: ‘bool boost::locale::conv::impl::iconv_to_utf::open(const char*, boost::locale::conv::method_type) [with CharType = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/iconv_codepage.ipp:194:25: warning: ‘boost::locale::conv::impl::iconv_to_utf::string_type boost::locale::conv::impl::iconv_to_utf::convert(const char*, const char*) [with CharType = char; boost::locale::conv::impl::iconv_to_utf::string_type = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/uconv_codepage.ipp:117:22: warning: ‘virtual bool boost::locale::conv::impl::uconv_between::open(const char*, const char*, boost::locale::conv::method_type)’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/uconv_codepage.ipp:136:29: warning: ‘virtual std::__cxx11::string boost::locale::conv::impl::uconv_between::convert(const char*, const char*)’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/uconv_codepage.ipp:32:22: warning: ‘bool boost::locale::conv::impl::uconv_to_utf::open(const char*, boost::locale::conv::method_type) [with CharType = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/uconv_codepage.ipp:51:29: warning: ‘boost::locale::conv::impl::uconv_to_utf::string_type boost::locale::conv::impl::uconv_to_utf::convert(const char*, const char*) [with CharType = char; boost::locale::conv::impl::uconv_to_utf::string_type = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/uconv_codepage.ipp:76:22: warning: ‘bool boost::locale::conv::impl::uconv_from_utf::open(const char*, boost::locale::conv::method_type) [with CharType = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/encoding/uconv_codepage.ipp:95:29: warning: ‘std::__cxx11::string boost::locale::conv::impl::uconv_from_utf::convert(const CharType*, const CharType*) [with CharType = char; std::__cxx11::string = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/boundary.cpp:192:16: warning: ‘boost::locale::boundary::index_type boost::locale::boundary::impl_icu::boundary_indexing_impl::map(boost::locale::boundary::boundary_type, const CharType*, const CharType*) const [with CharType = char; boost::locale::boundary::index_type = std::vector]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/codecvt.cpp:110:21: warning: ‘virtual int boost::locale::impl_icu::uconv_converter::max_len() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/codecvt.cpp:57:22: warning: ‘virtual bool boost::locale::impl_icu::uconv_converter::is_thread_safe() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/codecvt.cpp:62:34: warning: ‘virtual boost::locale::impl_icu::uconv_converter* boost::locale::impl_icu::uconv_converter::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/codecvt.cpp:67:18: warning: ‘virtual uint32_t boost::locale::impl_icu::uconv_converter::to_unicode(const char*&, const char*)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/codecvt.cpp:84:18: warning: ‘virtual uint32_t boost::locale::impl_icu::uconv_converter::from_unicode(uint32_t, char*, const char*)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/collator.cpp:105:45: warning: ‘std::__cxx11::basic_string<_CharT> boost::locale::impl_icu::collate_impl::do_transform(boost::locale::impl_icu::collate_impl::level_type, const CharType*, const CharType*) const [with CharType = char; boost::locale::impl_icu::collate_impl::level_type = boost::locale::collator_base::level_type]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/collator.cpp:111:22: warning: ‘long int boost::locale::impl_icu::collate_impl::do_hash(boost::locale::impl_icu::collate_impl::level_type, const CharType*, const CharType*) const [with CharType = char; boost::locale::impl_icu::collate_impl::level_type = boost::locale::collator_base::level_type]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/collator.cpp:73:29: warning: ‘int boost::locale::impl_icu::collate_impl::do_compare(boost::locale::impl_icu::collate_impl::level_type, const CharType*, const CharType*, const CharType*, const CharType*) const [with CharType = char; boost::locale::impl_icu::collate_impl::level_type = boost::locale::collator_base::level_type]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/conversion.cpp:148:29: warning: ‘virtual std::__cxx11::string boost::locale::impl_icu::utf8_converter_impl::convert(boost::locale::converter_base::conversion_type, const char*, const char*, int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/conversion.cpp:72:29: warning: ‘boost::locale::impl_icu::converter_impl::string_type boost::locale::impl_icu::converter_impl::convert(boost::locale::converter_base::conversion_type, const char_type*, const char_type*, int) const [with CharType = wchar_t; boost::locale::impl_icu::converter_impl::string_type = std::__cxx11::basic_string; boost::locale::impl_icu::converter_impl::char_type = wchar_t]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:138:22: warning: ‘virtual void boost::locale::impl_icu::calendar_impl::set_time(const boost::locale::posix_time&)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:145:22: warning: ‘virtual void boost::locale::impl_icu::calendar_impl::normalize()’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:153:28: warning: ‘virtual boost::locale::posix_time boost::locale::impl_icu::calendar_impl::get_time() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:172:22: warning: ‘virtual void boost::locale::impl_icu::calendar_impl::set_option(boost::locale::abstract_calendar::calendar_option_type, int)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:183:21: warning: ‘virtual int boost::locale::impl_icu::calendar_impl::get_option(boost::locale::abstract_calendar::calendar_option_type) const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:200:22: warning: ‘virtual void boost::locale::impl_icu::calendar_impl::adjust_value(boost::locale::period::marks::period_mark, boost::locale::abstract_calendar::update_type, int)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:213:21: warning: ‘virtual int boost::locale::impl_icu::calendar_impl::difference(const boost::locale::abstract_calendar*, boost::locale::period::marks::period_mark) const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:239:22: warning: ‘virtual void boost::locale::impl_icu::calendar_impl::set_timezone(const string&)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:243:29: warning: ‘virtual std::__cxx11::string boost::locale::impl_icu::calendar_impl::get_timezone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:250:22: warning: ‘virtual bool boost::locale::impl_icu::calendar_impl::same(const boost::locale::abstract_calendar*) const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:272:36: warning: ‘virtual boost::locale::abstract_calendar* boost::locale::impl_icu::icu_calendar_facet::create_calendar() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:89:24: warning: ‘virtual boost::locale::impl_icu::calendar_impl* boost::locale::impl_icu::calendar_impl::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:94:14: warning: ‘virtual void boost::locale::impl_icu::calendar_impl::set_value(boost::locale::period::marks::period_mark, int)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/date_time.cpp:99:13: warning: ‘virtual int boost::locale::impl_icu::calendar_impl::get_value(boost::locale::period::marks::period_mark, boost::locale::abstract_calendar::value_type) const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:158:33: warning: ‘boost::locale::impl_icu::date_format::string_type boost::locale::impl_icu::date_format::format(double, size_t&) const [with CharType = wchar_t; boost::locale::impl_icu::date_format::string_type = std::__cxx11::basic_string; size_t = long unsigned int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:162:33: warning: ‘boost::locale::impl_icu::date_format::string_type boost::locale::impl_icu::date_format::format(int64_t, size_t&) const [with CharType = wchar_t; boost::locale::impl_icu::date_format::string_type = std::__cxx11::basic_string; int64_t = long int; size_t = long unsigned int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:167:33: warning: ‘boost::locale::impl_icu::date_format::string_type boost::locale::impl_icu::date_format::format(int32_t, size_t&) const [with CharType = wchar_t; boost::locale::impl_icu::date_format::string_type = std::__cxx11::basic_string; int32_t = int; size_t = long unsigned int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:172:28: warning: ‘size_t boost::locale::impl_icu::date_format::parse(const string_type&, double&) const [with CharType = char; size_t = long unsigned int; boost::locale::impl_icu::date_format::string_type = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:176:28: warning: ‘size_t boost::locale::impl_icu::date_format::parse(const string_type&, int64_t&) const [with CharType = char; size_t = long unsigned int; boost::locale::impl_icu::date_format::string_type = std::__cxx11::basic_string; int64_t = long int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:180:28: warning: ‘size_t boost::locale::impl_icu::date_format::parse(const string_type&, int32_t&) const [with CharType = char; size_t = long unsigned int; boost::locale::impl_icu::date_format::string_type = std::__cxx11::basic_string; int32_t = int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:51:33: warning: ‘boost::locale::impl_icu::number_format::string_type boost::locale::impl_icu::number_format::format(double, size_t&) const [with CharType = char; boost::locale::impl_icu::number_format::string_type = std::__cxx11::basic_string; size_t = long unsigned int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:58:33: warning: ‘boost::locale::impl_icu::number_format::string_type boost::locale::impl_icu::number_format::format(int64_t, size_t&) const [with CharType = char; boost::locale::impl_icu::number_format::string_type = std::__cxx11::basic_string; int64_t = long int; size_t = long unsigned int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:66:33: warning: ‘boost::locale::impl_icu::number_format::string_type boost::locale::impl_icu::number_format::format(int32_t, size_t&) const [with CharType = char; boost::locale::impl_icu::number_format::string_type = std::__cxx11::basic_string; int32_t = int; size_t = long unsigned int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:78:28: warning: ‘size_t boost::locale::impl_icu::number_format::parse(const string_type&, double&) const [with CharType = char; size_t = long unsigned int; boost::locale::impl_icu::number_format::string_type = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:83:28: warning: ‘size_t boost::locale::impl_icu::number_format::parse(const string_type&, int64_t&) const [with CharType = char; size_t = long unsigned int; boost::locale::impl_icu::number_format::string_type = std::__cxx11::basic_string; int64_t = long int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/formatter.cpp:87:28: warning: ‘size_t boost::locale::impl_icu::number_format::parse(const string_type&, int32_t&) const [with CharType = char; size_t = long unsigned int; boost::locale::impl_icu::number_format::string_type = std::__cxx11::basic_string; int32_t = int]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/icu_backend.cpp:40:43: warning: ‘virtual boost::locale::impl_icu::icu_localization_backend* boost::locale::impl_icu::icu_localization_backend::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/icu_backend.cpp:45:14: warning: ‘virtual void boost::locale::impl_icu::icu_localization_backend::set_option(const string&, const string&)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/icu_backend.cpp:58:14: warning: ‘virtual void boost::locale::impl_icu::icu_localization_backend::clear_options()’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/icu_backend.cpp:89:29: warning: ‘virtual std::locale boost::locale::impl_icu::icu_localization_backend::install(const std::locale&, boost::locale::locale_category_type, boost::locale::character_facet_type)’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:130:23: warning: ‘boost::locale::impl_icu::num_format::iter_type boost::locale::impl_icu::num_format::do_put(boost::locale::impl_icu::num_format::iter_type, std::ios_base&, boost::locale::impl_icu::num_format::char_type, long int) const [with CharType = char; boost::locale::impl_icu::num_format::iter_type = std::ostreambuf_iterator >; boost::locale::impl_icu::num_format::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:134:23: warning: ‘boost::locale::impl_icu::num_format::iter_type boost::locale::impl_icu::num_format::do_put(boost::locale::impl_icu::num_format::iter_type, std::ios_base&, boost::locale::impl_icu::num_format::char_type, long unsigned int) const [with CharType = char; boost::locale::impl_icu::num_format::iter_type = std::ostreambuf_iterator >; boost::locale::impl_icu::num_format::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:138:23: warning: ‘boost::locale::impl_icu::num_format::iter_type boost::locale::impl_icu::num_format::do_put(boost::locale::impl_icu::num_format::iter_type, std::ios_base&, boost::locale::impl_icu::num_format::char_type, double) const [with CharType = char; boost::locale::impl_icu::num_format::iter_type = std::ostreambuf_iterator >; boost::locale::impl_icu::num_format::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:142:23: warning: ‘boost::locale::impl_icu::num_format::iter_type boost::locale::impl_icu::num_format::do_put(boost::locale::impl_icu::num_format::iter_type, std::ios_base&, boost::locale::impl_icu::num_format::char_type, long double) const [with CharType = char; boost::locale::impl_icu::num_format::iter_type = std::ostreambuf_iterator >; boost::locale::impl_icu::num_format::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:148:23: warning: ‘boost::locale::impl_icu::num_format::iter_type boost::locale::impl_icu::num_format::do_put(boost::locale::impl_icu::num_format::iter_type, std::ios_base&, boost::locale::impl_icu::num_format::char_type, long long int) const [with CharType = char; boost::locale::impl_icu::num_format::iter_type = std::ostreambuf_iterator >; boost::locale::impl_icu::num_format::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:152:23: warning: ‘boost::locale::impl_icu::num_format::iter_type boost::locale::impl_icu::num_format::do_put(boost::locale::impl_icu::num_format::iter_type, std::ios_base&, boost::locale::impl_icu::num_format::char_type, long long unsigned int) const [with CharType = char; boost::locale::impl_icu::num_format::iter_type = std::ostreambuf_iterator >; boost::locale::impl_icu::num_format::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:229:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, long int&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:234:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, short unsigned int&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:239:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, unsigned int&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:244:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, long unsigned int&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:249:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, float&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:254:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, double&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:259:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, long double&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:265:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, long long int&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/icu/numeric.cpp:270:23: warning: ‘boost::locale::impl_icu::num_parse::iter_type boost::locale::impl_icu::num_parse::do_get(boost::locale::impl_icu::num_parse::iter_type, boost::locale::impl_icu::num_parse::iter_type, std::ios_base&, std::ios_base::iostate&, long long unsigned int&) const [with CharType = char; boost::locale::impl_icu::num_parse::iter_type = std::istreambuf_iterator >; std::ios_base::iostate = std::_Ios_Iostate]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/codecvt.cpp:108:22: warning: ‘virtual bool boost::locale::impl_posix::mb2_iconv_converter::is_thread_safe() const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/codecvt.cpp:113:38: warning: ‘virtual boost::locale::impl_posix::mb2_iconv_converter* boost::locale::impl_posix::mb2_iconv_converter::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/codecvt.cpp:118:18: warning: ‘virtual uint32_t boost::locale::impl_posix::mb2_iconv_converter::to_unicode(const char*&, const char*)’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/codecvt.cpp:152:18: warning: ‘virtual uint32_t boost::locale::impl_posix::mb2_iconv_converter::from_unicode(uint32_t, char*, const char*)’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/codecvt.cpp:203:21: warning: ‘virtual int boost::locale::impl_posix::mb2_iconv_converter::max_len() const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/collate.cpp:69:17: warning: ‘int boost::locale::impl_posix::collator::do_compare(const char_type*, const char_type*, const char_type*, const char_type*) const [with CharType = char; boost::locale::impl_posix::collator::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/collate.cpp:69:17: warning: ‘int boost::locale::impl_posix::collator::do_compare(const char_type*, const char_type*, const char_type*, const char_type*) const [with CharType = wchar_t; boost::locale::impl_posix::collator::char_type = wchar_t]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/collate.cpp:80:18: warning: ‘long int boost::locale::impl_posix::collator::do_hash(const char_type*, const char_type*) const [with CharType = wchar_t; boost::locale::impl_posix::collator::char_type = wchar_t]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/collate.cpp:87:25: warning: ‘boost::locale::impl_posix::collator::string_type boost::locale::impl_posix::collator::do_transform(const char_type*, const char_type*) const [with CharType = char; boost::locale::impl_posix::collator::string_type = std::__cxx11::basic_string; boost::locale::impl_posix::collator::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/converter.cpp:107:25: warning: ‘virtual std::__cxx11::string boost::locale::impl_posix::utf8_converter::convert(boost::locale::converter_base::conversion_type, const char*, const char*, int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/converter.cpp:70:25: warning: ‘boost::locale::impl_posix::std_converter::string_type boost::locale::impl_posix::std_converter::convert(boost::locale::converter_base::conversion_type, const char_type*, const char_type*, int) const [with CharType = char; boost::locale::impl_posix::std_converter::string_type = std::__cxx11::basic_string; boost::locale::impl_posix::std_converter::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:164:23: warning: ‘boost::locale::impl_posix::time_put_posix::iter_type boost::locale::impl_posix::time_put_posix::do_put(boost::locale::impl_posix::time_put_posix::iter_type, std::ios_base&, CharType, const tm*, char, char) const [with CharType = char; boost::locale::impl_posix::time_put_posix::iter_type = std::ostreambuf_iterator >]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:288:10: warning: ‘virtual bool boost::locale::impl_posix::ctype_posix::do_is(std::ctype_base::mask, wchar_t) const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:310:20: warning: ‘virtual const wchar_t* boost::locale::impl_posix::ctype_posix::do_is(const wchar_t*, const wchar_t*, std::ctype_base::mask*) const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:340:20: warning: ‘virtual const wchar_t* boost::locale::impl_posix::ctype_posix::do_scan_is(std::ctype_base::mask, const wchar_t*, const wchar_t*) const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:347:20: warning: ‘virtual const wchar_t* boost::locale::impl_posix::ctype_posix::do_scan_not(std::ctype_base::mask, const wchar_t*, const wchar_t*) const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:430:22: warning: ‘CharType boost::locale::impl_posix::num_punct_posix::do_decimal_point() const [with CharType = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:434:22: warning: ‘CharType boost::locale::impl_posix::num_punct_posix::do_thousands_sep() const [with CharType = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:438:25: warning: ‘std::__cxx11::string boost::locale::impl_posix::num_punct_posix::do_grouping() const [with CharType = char; std::__cxx11::string = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:442:25: warning: ‘boost::locale::impl_posix::num_punct_posix::string_type boost::locale::impl_posix::num_punct_posix::do_truename() const [with CharType = char; boost::locale::impl_posix::num_punct_posix::string_type = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:447:25: warning: ‘boost::locale::impl_posix::num_punct_posix::string_type boost::locale::impl_posix::num_punct_posix::do_falsename() const [with CharType = char; boost::locale::impl_posix::num_punct_posix::string_type = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/numeric.cpp:55:23: warning: ‘boost::locale::impl_posix::num_format::iter_type boost::locale::impl_posix::num_format::do_format_currency(bool, boost::locale::impl_posix::num_format::iter_type, std::ios_base&, boost::locale::impl_posix::num_format::char_type, long double) const [with CharType = char; boost::locale::impl_posix::num_format::iter_type = std::ostreambuf_iterator >; boost::locale::impl_posix::num_format::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/posix_backend.cpp:107:29: warning: ‘virtual std::locale boost::locale::impl_posix::posix_localization_backend::install(const std::locale&, boost::locale::locale_category_type, boost::locale::character_facet_type)’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/posix_backend.cpp:44:45: warning: ‘virtual boost::locale::impl_posix::posix_localization_backend* boost::locale::impl_posix::posix_localization_backend::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/posix_backend.cpp:49:14: warning: ‘virtual void boost::locale::impl_posix::posix_localization_backend::set_option(const string&, const string&)’ can be marked override [-Wsuggest-override]
libs/locale/src/posix/posix_backend.cpp:60:14: warning: ‘virtual void boost::locale::impl_posix::posix_localization_backend::clear_options()’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/localization_backend.cpp:119:41: warning: ‘virtual boost::locale::localization_backend_manager::impl::actual_backend* boost::locale::localization_backend_manager::impl::actual_backend::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/localization_backend.cpp:123:30: warning: ‘virtual void boost::locale::localization_backend_manager::impl::actual_backend::set_option(const string&, const string&)’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/localization_backend.cpp:128:30: warning: ‘virtual void boost::locale::localization_backend_manager::impl::actual_backend::clear_options()’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/localization_backend.cpp:133:37: warning: ‘virtual std::locale boost::locale::localization_backend_manager::impl::actual_backend::install(const std::locale&, boost::locale::locale_category_type, boost::locale::character_facet_type)’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/message.cpp:503:42: warning: ‘const char_type* boost::locale::gnu_gettext::mo_message::get(int, const char_type*, const char_type*) const [with CharType = char; boost::locale::gnu_gettext::mo_message::char_type = char]’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/message.cpp:531:29: warning: ‘int boost::locale::gnu_gettext::mo_message::domain(const string&) const [with CharType = char; std::__cxx11::string = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/message.cpp:587:34: warning: ‘const char_type* boost::locale::gnu_gettext::mo_message::convert(const char_type*, boost::locale::gnu_gettext::mo_message::string_type&) const [with CharType = char; boost::locale::gnu_gettext::mo_message::char_type = char; boost::locale::gnu_gettext::mo_message::string_type = std::__cxx11::basic_string]’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:112:21: warning: ‘virtual int boost::locale::gnu_gettext::lambda::{anonymous}::div::operator()(int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:118:23: warning: ‘virtual boost::locale::gnu_gettext::lambda::{anonymous}::div* boost::locale::gnu_gettext::lambda::{anonymous}::div::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:177:21: warning: ‘virtual int boost::locale::gnu_gettext::lambda::{anonymous}::conditional::operator()(int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:181:30: warning: ‘virtual boost::locale::gnu_gettext::lambda::{anonymous}::conditional* boost::locale::gnu_gettext::lambda::{anonymous}::conditional::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:19:21: warning: ‘virtual int boost::locale::gnu_gettext::lambda::{anonymous}::identity::operator()(int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:23:27: warning: ‘virtual boost::locale::gnu_gettext::lambda::{anonymous}::identity* boost::locale::gnu_gettext::lambda::{anonymous}::identity::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:57:21: warning: ‘virtual int boost::locale::gnu_gettext::lambda::{anonymous}::number::operator()(int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:61:25: warning: ‘virtual boost::locale::gnu_gettext::lambda::{anonymous}::number* boost::locale::gnu_gettext::lambda::{anonymous}::number::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:75:21: warning: ‘virtual int boost::locale::gnu_gettext::lambda::{anonymous}::bin_not::operator()(int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:79:23: warning: ‘virtual boost::locale::gnu_gettext::lambda::{anonymous}::bin_not* boost::locale::gnu_gettext::lambda::{anonymous}::bin_not::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:94:21: warning: ‘virtual int boost::locale::gnu_gettext::lambda::{anonymous}::add::operator()(int) const’ can be marked override [-Wsuggest-override]
libs/locale/src/shared/mo_lambda.cpp:98:23: warning: ‘virtual boost::locale::gnu_gettext::lambda::{anonymous}::add* boost::locale::gnu_gettext::lambda::{anonymous}::add::clone() const’ can be marked override [-Wsuggest-override]
libs/locale/src/std/collate.cpp:27:17: warning: ‘virtual int boost::locale::impl_std::utf8_collator_from_wide::do_compare(const char*, const char*, const char*, const char*) const’ can be marked override [-Wsuggest-override]
libs/locale/src/std/collate.cpp:34:18: warning: ‘virtual long int boost::locale::impl_std::utf8_collator_from_wide::do_hash(const char*, const char*) const’ can be marked override [-Wsuggest-override]
libs/locale/src/std/collate.cpp:39:25: warning: ‘virtual std::__cxx11::string boost::locale::impl_std::utf8_collator_from_wide::do_transform(const char*, const char*) const’ can be marked override [-Wsuggest-override]
libs/locale/src/std/converter.cpp:43:25: warning: ‘boost::locale::impl_std::std_converter::string_type boost::locale::impl_std::std_converter::convert(boost::locale::converter_base::conversion_type, const char_type*, const char_type*, int) const [with CharType = char; boost::locale::impl_std::std_converter

Use clang-format

@artyom-beilis Would you mind if I used clang-format to format the sources and a CI job to enforce this? I'd do the same as in Boost.Nowide.

The changeset will be rather large, but the inconsistencies keep bugging me, like spaces or not around operators, trailing whitespace etc. So I'd rather do a one-time reformat and then have cleaner diffs later on.

std::auto_ptr is deprecated

While building 1.68.0 beta 1 on x86_64 linux with g++ 8.2.0, I get the following warnings and many more of the same king in boost::locale:

gcc.compile.c++ bin.v2/libs/locale/build/gcc-8.2.0/release/link-static/threading-multi/util/default_locale.o

    "g++-8.2.0"   -I/softs/lin64-gcc-8.2.0/release/iconv/include -I/softs/lin64-gcc-8.2.0/release/gettext/include -I/softs/lin64-gcc-8.2.0/release/bzip2/include -I/softs/lin64-gcc-8.2.0/release/zlib/inc>

In file included from libs/locale/src/util/default_locale.cpp:10:
./boost/locale/util.hpp:180:28: warning: « template<class> class std::auto_ptr » est obsolète [-Wdeprecated-declarations]
     BOOST_LOCALE_DECL std::auto_ptr<base_converter> create_utf8_converter();
                            ^~~~~~~~
In file included from /softs/gcc-8.2.0/include/c++/8.2.0/bits/locale_conv.h:41,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/locale:43,
                 from ./boost/locale/util.hpp:10,
                 from libs/locale/src/util/default_locale.cpp:10:
/softs/gcc-8.2.0/include/c++/8.2.0/bits/unique_ptr.h:53:28: note: déclaré ici
   template<typename> class auto_ptr;
                            ^~~~~~~~
In file included from libs/locale/src/util/default_locale.cpp:10:
./boost/locale/util.hpp:188:28: warning: « template<class> class std::auto_ptr » est obsolète [-Wdeprecated-declarations]
     BOOST_LOCALE_DECL std::auto_ptr<base_converter> create_simple_converter(std::string const &encoding);
                            ^~~~~~~~
In file included from /softs/gcc-8.2.0/include/c++/8.2.0/bits/locale_conv.h:41,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/locale:43,
                 from ./boost/locale/util.hpp:10,
                 from libs/locale/src/util/default_locale.cpp:10:
/softs/gcc-8.2.0/include/c++/8.2.0/bits/unique_ptr.h:53:28: note: déclaré ici
   template<typename> class auto_ptr;
                            ^~~~~~~~
In file included from libs/locale/src/util/default_locale.cpp:10:
./boost/locale/util.hpp:203:59: warning: « template<class> class std::auto_ptr » est obsolète [-Wdeprecated-declarations]
     std::locale create_codecvt(std::locale const &in,std::auto_ptr<base_converter> cvt,character_facet_type type);
                                                           ^~~~~~~~

std::unique_ptr should be used if possible.

Stream read/write with generated locales defaults to classic format/parse

I've just seen that the default flag for stream operations with locales generated by Boost.Locale is posix, i.e. "classic".

This leads to the bug/inconsistency I've encountered at #64 (comment)

Assume this:

std::locale::global(boost::locale::generator{}("de_DE"));
external_func(); // Some other library not aware of Boost.Locale

And then:

std::string external_func(){
  std::cout << "Enter 1 + " << 123.25 << std::endl;
  float num;
  std::cin >> num;
  std::ostringstream os; // Uses global-locale
  os << num;
  return os.str();
}

So although a German locale is globally set and one can reasonably expect that all formatting and parsing is done with the German decimal separator "," this is not the case: It will use the classic locale which outputs "123.25" and entering "124,25" will be parsed as "12425" if it doesn't fail.

I traced this to base_num_format/base_num_parse which access the ios_info.display_flags() of the stream which defaults to posix

So one would need to do outstream << as::number and instream >> as::number first for every stream created which external libraries are not aware of leading to those issues.

Hence I'd change the default from posix to number so results are more intuitive and external libraries unware of Boost.Locale work.

Although I'd consider this a bugfix, this is a breaking change, hence I wanted to make sure I didn't miss anything.

But the described behavior really does sound like a bug to me, see also #64 (comment): Imagine an external library unaware of Boost.Locale using the global locale and inspecting the numpunct facet and then find that formatting/parsing doesn't behave as expected.

boost::locale::conv::from_utf with charset Latin1

I write a little test program on windows :

`#include <boost/locale.hpp>
#include
#include
int main()
{
using namespace boost::locale;
using namespace std;

std::string my_string = "aiou";
try
{
    cout << "initial string " << my_string << endl;

    my_string = conv::from_utf(my_string, "Latin1");
    cout << "This is conversion from utf8 from latin " << my_string << endl;

	my_string = conv::to_utf<char>(my_string,"Latin1");
    cout << "This is conversion from to from latin " << my_string << endl;
}
catch(conv::conversion_error e)
{
	std::cout << "Error !" << std::endl;
}
catch (conv::invalid_charset_error e)
{
    std::cout << "invalid char set error" << std::endl;
    std::cout << e.what() << std::endl;
}

cout<<"end!"<<endl;

}`

I tried to run it with mingw and visual studio 2019 (with cmake). On mingw, I have no problems, it works well. On the other hand, with visual studio 2019, the program throw an exception : Invalid or unsupported charset:Latin1 (version of boost 1.75). it works if replace Latin1 by ISO-8859-1.

According to the documentation (https://www.boost.org/doc/libs/1_73_0/libs/locale/doc/html/charset_handling.html), it looks like a bug for me ... ?

GCC -Wimplicit-fallthrough warnings and couple of misspellings

I built Boost 1.72 with GCC 7.5 with added -Wextra in cxxflags. Problems are still in current code:

./boost/locale/utf.hpp:224:17: warning: this statement may fall through [-Wimplicit-fallthrough=]
./boost/locale/utf.hpp:231:17: warning: this statement may fall through [-Wimplicit-fallthrough=]
libs/locale/src/util/gregorian.cpp:206:27: warning: this statement may fall through [-Wimplicit-fallthrough=]
libs/locale/src/util/gregorian.cpp:505:21: warning: this statement may fall through [-Wimplicit-fallthrough=]
libs/locale/src/util/gregorian.cpp:529:21: warning: this statement may fall through [-Wimplicit-fallthrough=]
libs/locale/src/util/gregorian.cpp:725:28: warning: this statement may fall through [-Wimplicit-fallthrough=]

I may be mistaken, but looks Boost has BOOST_FALLTHROUGH for this purpose.

Related: #26

Misspellings:

libs/locale/src/util/gregorian.cpp:206 throght
libs/locale/src/util/gregorian.cpp:234 indecator

warning: this statement may fall through

This program:

#include <iostream>
#include <string>
#include <boost/locale/utf.hpp>
int main() {
  std::string s{"a"};
  auto begin = &s[0];
  auto end = begin + s.size();
  boost::locale::utf::code_point c =
      boost::locale::utf::utf_traits<char>::decode(begin, end);
  std::cout << "codepoint = " << c << '\n';
  return 0;
}

compiled on linux x86_64 with g++ 8.2.0 (same with 7.3.0) and boost 1.68.0 beta 1 with the following command:
g++-8.2.0 -o locale.o -c -Wall -Wextra -std=c++14 -I/softs/lin64-gcc-8.2.0/debug/boost/include locale.cpp
gives the following warnings:

In file included from locale.cpp:3:
/softs/lin64-gcc-8.2.0/debug/boost/include/boost/locale/utf.hpp: In static member function 'static boost::locale::utf::code_point boost::locale::utf::utf_traits<CharType, 1>::decode(Iterator&, Iterator) [with Iterator = char*; CharType = char]':
/softs/lin64-gcc-8.2.0/debug/boost/include/boost/locale/utf.hpp:224:17: warning: this statement may fall through [-Wimplicit-fallthrough=]
                 c = (c << 6) | ( tmp & 0x3F);
                 ^
/softs/lin64-gcc-8.2.0/debug/boost/include/boost/locale/utf.hpp:225:13: note: here
             case 2:
             ^~~~
/softs/lin64-gcc-8.2.0/debug/boost/include/boost/locale/utf.hpp:231:17: warning: this statement may fall through [-Wimplicit-fallthrough=]
                 c = (c << 6) | ( tmp & 0x3F);
                 ^
/softs/lin64-gcc-8.2.0/debug/boost/include/boost/locale/utf.hpp:232:13: note: here
             case 1:
             ^~~~

C++17 fixes not merged to master

The commit

322437a

that enables compilation on C++17 has not been applied to master. It's too late for the 1.66 beta, but perhaps it needs to be applied for the 1.66 release once master is open for changes?

Enable UTF char types (char16_t, char8_t etc.) when available

Currently there are defines like BOOST_LOCALE_ENABLE_CHAR16_T to enable char16_t and similar for others.

I find that a bit... difficult. We have BOOST_NO_CHAR16_T and similar to detect the availability automatically. So the defines are usually not required when that is used.

And the major obstacle: The library must be build with the same value (defined / not-defined) of those BOOST_LOCALE_ENABLE_CHAR*_T defines or the locale facets will have undefined IDs. But there is no way to know how the library was built.

Using the availability-test-macros one can assume, that if the library was built with a recent-ish compiler/standard then those types can be used.

So @artyom-beilis : Switch to using BOOST_NO_CHAR16_T and the like?

Add test for wconv_to_utf

That may have a bug. Add tests and verify.

Build fails on Windows

In https://github.com/boostorg/locale/blob/develop/src/win32/lcid.cpp

use of boost::unique_lock without including <boost/thread/lock_types.hpp> makes the compilation fail.

Visual Studio 2017
boost::locale latest (boost 1.67.0)

Default for iconv on Solaris?

@artyom-beilis What is the intended semantic behind

locale/build/Jamfile.v2

Lines 212 to 214 in f44b3bb

	if <boost.locale.iconv>on in $(properties)
	|| ! <boost.locale.iconv> in $(properties:G)
	&& ! <target-os>solaris in $(properties)

Should that mean either:

• Never use Iconv on Solaris ?
• Default to not using Iconv on Solaris ?

util.hpp does not respect BOOST_NO_RTTI

The typeid usage in the following line is not protected by the BOOST_NO_RTTI macro, and therefore generates a warning if the compiler has run-time type info disabled, even if the BOOST_NO_RTTI is defined correctly in boost/config/user.hpp.

locale/include/boost/locale/util.hpp

Line 122 in 76f6226

BOOST_ASSERT(typeid(*this)==typeid(base_converter));

Minimal program to reproduce: (using non-icu backend, not sure if that has an impact on this or not)

#include <boost/locale.hpp>

int main()
{
	auto result = boost::locale::conv::utf_to_utf<char>("input");
	return 0;
}

The warning message using the latest msvc compiler:

Warning	C4541	'typeid' used on polymorphic type 'boost::locale::util::base_converter' with /GR-; unpredictable behavior may result	include\boost\locale\util.hpp	122	

[Q] What does `time_zone::global` do exactly?

While trying to fix/improve unittests I came across the date_time class and the time_zone::global() function.

I can't really figure out what they are supposed to do.

@artyom-beilis Can you explain what the observable effect of the time_zone::global call below should be?

                time_zone::global("GMT");
                date_time tp_1;
                time_zone::global("GMT+01:00");
                date_time tp_2;

I see the default ctor uses the global TZ so I would expect that e.g. tp_1 != tp_2 but that doesn't seem to be the case. In fact I fail to find any influence of the global TZ on anything observable of the date_time instances.

ICU detection broken on Windows with mixed-/upper-case ICU_PATH

Hello,

I was trying to build Boost.Locale 1.81.0 w/ ICU support via Conan. For this the following flags are passed to b2:

b2 -q target-os=windows architecture=x86 address-model=64 binary-format=pe abi=ms --layout=system --user-config=C:\.conan\ab25f2\1\src\tools\build\user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=on boost.locale.iconv=off --disable-iconv runtime-link=shared runtime-debugging=on threading=multi visibility=hidden link=shared variant=release --with-atomic --with-chrono --with-container --with-date_time --with-exception --with-filesystem --with-iostreams --with-locale --with-log --with-program_options --with-random --with-regex --with-system --with-test --with-thread --with-url toolset=msvc cxxflags=/std:c++17 pch=on -sICU_PATH=C:\Users\vagrant\.conan\data\icu\72.1\_\_\package\37c0ff0745e1f400836948682306cb21048ba438 linkflags="" cxxflags="-fPIC" install --prefix=C:\.conan\4f301d\1 -j10 --abbreviate-paths -d0 --debug-configuration --build-dir="C:\.conan\2669f9\1\build-release"

With these flags, ICU isn't found during configuration and as such, ICU support is disabled. Changing ICU_PATH to be in lower case fixes the detection:

b2 -q target-os=windows architecture=x86 address-model=64 binary-format=pe abi=ms --layout=system --user-config=C:\.conan\ab25f2\1\src\tools\build\user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=on boost.locale.iconv=off --disable-iconv runtime-link=shared runtime-debugging=on threading=multi visibility=hidden link=shared variant=release --with-atomic --with-chrono --with-container --with-date_time --with-exception --with-filesystem --with-iostreams --with-locale --with-log --with-program_options --with-random --with-regex --with-system --with-test --with-thread --with-url toolset=msvc cxxflags=/std:c++17 pch=on -sICU_PATH=c:\users\vagrant\.conan\data\icu\72.1\_\_\package\37c0ff0745e1f400836948682306cb21048ba438 linkflags="" cxxflags="-fPIC" install --prefix=C:\.conan\4f301d\1 -j10 --abbreviate-paths -d0 --debug-configuration --build-dir="C:\.conan\2669f9\1\build-release"

The diff between these is only the ICU_PATH:

-C:\Users\vagrant\.conan\data\icu\72.1\_\_\package\37c0ff0745e1f400836948682306cb21048ba438
+c:\users\vagrant\.conan\data\icu\72.1\_\_\package\37c0ff0745e1f400836948682306cb21048ba438

One side-note which doesn't seem to affect all this but got me a bit off-track at first: In the logs of b2, Boost.Locale logs the following:

Search 64 bit ICU in /c:/users/vagrant/.conan/data/icu/72.1/_/_/package/0c952a6d19217f10d57daca31c441d7acd23430b/

The trailing slash here seems to be wrong on Windows (but I think that's just a faulty log message).

CodeCvt in locales don't work on Windows with multi-byte wide-chars

I see a failing test for test_ok<Char>("\xf0\xa0\x82\x8a",g("en_US.UTF-8")); // U+2008A which I traced down to Boost.Locale ultimately using std::codecvt<wchar_t, char, mbstate_t> which on VS 2019 uses _Mbrtowc under the hood which can only decode to a single wchar_t but the above would decode to a surrogate pair. Hence the decoding of that UTF-8 char fails which makes the read fail and the string read from the so-imbued file fails

Regex Fails when locale from boost on clang 6 and libc++

On FreeBSD 11.2 + 12.0 amd64 and boost 1.68.0 This fails: On all other tested platforms it matches ok. FreeBSD maintainer said, that it might be a bug in boost::locale.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233994

bug.txt

segfault on locale "hy_AM.ARMSCII-8" on FreeBSD

// build with
// clang --std=c++17 bug.cpp -O0 -g  -I /usr/local/include -lc++ -L/usr/local/lib/ -lboost_locale
// freebsd-version  11.2-RELEASE-p1
// clang FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)

#include <boost/locale.hpp>

int main(int argc, char** argv)
{
	// Set global localization backend
	using namespace boost::locale;
	localization_backend_manager my = localization_backend_manager::global();
	my.select("icu");
	localization_backend_manager::global(my);

	boost::locale::generator gen;
	std::locale loc{ gen("hy_AM.ARMSCII-8") };
	std::locale::global(loc);
}

This segfaults:

	Program received signal SIGABRT, Aborted.
0x00000008012c147a in thr_kill () from /lib/libc.so.7
(gdb) bt
#0  0x00000008012c147a in thr_kill () from /lib/libc.so.7
#1  0x00000008012c1444 in raise () from /lib/libc.so.7
#2  0x00000008012c13b9 in abort () from /lib/libc.so.7
#3  0x0000000800d5cb2b in boost::locale::impl_icu::icu_std_converter<char, 1>::uconv::uconv(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, boost::locale::impl_icu::cpcvt_type) () from /usr/local/lib/libboost_locale.so.1.68.0
#4  0x0000000800d5c914 in boost::locale::impl_icu::icu_std_converter<char, 1>::icu_std_converter(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, boost::locale::impl_icu::cpcvt_type) () from /usr/local/lib/libboost_locale.so.1.68.0
#5  0x0000000800d834cf in boost::locale::impl_icu::collate_impl<char>::collate_impl(boost::locale::impl_icu::cdata const&) ()
from /usr/local/lib/libboost_locale.so.1.68.0
#6  0x0000000800d83316 in boost::locale::impl_icu::create_collate(std::__1::locale const&, boost::locale::impl_icu::cdata const&, unsigned int) ()
from /usr/local/lib/libboost_locale.so.1.68.0
#7  0x0000000800d8e10f in boost::locale::impl_icu::icu_localization_backend::install(std::__1::locale const&, unsigned int, unsigned int) ()
from /usr/local/lib/libboost_locale.so.1.68.0
#8  0x0000000800d69245 in boost::locale::localization_backend_manager::impl::actual_backend::install(std::__1::locale const&, unsigned int, unsigned int)
	() from /usr/local/lib/libboost_locale.so.1.68.0
#9  0x0000000800d658d7 in boost::locale::generator::generate(std::__1::locale const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const () from /usr/local/lib/libboost_locale.so.1.68.0
#10 0x0000000800d6571d in boost::locale::generator::generate(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const () from /usr/local/lib/libboost_locale.so.1.68.0
#11 0x00000000004011c4 in boost::locale::generator::operator() (this=0x7fffffffe1f0, id=...) at /usr/local/include/boost/locale/generator.hpp:203
#12 0x0000000000401093 in main (argc=1, argv=0x7fffffffe2f8) at bug.cpp:17

utf8_codecvt fails when UTF-16 input ends with surrogate

This is a bug I detected in Boost.Nowide which uses the old code from the generic_codecvt used here. But the issue remains.

Situation:

• Pass UTF-16 string that ends in (or consists only of) U+D800 to do_out
• Not end of string -> loop continued.

  locale/include/boost/locale/generic_codecvt.hpp

  Line 355 in ccb8fbb

  while(to < to_end && from < from_end)

• Not in surrogate, so state is zero, so ch = 0xD800

  locale/include/boost/locale/generic_codecvt.hpp

  Line 363 in ccb8fbb

  if(state != 0) {

• First surrogate detected -> state = 0xD800, from increased -> now equal to from_end:

  locale/include/boost/locale/generic_codecvt.hpp

  Line 390 in ccb8fbb

  from++;

• Loop is exited

  locale/include/boost/locale/generic_codecvt.hpp

  Line 355 in ccb8fbb

  while(to < to_end && from < from_end)

• Result is still ok and from==from_end -> check fails

  locale/include/boost/locale/generic_codecvt.hpp

  Line 421 in ccb8fbb

  if(r==std::codecvt_base::ok && from!=from_end)

• ok is returned although partial should because "unexpected end of source buffer"

Solution should be to also check the state.

test_formatting failure, ICU >= 72.1

The test_formatting.cpp unit test fails for boost 1.82.0 and for ICU version greater than or equal to 72.1 due to a change in ICU time formatting. Date-time conversion string compares fail because ICU now uses a three byte UTF-8 narrow no-break space character where before an ordinary single byte space character was used.

Note that other projects have experience similar unit test failures do to this ICU change, for example: node.

Here is the failure output:

testing.capture-output bin.v2/libs/locale/test/test_formatting.test/clang-darwin-14/debug/threadapi-pthread/threading-multi/visibility-hidden/test_formatting.run
====== BEGIN OUTPUT ======
Testing char, UTF-8
Error in line 319: 
---- [3:33:13 PM] != [3:33:13 PM]
Error in line 320: 
---- [3:33 PM] != [3:33 PM]
Error in line 321: 
---- [3:33:13 PM] != [3:33:13 PM]
Error in line 322: 
---- [3:33:13 PM GMT] != [3:33:13 PM GMT]
Error in line 330: 
---- [3:33:13 PM Greenwich Mean Time] != [3:33:13 PM Greenwich Mean Time]
Error in line 334: 
---- [4:33:13 PM] != [4:33:13 PM]
Error in line 335: 
---- [4:33 PM] != [4:33 PM]
Error in line 341: 
---- [4:33:13 PM] != [4:33:13 PM]
Error in line 362: 
---- [4:33:13 PM GMT+1] != [4:33:13 PM GMT+1]
Error in line 369: 
---- [4:33:13 PM GMT+01:00] != [4:33:13 PM GMT+01:00]
Error in line 372: 
---- [Feb 5, 1970, 3:33:13 PM] != [Feb 5, 1970, 3:33:13 PM]
Error in line 379: 
---- [2/5/70, 3:33 PM] != [2/5/70, 3:33 PM]
Error in line 385: 
---- [Feb 5, 1970, 3:33:13 PM] != [Feb 5, 1970, 3:33:13 PM]
Error in line 391: 
---- [February 5, 1970 at 3:33:13 PM GMT] != [February 5, 1970 at 3:33:13 PM GMT]
Error in line 399: 
---- [Thursday, February 5, 1970 at 3:33:13 PM Greenwich Mean Time] != [Thursday, February 5, 1970 at 3:33:13 PM Greenwich Mean Time]
Test: %a
Test: %A
Test: %b
Test: %B
Test: %c
Error in line 439: ss.str() == to<CharType>(mark_result.second)
---- [Thursday, February 5, 1970 at 3:33:13 PM Greenwich Mean Time] != [Thursday, February 5, 1970 at 3:33:13 PM Greenwich Mean Time]
Test: %d
Test: %e
Test: %h
Test: %H
Test: %I
Test: %j
Test: %m
Test: %M
Test: %n
Test: %p
Test: %r
Test: %R
Test: %S
Test: %t
Test: %T
Test: %x
Test: %X
Error in line 439: ss.str() == to<CharType>(mark_result.second)
---- [3:33:13 PM] != [3:33:13 PM]
Test: %y
Test: %Y
Test: %Z
Test: %%
Test: 'Now is %A, %H o'clo''ck ' or not ' '
Test: ''test %H''
Test: '%H''
Test: ''%H''
Error in line 660: Format: {1,time,gmt}
---- [3:33:13 PM] != [3:33:13 PM]
Error in line 661: Format: {1,datetime,gmt}
---- [Feb 5, 1970, 3:33:13 PM] != [Feb 5, 1970, 3:33:13 PM]
Error in line 662: Format: {1,dt,gmt}
---- [Feb 5, 1970, 3:33:13 PM] != [Feb 5, 1970, 3:33:13 PM]
Error in line 664: Format: {1,time=short,gmt}
---- [3:33 PM] != [3:33 PM]
Error in line 665: Format: {1,time=s,gmt}
---- [3:33 PM] != [3:33 PM]
Failed Error limits reached, stopping unit test

EXIT STATUS: 1
====== END OUTPUT ======

Here is script that you can run on MacOS that will download ICU 73.1 and Boost 1.82.0, build and test ICU, build boost, and run the locale tests. The boost locale tests are expected to fail, as shown above.

#! /bin/zsh -x

# Download icu and boost to boost_local_test.d directory. Build icu
# and run tests. Build boost --with-locale and run locale tests.
#
# Expect results: PASS for ICU <= 71.1, FAIL for ICU >= 72.1
#
# The test failure is due to the introduction of the narrow
# non-breaking space characters in date/time conversions in icu 72.1.
#
# Tested with: MacOS 13.4, XCode 14.3.1
#
# Log file is written to: boost_locale_test.log

set -e

# setup error trap
function ErrorTrap {
    echo FAILED
}
trap ErrorTrap INT TERM EXIT

mkdir boost_locale_test.d
cd    boost_locale_test.d

# setup log
exec > >(tee boost_locale_test.log)
exec 2> >(tee -a boost_locale_test.log)

ICU_VER="73.1"
ICU_RELEASE=release-$(echo $ICU_VER | sed s/\\./-/)
ICU4C_SRC=icu4c-$(echo $ICU_VER | sed s/\\./_/)-src

BOOST_VER="1.82.0"
BOOST_SRC=boost_$(echo $BOOST_VER | sed s/\\./_/g)

JOBS=$(sysctl -n hw.ncpu)
INSTDIR=$(pwd)/install

# get icu
wget -q "https://github.com/unicode-org/icu/releases/download/${ICU_RELEASE}/${ICU4C_SRC}.tgz"
tar xzf "${ICU4C_SRC}".tgz
mv icu "${ICU4C_SRC}"

# get boost
wget -q "https://boostorg.jfrog.io/artifactory/main/release/"${BOOST_VER}"/source/"${BOOST_SRC}".tar.gz"
tar xzf "${BOOST_SRC}".tar.gz

# build icu
(
  cd "${ICU4C_SRC}"/source
  CXXFLAGS=-std=c++11 ./runConfigureICU MacOSX --enable-static --prefix="${INSTDIR}"
  for target in check install
  do
      gnumake -j${JOBS} CXXFLAGS=-std=c++11 $target
  done
)

# build boost
(
  cd "${BOOST_SRC}"
  ./bootstrap.sh --prefix="${INSTDIR}"
  for target in stage libs/locale/test
  do
      ./b2  -j${JOBS} -sICU_PATH="${INSTDIR}" boost.locale.icu=on boost.locale.iconv=off boost.locale.winapi=off boost.locale.std=off boost.locale.posix=off --with-locale cxxflags=-std=c++11 toolset=clang $target
  done
)

# clear error trap
trap - INT TERM EXIT

echo PASSED

BOOST_LOCALE_HAVE_WCSFTIME_L unused

The define at

locale/src/posix/numeric.cpp

Line 35 in b889b35

#define BOOST_LOCALE_HAVE_WCSFTIME_L

is not used.

Instead the non-prefixed one is used at

locale/src/posix/numeric.cpp

Line 125 in b889b35

#ifdef HAVE_WCSFTIME_L

@artyom-beilis Do you know which test should detect this problem?

Not supporting en_001, en_150 or en_US_POSIX locale

I'm starting to use Boost.Locale in a project and I've hit up against a problem when the system locale is en_001. And I believe it will also be a problem with en_150 and en_US_POSIX as well.
The problem is best described with the following test case:

#include <boost/locale.hpp>
#include <iostream>

int main(int argc, char** argv) {
  boost::locale::generator gen;
  std::locale loc(gen(""));
  std::locale::global(loc);
  std::cout.imbue(loc);

  std::cout << "LOCALE NAME: " << std::use_facet<boost::locale::info>(loc).name() << std::endl;
  std::cout << "LOCALE LANG: " << std::use_facet<boost::locale::info>(loc).language() << std::endl;
  std::cout << "LOCALE COUNTRY: " << std::use_facet<boost::locale::info>(loc).country()
            << std::endl;
  std::cout << "LOCALE ENCODING: " << std::use_facet<boost::locale::info>(loc).encoding()
            << std::endl;
  std::cout << "LOCALE UTF8: " << std::use_facet<boost::locale::info>(loc).utf8() << std::endl;

  return 0;
}

If the system is in en_001, which on Windows is the "English (World)" region name, then the following will be output:

LOCALE NAME: en_001.UTF-8
LOCALE LANG: en
LOCALE COUNTRY:
LOCALE ENCODING: us-ascii
LOCALE UTF8: 0

I would expect the output to be:

LOCALE NAME: en_001.UTF-8
LOCALE LANG: en
LOCALE COUNTRY: 001
LOCALE ENCODING: utf-8
LOCALE UTF8: 1

It's coming from the fact that in boost::locale::util::locale_data::parse_from_country, we are assuming the country needs to contain only 'a' to 'z' or 'A' to 'Z'. But en_001 (and en_150) are valid locales. Probably en_US_POSIX should be handled separately as it's special.

This code will fail on FreeBSD

std::string Utils::make_string(const std::string &nickname, const std::string& fmt, ...) {
generator gen;
locale loc;
gen.add_messages_path("lang");
gen.add_messages_domain("zeus");

if (nickname != "")
	loc = gen(NickServ::GetLang(nickname));
else if (!config->Getvalue("language").empty())
	loc = gen(config->Getvalue("language"));
else
	loc = gen("en");

std::string msg = translate(fmt).str(loc);

char buffer[512];
va_list args;
va_start (args, fmt);
vsnprintf (buffer, 512, msg.c_str(), args);
va_end (args);
return buffer;

}

Non-standard environment variable priority

Current Boost.Locale code uses, in this order, environment variables LC_CTYPE, LC_ALL and LANG to detect the default system locale. This is different from the standard behaviour, which is to use, in the order, LC_ALL, LC_* and LANG (see, for instance, the POSIX reference).

I think it would be great for Boost to adhere to the standard behaviour.

This bug was first reported to the Debian bug tracking system: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=893884

global locale from boost make std::regex fail on FreeBSD with libc++

This is a dependent of
#35
but with more concentrated and more distilled information.

bug.txt
result-csv.ods.tar.gz

The regex fails if one of this facets is set:
"all_characters , collation_facet, all_categories".

boost::locale::generator gen;

// all_characters , collation_facet, all_categories trigger the bug",
gen.categories(collation_facet);
std::locale loc{ gen("de_DE.UTF-8") };
std::locale::global(loc);

std::regex re_datetime("^Date\\s*:[\\s\\t]*(\\d{4}\\-\\d{2}\\-\\d{2} "
						"\\d{2}:\\d{2}:\\d{2}?)$",
						std::regex_constants::icase);
std::smatch match;
if (std::regex_match(test, match, re_datetime) && match.size() > 1)
	std::cout << "All ok" << std::endl;
else
	std::cout << "Bug triggered" << std::endl;

I setup a testprogram that tries every locale (192),every facet(12(no wchar_t on freebsd)), every backend(std/icu/posix) and icase on/off.

The pivot analysis of the result shows, this happens on every locale (except posix/C) with facet "all_characters , collation_facet, all_categories".

This happens only on FreeBSD with clang and libc++.

Boost\libs\locale\test failed to build during "Testing codepoints above 0xFFFF", error occurred on MSVC

Issue Description:
Boost\libs\locale\test failed to build during "Testing codepoints above 0xFFFF", error occurred on MSVC . Could you please take a look? The master branch commit we used is a1552ab.

Build Steps:
1. git clone -c core.autocrlf=true --recursive ​ https://github.com/boostorg/boost.git boost
2. open a VS 2019 x86 command prompt and browse to boost
3. .\bootstrap
4. .\b2 headers variant=release --build-dir=..\out\x86rel address-model=32
5. .\b2 variant=release --build-dir=..\out\x86rel address-model=32
6. .\b2 -j16 variant=release --build-dir=..\out\x86rel libs\locale\test

Error Info:
testing.capture-output ..\out\x86rel\boost\bin.v2\libs\locale\test\test_std_collate.test\msvc-14.2\release\threading-multi\test_std_collate.run
1 file(s) copied.
testing.capture-output ..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.run
====== BEGIN OUTPUT ======

• Testing correct invalid bytes skipping
  -- ISO-8859-8
  -- cp932
  Warning: detected unproper support of ja_JP.SJIS locale, disableling it
  Testing for backend std
  Testing wide I/O
  wchar_t
  UTF-8
  Testing codepoints above 0xFFFF
  Single U+2008A
  Error in line:67 read_file(f1) == cmp
  Error in line:76 read_file(testo) == file
  Single U+2008A withing text
  Error in line:67 read_file(f1) == cmp
  Error in line:76 read_file(testo) == file
  U+2008A x 1000
  Error in line:67 read_file(f1) == cmp
  Error in line:76 read_file(testo) == file
  ISO8859-8
  ISO8859-1
  Testing charset to/from UTF conversion functions
  char
  wchar_t
  Testing utf_to_utf
  char<-char
  char<-wchar
  wchar<-char
  wchar<-wchar
  Testing for backend winapi
  Testing wide I/O
  wchar_t
  UTF-8
  Testing codepoints above 0xFFFF
  Single U+2008A
  Single U+2008A withing text
  U+2008A x 1000
  Testing charset to/from UTF conversion functions
  char
  wchar_t
  Testing utf_to_utf
  char<-char
  char<-wchar
  wchar<-char
  wchar<-wchar

Passed 366 tests
Failed 6 tests
98.4% of tests completed sucsessefully

EXIT STATUS: 1
====== END OUTPUT ======

set Path=F:\gitP\boostorg\out\x86rel\boost\bin.v2\libs\locale\build\msvc-14.2\release\threading-multi;F:\gitP\boostorg\out\x86rel\boost\bin.v2\libs\system\build\msvc-14.2\release\threading-multi;%Path%

set status=0
if %status% NEQ 0 (
    echo Skipping test execution due to testing.execute=off
    exit 0
)
 "..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.exe"   > "..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.output" 2>&1 
set status=%ERRORLEVEL%
echo. >> "..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.output"
echo EXIT STATUS: %status% >> "..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.output"
if %status% EQU 0 (
    copy "..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.output" "..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.run"
)
set verbose=0
if %status% NEQ 0 (
    set verbose=1
)
if %verbose% EQU 1 (
    echo ====== BEGIN OUTPUT ======
    type "..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.output"
    echo ====== END OUTPUT ======
)
exit %status%

...failed testing.capture-output ..\out\x86rel\boost\bin.v2\libs\locale\test\test_codepage.test\msvc-14.2\release\threading-multi\test_codepage.run...

See more details in log:
test.log.txt

[Q] Meaning of utf8_from_wide/utf8_native_with_wide

Due to a recent issue related to the different handling of facet creation depending on utf8_native_with_wide etc. (now the enum class utf8_support) I wanted to ask for clarification:

utf8_native seems to be unused, it is checked for but never set, so it can be removed, can't it?

What is the intended difference between utf8_native_with_wide and utf8_from_wide?

• Both are the same for converter: https://github.com/boostorg/locale/blob/boost-1.48.0/src/std/converter.cpp#LL113C27-L113C73
• For "numeric" they seem to be almost the same: https://github.com/boostorg/locale/blob/boost-1.48.0/src/std/numeric.cpp#L320-L327 vs https://github.com/boostorg/locale/blob/boost-1.48.0/src/std/numeric.cpp#L338-L345
• In numeric.cpp from_wide uses utf8_time_put_from_wide instead of time_put_from_base where the latter looks like it could have been std::time_put_byname similar to many other facets
• In codecvt.cpp from_wide creates the utf8_codecvt: https://github.com/boostorg/locale/blob/boost-1.48.0/src/std/codecvt.cpp#L30-L32 / https://github.com/boostorg/locale/blame/boost-1.60.0/src/std/codecvt.cpp#L31
• In collator.cpp from_wide creates the utf8_collator_from_wide: https://github.com/boostorg/locale/blob/boost-1.48.0/src/std/collate.cpp#L77-L81

Current logic seems to be that on Windows utf8_from_wide is used otherwise utf8_native_with_wide is used and when the requested encoding isn't UTF-8 then utf8_none is used on all platforms.

However I'm confused that in numeric.cpp the utf8_*_from_wide classes are used (except for time_put) while the collator uses the *from_wide variant only on Windows.

To me it looks like either the utf8_*_from_wide classes should always be used (which might be a performance issue due to the required 2 conversions) or the standard classes are enough already.

So questions (assuming an UTF-8 locale is requested):

• Any reason not to always use the utf8_codecvt given that it should work for UTF-16 and UTF-32 wchar_ts?
• In which case would time_put_from_base/std::time_put_byname fail, that utf8_time_put_from_wide avoids?
• Same for utf8_collator_from_wide vs std::collate_byname

And (generally): Can time_put_from_base be replaced by std::time_put_byname?

Only possible reasoning I can see is:
std::locale("foo.UTF-8") fails but std::locale("foo") or std::locale("Windows-name-of-foo") works, i.e. the standard library does not support the UTF-8 encoding and that has to be emulated.

Is this correct? In that case we would need utf8_support::none (non-UTF-8 locale requested), utf8_support::native and utf8_support::from_wide for when std::locale("foo.UTF-8") works and when it doesn't respectively

install_all/install_by does "nothing"

@artyom-beilis What exactly are the install_all/install_by structs supposed to do?

They call has_facet on the classic locale at e.g.

locale/src/shared/ids.cpp

Line 71 in 9259845

std::has_facet<info>(l);

However as it ignores the return value the compiler rightfully warns on such lines as they do nothing.

What is the intended purpose of those has_facet calls? I'd like to at least rename them to reflect what they do. And "install" is not the right word as they do effectively nothing.

Android NDK - build fail

lib/boost/libs/locale/src/encoding/codepage.cpp:43:30: error: use of undeclared identifier 'converter_between'; did you mean 'convert_between'?
                    hold_ptr<converter_between> cvt;
                             ^~~~~~~~~~~~~~~~~
                             convert_between
lib/boost/libs/locale/src/encoding/codepage.cpp:37:29: note: 'convert_between' declared here
                std::string convert_between(char const *begin,
                            ^
lib/boost/libs/locale/src/encoding/codepage.cpp:43:30: error: template argument for template type parameter must be a type
                    hold_ptr<converter_between> cvt;
                             ^~~~~~~~~~~~~~~~~
lib/boost/libs/locale/include\boost/locale/hold_ptr.hpp:17:23: note: template parameter is declared here
    template<typename T>
                      ^
lib/boost/libs/locale/src/encoding/codepage.cpp:69:30: error: use of undeclared identifier 'converter_to_utf'
                    hold_ptr<converter_to_utf<CharType> > cvt;
                             ^
lib/boost/libs/locale/src/encoding/codepage.cpp:95:30: error: use of undeclared identifier 'converter_from_utf'; did you mean 'convert_from'?
                    hold_ptr<converter_from_utf<CharType> > cvt;
                             ^~~~~~~~~~~~~~~~~~
                             convert_from
lib/boost/libs/locale/src/encoding/codepage.cpp:89:29: note: 'convert_from' declared here
                std::string convert_from(
                            ^
lib/boost/libs/locale/src/encoding/codepage.cpp:95:30: error: template argument for template type parameter must be a type
                    hold_ptr<converter_from_utf<CharType> > cvt;
                             ^~~~~~~~~~~~~~~~~~
lib/boost/libs/locale/include\boost/locale/hold_ptr.hpp:17:23: note: template parameter is declared here
    template<typename T>
                      ^
5 errors generated.

ISO-2022-JP as well as some other charsets not supported on Windows

According to MultiByteToWideChar's docs the flags parameter has to be 0 for some code pages:

For the code pages listed below, dwFlags must be set to 0. Otherwise, the function fails with ERROR_INVALID_FLAGS.

50220
50221
50222
50225
50227
50229
57002 through 57011
65000 (UTF-7)
42 (Symbol)

Because of this the following code snippet returns an empty string instead of the expected 冬季:

    std::string res = boost::locale::conv::to_utf<char>("\xe5\x86\xac\xe5\xad\xa3", "iso-2022-jp");

My suggestion for a fix would be to check whether the code page we're dealing with is from this short-list and not add the flag if it is. This should be done in all calls to MultiByteToWideChar like here: wconv_codepage.ipp#L100.

Inconsistent behavior - boost::locale defines `numpunct` for non ICU backends

#include <boost/locale.hpp>
#include <iostream>
int main() {
  boost::generator gen;
  auto &facet = std::use_facet<std::numpunct<char>>(gen("it_IT.UTF-8"));
  std::cout << facet.thousand_separator() << ' ' << facet.decimal_point() << '\n';
}

This code always outputs ", ." regardless of locale.
Right now the only way to extract std::numpunct information is to format an arbitrary number to a string and extract its, say, second and sixth character, and that's not a very efficient implementation.
I would suggest creating a std::numpunct subclass in icu/numeric.cpp that extracts that information from the icu::DecimalFormatSymbols class using getSymbol

Can't statically link Boost Locale

Trying to statically link Boost Locale with cmake returns an error.

CMakeLists.txt

cmake_minimum_required(VERSION 3.16)
project(main)

#set(Boost_USE_STATIC_LIBS ON)
find_package(Boost COMPONENTS locale REQUIRED)
if(NOT Boost_FOUND)
    message(FATAL_ERROR "Some Boost libraries missing in the system")
endif()


add_executable(main main.cpp)
target_link_libraries(main ${Boost_LIBRARIES})

main.cpp

#include <iostream>
#include <locale>
#include <boost/locale.hpp>

int main() {
    boost::locale::generator gen;
    std::locale loc1 = gen("pt_BR.UTF-8");
    return 0;
}

Dynamic linkage works fine but uncommenting set(Boost_USE_STATIC_LIBS ON) will result in an error.
The boost library was installed via package manager with sudo apt install libboost-all-dev.

As we can see in the prints I have Locale installed and also the icu stuff.

ldd of the binary if not statically linked.

I can statically link other boost libraries, but locale in specific gives me an error.

Output of the make command: log.txt

Ubuntu version: 20.04 and 22.04
Cmake version: >= 3.16
Boost version: 1.71 and 1.74
g++ version: 10.3 and 11.3

Require C++11 in Boost 1.81

It might be worth dropping support for C++98. In some PRs move-support was requested and some code places may benefit from defaulted functions.
So C++11 features I'd like to use:

• Rvalue references (move-assign/construct where useful)
• std::unique_ptr, dropping std::auto_ptr which removes a ton of warnings
• defaulted functions

Nothing really big but keeping support for C++98 makes the code harder to write and read due to ifdef-macros etc.
Additionally ICU now requires C++11 as they use char16_t etc. in some of their headers. So building Boost.Locale against such an ICU version requires C++11 anyway.

undefined reference to `boost::locale::ios_info::get(std::ios_base&)'

I'm having an issue getting my code to compile using boost::locale. I have the library built using flatpak:

{
	    "name": "boost",
	    "buildsystem": "simple",
	    "build-commands": [
		"./bootstrap.sh --prefix=/app --with-libraries='system multiprecision locale'",
		"./b2 -j $FLATPAK_BUILDER_N_JOBS install"
	    ],
	    "sources": [
		{
		    "type": "archive",
		    "url": "https://boostorg.jfrog.io/artifactory/main/release/1.80.0/source/boost_1_80_0.tar.bz2",
		    "sha256": "1e19565d82e43bc59209a168f5ac899d3ba471d55c7610c677d4ccf2c9c500c0"
		}
	    ]
},

and am using the library as such:

#include "moneyhelpers.hpp"
#include <sstream>
#include <boost/locale.hpp>

std::string MoneyHelpers::boostMoneyToLocaleString(boost::multiprecision::cpp_dec_float_50 amount, const std::locale& locale)
{
    std::stringstream builder;
    builder.imbue(locale);
    builder << boost::locale::as::currency << amount;
    return builder.str();
}

However, when I try to build and run the application, I'm getting this undefined reference error:

moneyhelpers.cpp:(.text+0x23d): undefined reference to `boost::locale::ios_info::get(std::ios_base&)'
/usr/lib/gcc/x86_64-unknown-linux-gnu/12.1.0/../../../../x86_64-unknown-linux-gnu/bin/ld: moneyhelpers.cpp:(.text+0x24a): undefined reference to `boost::locale::ios_info::display_flags(unsigned long)'

Any help would be greatly appreciated as I cannot find any information on boost::locale::ios_info

BOOST_LOCALE_SOURCE not defined in all source files

All source files except shared/mo_lambda.cpp and icu/formatters_cache.cpp define BOOST_LOCALE_SOURCE.

Add replace method to utf-to-utf

During conversion between UTF encodings it might be desirable to replace invalid characters by a replacement character, e.g. 0xFFFD. But the current methods only allow to skip or abort.

ICU_LINK option does not work anymore

ICU_LINK option does not work now, because boost.regex errors build if sees it now.
See boostorg/regex@fc4dc17 and boostorg/regex#89

I would suggest moving ICU finding/configuration into shared file and using it from there like its done for zlib, bzip2, etc, and not doing the same thing in different ways.

@jzmaddock

boost::locale::boundary::segment_index throws bad_cast

I am trying to execute the code below. It is a code sample from boost documentation that perfoms word extraction from a text. Unfortunately it crashes with a bad_cast thrown.

using namespace boost::locale::boundary;
boost::locale::generator gen;
std::string text = "To be or not to be, that is the question.";
ssegment_index map(word, text.begin(), text.end(), gen("en_US.UTF-8"));
for (ssegment_index::iterator it = map.begin(), e = map.end(); it != e; ++it)
    std::cout << "\"" << *it << "\", ";
std::cout << std::endl;

The exception is thrown when trying to construct segment_index, I do not understand why.

Here is the call stack:

KernelBase.dll!00007ffc391ca839()   Unknown
vcruntime140d.dll!00007ffc13dfb230()    Unknown
Test.exe!std::_Throw_bad_cast() Line 66 C++
Test.exe!std::use_facet<boost::locale::boundary::boundary_indexing<char>>(const std::locale & _Loc) Line 439    C++
Test.exe!boost::locale::boundary::details::mapping_traits<std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>>,std::random_access_iterator_tag>::map(boost::locale::boundary::boundary_type t, std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>> b, std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>> e, const std::locale & l) Line 98  C++
Test.exe!boost::locale::boundary::details::mapping<std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>>>::mapping<std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>>>(boost::locale::boundary::boundary_type type, std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>> begin, std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>> end, const std::locale & loc) Line 127    C++
Test.exe!boost::locale::boundary::segment_index<std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>>>::segment_index<std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>>>(boost::locale::boundary::boundary_type type, std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>> begin, std::_String_const_iterator<std::_String_val<std::_Simple_types<char>>> end, const std::locale & loc) Line 630 C++
Test.exe!main() Line 12 C++

I am using Visual Studio 2019 with boost 1.73 installed using vcpkg (vcpkg.exe install boost-locale[icu]:x64-windows).

Thanks in advance for your help.

Confused with the expected behavior regarding the capitalization of “ß”

This example (https://www.boost.org/doc/libs/1_71_0/libs/locale/doc/html/conversions.html) indicates that the correct case for the word "grüßen" is as follow:

Upper GRÜSSEN
Lower grüßen
Title Grüßen
Fold  grüssen

However, the wording used in the example examples/conversions.cpp seems to indicate that replacing ß by SS is incorrect when using uppercase :

cout<<"For example:"<<endl;
cout<<"   German grüßen correctly converted to "<<to_upper("grüßen")<<", instead of incorrect "
                <<boost::to_upper_copy(std::string("grüßen"))<<endl;
cout<<"     where ß is replaced with SS"<<endl;

I tried running this example (note that I used Boost 1.71), and this is the result I get:

For example:
   German grüßen correctly converted to GRÜßEN, instead of incorrect GRüßEN
     where ß is replaced with SS

...and now I'm even more confused. What is the actual expected behavior for this case?

Encoding with utf_traits can be wrong with C++11 string types

The utf_traits class uses the size of the char to switch between UTF-8, UTF-16, UTF32, e.g. for sizeof(CHAR)==2 it uses UTF-16: https://github.com/boostorg/locale/blob/develop/include/boost/locale/utf.hpp#L312

C++11 introduced standardized strings with encoding via char16_t, char32_t (and later char8_t). Those are "at least" 2 or 4 bytes respectively wide. So using utf_traits<char16_t> might fail if that happens to be 4 bytes wide on some platform.

Proposed solution: Templatize utf_traits over an encoding instead. This can be a plain-old-enum where values UTF8, UTF16, UTF32 are set to 1, 2, 4 to allow this to be backwards compatible. Then dispatch (via an indirection if required) the charN_t types to their known encoding. Not sure if with the current impl one can directly make utf_traits<char16_t, N>==utf_traits<char16_t, 2> so indirection might be required.

Std backend falls back to system locale instead of C locale on Windows

The fallback in the std backend on Windows uses to_windows_name at

locale/src/boost/locale/std/std_backend.cpp

Lines 109 to 111 in 1dfe26c

	const std::pair<std::string, std::string> wl_inf = to_windows_name(lid);
	const std::string& win_name = wl_inf.first;
	const auto& win_codepage = wl_inf.second;

That is supposed to get a locale name based on the Windows LCID by calling boost::locale::impl_win::locale_to_lcid.

However the function does not check for a zero lcid returned by locale_to_lcid in case the locale is not found and continues which GetLocaleInfoA treats as getting the system default, see https://learn.microsoft.com/en-us/windows/win32/intl/locale-neutral

Instead, use either LOCALE_SYSTEM_DEFAULT or LOCALE_USER_DEFAULT. The value of LOCALE_NEUTRAL is 0x0000.

Judging by the initialization at

locale/src/boost/locale/std/std_backend.cpp

Line 32 in 1dfe26c

std::pair<std::string, std::string> res("C", "0");

I assume that the "C" locale should be used instead.

Is that the correct intention? @artyom-beilis

Additionally: IMO it is at least surprising to silently fall back to the "C" locale when the correct locale couldn't be generated without any chance to notice this (the boost::locale::info facet will still report the requested locale) especially combined with the WinAPI backend not supporting non-UTF-8 locales and always falling back to the "C" locale with UTF-8 encoding when any language with a non-UTF-8 encoding is requested.

I would rather throw an exception in that case: The generator couldn't do as requested -> No good way to continue.

Any objections?

Boost 1.76 and 1.77 (beta) don't create static libraries on macOS with build-type=complete

Boost 1.76 and 1.77 Boost.Locale doesn't create the static libraries (s, sd flavors) with build-type=complete on macOS (Mojave, Bug Sur, both arm & intel). I'm using boost.locale.icu=off boost.locale.iconv=on parameters and you can see the complete b2 command below.

On Boost 1.75 it works and creates the following libraries:

libboost_locale-mt-a64.a
libboost_locale-mt-a64.dylib
libboost_locale-mt-d-a64.a
libboost_locale-mt-d-a64.dylib
libboost_locale-mt-s-a64.a
libboost_locale-mt-sd-a64.a

On Boost 1.76 and 1.77 does not create the static versions (sd & s flavors), but just the dynamic versions:

libboost_locale-mt-a64.a
libboost_locale-mt-a64.dylib
libboost_locale-mt-d-a64.a
libboost_locale-mt-d-a64.dylib

The complete b2 command used is the following:

./b2 --prefix=./bld --stagedir=./bld/stage --build-dir=./bld/int -j16 --build-type=complete --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxflags="-stdlib=libc++ -std=c++11 -fPIC" linkflags="-stdlib=libc++ -std=c++11 -fPIC" --without-python --without-iostreams boost.locale.icu=off boost.locale.iconv=on install > ./build.log

Boost.Locale don't build on macOS (missing iconv)

During build Boost.Locale, library iconv (which comes with Xcode) is not detected, so build Boost.Locale is skipped. This appeared in version 1.76 - in version 1.75 the build is successful.

Performing configuration checks
- default address-model    : 64-bit [1]
- default architecture     : x86 [1]
- has stat::st_mtim        : no [2]
- has stat::st_mtimensec   : no [2]
- has stat::st_mtimespec   : yes [2]
- has stat::st_birthtim    : no [2]
- has stat::st_birthtimensec : no [2]
- has stat::st_birthtimespec : yes [2]
- has statx                : no [2]
- has statx syscall        : no [2]
- has_icu builds           : no [2]
- iconv (libc)             : no [2]
- iconv (separate)         : no [2] <--this
- icu                      : no [2]
- icu (lib64)              : no [2]
- Boost.Locale needs either iconv or ICU library to be built.
- iconv (libc)             : no [3]
- iconv (separate)         : no [3] <--this
- icu                      : no [3]
- icu (lib64)              : no [3]
- Boost.Locale needs either iconv or ICU library to be built.
- lockfree boost::atomic_flag : no [2]
- compiler supports SSE2   : yes [2]
- compiler supports SSE4.1 : yes [2]

I also tried to install iconv separately with brew and pointed the path to it through -sICONV_PATH. But this did not give any results.

iMac2019
macOS 11.4 (20F71)
Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.5.0

Loss of data (C4244) warnings in MSVC 64-bit build

Building with MSVC compilers for 64-bit results in several C4244 warnings in api.hpp, due to passing the difference of two pointers (ptrdiff_t) to Win APIs expecting an int. The attached patch adds casts to silence the warnings.

api.hpp.diff.txt