Giter Site home page Giter Site logo

filesystem's Introduction

Boost.Filesystem

Boost.Filesystem, part of collection of the Boost C++ Libraries, provides facilities to manipulate files and directories, and the paths that identify them.

Directories

  • doc - Documentation sources
  • include - Interface headers of Boost.Filesystem
  • src - Compilable source files of Boost.Filesystem
  • test - Boost.Filesystem unit tests
  • example - Boost.Filesystem usage examples

More information

Build status

Branch GitHub Actions AppVeyor Test Matrix Dependencies
master GitHub Actions AppVeyor Tests Dependencies
develop GitHub Actions AppVeyor Tests Dependencies

License

Distributed under the Boost Software License, Version 1.0.

filesystem's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

filesystem's Issues

`undefined reference` in boost filesystem from release 1.68 for gcc 5.2-6.3

After update boost from 1.67 to 1.68 our code can't be linked.

Here simple code: https://wandbox.org/permlink/MKhgrdMO1p1aHEOn
/tmp/ccXDdraA.o: In function `boost::system::generic_category()': prog.cc:(.text._ZN5boost6system16generic_categoryEv[_ZN5boost6system16generic_categoryEv]+0x5): undefined reference to `boost::system::detail::generic_category_instance' collect2: error: ld returned 1 exit status

P.S.
Can you test others boost component? I think this is not the only problem in this release.

Compile error for Android Level API lower then 21 with NDK r16

Boost 1.65.1
Compiler: clang from android NDK
clang_version__ "5.0.300080 "
NDK version: 16.0.4442984
Build system: cmake (3.6.4111459)
Std librariy: libc++_static
Build machine: Ubuntu 16.04.3 LTS

Since Google introduced unified headers the following compile error occurs to build boost::filesystem for example Android Level 19:

clang-linux.compile.c++.without-pth bin.v2/libs/filesystem/build/clang-linux-android/release/link-static/target-os-android/threading-multi/operations.o
libs/filesystem/src/operations.cpp:1705:12: error: no member named 'truncate' in the global namespace
    error(!BOOST_RESIZE_FILE(p.c_str(), size) ? BOOST_ERRNO : 0, p, ec,
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libs/filesystem/src/operations.cpp:224:38: note: expanded from macro 'BOOST_RESIZE_FILE'
#   define BOOST_RESIZE_FILE(P,SZ)(::truncate(P, SZ)== 0)
                                   ~~^
1 error generated.

  "/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++" -c -x c++ -Qunused-arguments -std=c++11 -frtti -fexceptions -Oz -DNDEBUG  -isysroot /android-sdk/ndk-bundle/sysroot --target=armv7-none-linux-androideabi --gcc-toolchain=/android-sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -isystem /android-sdk/ndk-bundle/sysroot/usr/include -I/android-sdk/ndk-bundle/sources/cxx-stl/llvm-libc++/include -I/android-sdk/ndk-bundle/sources/android/support/include -I/android-sdk/ndk-bundle/sources/cxx-stl/llvm-libc++abi/include -O3 -Wno-inline -Wall -DBOOST_LOG_WITHOUT_SYSLOG -isystem /android-sdk/ndk-bundle/sysroot/usr/include/arm-linux-androideabi -D__ANDROID_API__=19 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fno-integrated-as -mthumb -Wa,--noexecstack -Wformat -Werror=format-security  -Qunused-arguments -DHAVE_PTHREADS -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DBOOST_SYSTEM_STATIC_LINK=1 -DNDEBUG -I"." -o "bin.v2/libs/filesystem/build/clang-linux-android/release/link-static/target-os-android/threading-multi/operations.o" "libs/filesystem/src/operations.cpp"

...failed clang-linux.compile.c++.without-pth bin.v2/libs/filesystem/build/clang-linux-android/release/link-static/target-os-android/threading-multi/operations.o...

Before they introduced the unified headers the header file unistd.h didn't care about the defines _FILE_OFFSET_BITS or __USE_FILE_OFFSET64. Here a snippit from unistd.h:

...
extern int lchown(const char *, uid_t, gid_t);
extern int truncate(const char *, off_t);
extern char *getcwd(char *, size_t);

extern int sync(void);

extern int close(int);
extern off_t lseek(int, off_t, int);
extern off64_t lseek64(int, off64_t, int);

extern ssize_t read(int, void *, size_t);
extern ssize_t write(int, const void *, size_t);
extern ssize_t pread(int, void *, size_t, off_t);
extern ssize_t pread64(int, void *, size_t, off64_t);
extern ssize_t pwrite(int, const void *, size_t, off_t);
extern ssize_t pwrite64(int, const void *, size_t, off64_t);

extern int dup(int);
extern int dup2(int, int);
extern int fcntl(int, int, ...);
extern int ioctl(int, int, ...);
extern int flock(int, int);
extern int fsync(int);
extern int fdatasync(int);
extern int ftruncate(int, off_t);
extern int ftruncate64(int, off64_t);
...

Since they introduced unified headers, unistd.h snippet:

...
#if defined(__USE_FILE_OFFSET64)

#if __ANDROID_API__ >= 21
int truncate(const char* __path, off_t __length) __RENAME(truncate64) __INTRODUCED_IN(21);
#endif /* __ANDROID_API__ >= 21 */


#if __ANDROID_API__ >= 12
ssize_t pread(int __fd, void* __buf, size_t __count, off_t __offset)
  __overloadable __RENAME(pread64) __INTRODUCED_IN(12);
ssize_t pwrite(int __fd, const void* __buf, size_t __count, off_t __offset)
  __overloadable __RENAME(pwrite64) __INTRODUCED_IN(12);
int ftruncate(int __fd, off_t __length) __RENAME(ftruncate64) __INTRODUCED_IN(12);
#endif /* __ANDROID_API__ >= 12 */

#else
int truncate(const char* __path, off_t __length);
ssize_t pread(int __fd, void* __buf, size_t __count, off_t __offset)
    __overloadable __RENAME_CLANG(pread);
ssize_t pwrite(int __fd, const void* __buf, size_t __count, off_t __offset)
    __overloadable __RENAME_CLANG(pwrite);
int ftruncate(int __fd, off_t __length);
#endif


#if __ANDROID_API__ >= 21
int truncate64(const char* __path, off64_t __length) __INTRODUCED_IN(21);
#endif /* __ANDROID_API__ >= 21 */


#if __ANDROID_API__ >= 12
ssize_t pread64(int __fd, void* __buf, size_t __count, off64_t __offset)
    __INTRODUCED_IN(12) __overloadable __RENAME_CLANG(pread64);
ssize_t pwrite64(int __fd, const void* __buf, size_t __count, off64_t __offset)
    __INTRODUCED_IN(12) __overloadable __RENAME_CLANG(pwrite64);
int ftruncate64(int __fd, off64_t __length) __INTRODUCED_IN(12);
#endif /* __ANDROID_API__ >= 12 */
...

Now they care about the define __USE_FILE_OFFSET64 and truncate for 64 Bit is not implemented before Android API level 21.

We compile and use boost::filesystem for Android API level 19 long time ago and we didn't have any problems. So I decided to "if def" the define of __USE_FILE_OFFSET64 around (see snippet):

...
#if !(defined(__ANDROID__) && __ANDROID_API__ < 21)
//  define 64-bit offset macros BEFORE including boost/config.hpp (see ticket #5355)
#if !(defined(__HP_aCC) && defined(_ILP32) && !defined(_STATVFS_ACPP_PROBLEMS_FIXED))
#define _FILE_OFFSET_BITS 64 // at worst, these defines may have no effect,
#endif
#if !defined(__PGI)
#define __USE_FILE_OFFSET64 // but that is harmless on Windows and on POSIX
      // 64-bit systems or on 32-bit systems which don't have files larger 
      // than can be represented by a traditional POSIX/UNIX off_t type. 
      // OTOH, defining them should kick in 64-bit off_t's (and thus 
      // st_size)on 32-bit systems that provide the Large File
      // Support (LFS)interface, such as Linux, Solaris, and IRIX.
      // The defines are given before any headers are included to
      // ensure that they are available to all included headers.
      // That is required at least on Solaris, and possibly on other
      // systems as well.
#else
#define _FILE_OFFSET_BITS 64
#endif
#endif
...

to skip this so that we have the same behavior as before, I think.

I create a patch file but I will not create a pull request because I think this is not the right solution and you should decided if this is a proper fix or not. Then I can create a pull request, if needed.

I added the patch file and a preprocessor output of the compile error in operations.cpp

operations.ii.txt

android-boost-1_65_1-patches.patch.txt

You need some help or additional information, let me know.

Unable to build on windows with <warnings-as-errors>on

While building Boost.Iostreams with warnings-as-errors enabled, I encountered these warnings:

compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\release\address-model-64\cxxstd-03-iso\link-static\threading-multi\unique_path.obj
unique_path.cpp
libs\filesystem\src\unique_path.cpp(114): error C2220: warning treated as error - no 'object' file generated
libs\filesystem\src\unique_path.cpp(114): warning C4267: 'argument': conversion from 'size_t' to 'DWORD', possible loss of data
    call "bin.v2\standalone\msvc\msvc-14.1\address-model-64\architecture-x86\msvc-setup.bat"  >nul
 cl /Zm800 -nologo @"bin.v2\libs\filesystem\build\msvc-14.1\release\address-model-64\cxxstd-03-iso\link-static\threading-multi\unique_path.obj.rsp" 
...failed compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\release\address-model-64\cxxstd-03-iso\link-static\threading-multi\unique_path.obj...
compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\release\address-model-64\cxxstd-03-iso\link-static\threading-multi\windows_file_codecvt.obj
windows_file_codecvt.cpp
libs\filesystem\src\windows_file_codecvt.cpp(43): error C2220: warning treated as error - no 'object' file generated
libs\filesystem\src\windows_file_codecvt.cpp(43): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
libs\filesystem\src\windows_file_codecvt.cpp(63): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
    call "bin.v2\standalone\msvc\msvc-14.1\address-model-64\architecture-x86\msvc-setup.bat"  >nul
 cl /Zm800 -nologo @"bin.v2\libs\filesystem\build\msvc-14.1\release\address-model-64\cxxstd-03-iso\link-static\threading-multi\windows_file_codecvt.obj.rsp" 
...failed compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\release\address-model-64\cxxstd-03-iso\link-static\threading-multi\windows_file_codecvt.obj...
compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\debug\address-model-64\cxxstd-03-iso\link-static\threading-multi\unique_path.obj
unique_path.cpp
libs\filesystem\src\unique_path.cpp(114): error C2220: warning treated as error - no 'object' file generated
libs\filesystem\src\unique_path.cpp(114): warning C4267: 'argument': conversion from 'size_t' to 'DWORD', possible loss of data
    call "bin.v2\standalone\msvc\msvc-14.1\address-model-64\architecture-x86\msvc-setup.bat"  >nul
 cl /Zm800 -nologo @"bin.v2\libs\filesystem\build\msvc-14.1\debug\address-model-64\cxxstd-03-iso\link-static\threading-multi\unique_path.obj.rsp" 
...failed compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\debug\address-model-64\cxxstd-03-iso\link-static\threading-multi\unique_path.obj...
compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\debug\address-model-64\cxxstd-03-iso\link-static\threading-multi\windows_file_codecvt.obj
windows_file_codecvt.cpp
libs\filesystem\src\windows_file_codecvt.cpp(43): error C2220: warning treated as error - no 'object' file generated
libs\filesystem\src\windows_file_codecvt.cpp(43): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
libs\filesystem\src\windows_file_codecvt.cpp(63): warning C4244: 'argument': conversion from '__int64' to 'int', possible loss of data
    call "bin.v2\standalone\msvc\msvc-14.1\address-model-64\architecture-x86\msvc-setup.bat"  >nul
 cl /Zm800 -nologo @"bin.v2\libs\filesystem\build\msvc-14.1\debug\address-model-64\cxxstd-03-iso\link-static\threading-multi\windows_file_codecvt.obj.rsp" 
...failed compile-c-c++ bin.v2\libs\filesystem\build\msvc-14.1\debug\address-model-64\cxxstd-03-iso\link-static\threading-multi\windows_file_codecvt.obj...

Perhaps boost::numeric_cast<...> could help resolve these.

boost::filesystem::canonical returns incorrect paths on older linux versions.

in operations.cpp in function path read_symlink(const path& p, system::error_code* ec) the GNU coreutils function readlink is used to resolve the symlink, however the function does not set a null termination byte in the buffer after the call is made.

The documentation for the gnu coreutils function readlink states that it does not set a null termination byte. When the buffer is then used to assign a new path it, in some cases, contain control characters that a call to the readlink cmdline tool does not return.

This is mainly an issue on older linux versions (I have tested on Centos 5.11) in conjunction with the prelink tool.

Code used to reproduce:

boost::system::error_code error_code;
boost::filesystem::path path("/proc");
if (boost::filesystem::is_directory(path, error_code) && !error_code)
{
    boost::filesystem::directory_iterator it(path, error_code);
    if (!error_code)
    {
        for (;it != boost::filesystem::directory_iterator(); ++it)
        {
            bool is_dir = boost::filesystem::is_directory(it->path(), error_code);
            if (is_dir && !error_code)
            {
                boost::filesystem::path exe_path = it->path() / "exe";
                auto is_link = boost::filesystem::is_symlink(exe_path, error_code);
                if (!error_code && is_link)
                {
                    auto resolved = boost::filesystem::canonical(exe_path, error_code);
                    if (!error_code)
                    {
                        std::cout << "resolved: " << resolved << '\n';
                    }
                }
            }
        }
    }
}

Sample output:

resolved: /usr/libexec/gnome-vfs-daemon�������֌ (deleted)
resolved: /usr/bin/bt-applet����5[j+���� (deleted)
resolved: /usr/bin/python.#prelink# (deleted)
resolved: /usr/bin/nm-applet_���5[j+P��� (deleted)
resolved: /usr/bin/pam-panel-icon!`˻ (deleted)
resolved: /sbin/pam_timestamp_checkǽ֌+ (deleted)
resolved: /usr/bin/gnome-power-manager.#prelink#.p2mBLg (deleted)
resolved: /usr/lib64/esc-1.1.0/escd
                                   5
                                    5
                                     5
                                      5
                                       5 (deleted)
resolved: /usr/sbin/nm-system-settingsV��O�+f��O�+ (deleted)
resolved: /usr/libexec/gam_server (deleted)
resolved: /usr/libexec/wnck-applet���1 (deleted)
resolved: /usr/libexec/trashapplet�h���� (deleted)

The readlink cmdline tool implementation and usage of the readlink function differs slightly from boost, but it explicitly sets the null terminating byte. And if the resulting path takes the null terminating byte into consideration when doing the assign call, the result does not contain the control characters.

Sample solution:

BOOST_FILESYSTEM_DECL
path read_symlink(const path& p, system::error_code* ec)
{
  path symlink_path;

# ifdef BOOST_POSIX_API

  for (std::size_t path_max = 64;; path_max *= 2)// loop 'til buffer large enough
  {
    boost::scoped_array<char> buf(new char[path_max]);
    ssize_t result;
    if ((result=::readlink(p.c_str(), buf.get(), path_max))== -1)
    {
      if (ec == 0)
        BOOST_FILESYSTEM_THROW(filesystem_error("boost::filesystem::read_symlink",
          p, error_code(errno, system_category())));
      else ec->assign(errno, system_category());
      break;
    }
    else
    {
      if(result != static_cast<ssize_t>(path_max))
      {
        // Set null terminating byte.
        buf[result] = 0;
        // Create a string from buffer
        std::string symlink{buf.get()};
        // Assign path from string.
        symlink_path.assign(symlink.begin(), symlink.end());
        if (ec != 0) ec->clear();
        break;
      }
    }
  }
  //...
#endif
}

Not sure if the solution is the proper one for boost, but the idea is there at least.

Windows permission documentation

We found the description of filesystem perms enumerator [1] confussing and started a discussion on boost mailinglist [2]. It appears the filesystem uses the readonly attribute to set permissions and does not use Windows ACL at all. We believe that the current docs should be updated to give a better description of the permissions functionality.

We also believe this windows warning[1] is somewhat hidden and it would be usefull if this warning was included in the permissons description [3].

[1] - https://www.boost.org/doc/libs/1_69_0/libs/filesystem/doc/reference.html#Enum-perms
[2] - https://lists.boost.org/boost-users/2018/12/89232.php
[3] - https://www.boost.org/doc/libs/1_69_0/libs/filesystem/doc/reference.html#permissions

kind regards, Lars

Can't use operator<< on path that contains '&' character

Hello,

I have a problem with the use of boost::filesystem::path on Windows when the path contains one '&' character.

For example if my path is like:

"C:\foo & bar\Hello & World.txt"

Then the result of the operator<< of boost::filesystem::path will be:

"C:\foo && bar\Hello && World.txt"

Is this behavior normal ? Seems that the operator<< consider the character "&" as an escape one. But then how do we do when there is this character in the path ?

Any idea ?

Thanks !

Kevin HAVRANEK

use of deprecated header

path.hpp now produces a deprecation warning due to the inclusion of boost/io/detail/quoted_manip.hpp, which is apparently replaced by boost/io/quoted.hpp

Windows only: filesystem::path::canonical() hangs forever when a junction targets to a directory on different drive

Windows only ...

filesystem::path::canonical() hangs forever when a junction points to a different drive

Example:

Drive D:
mkdir  \MyJunctionTarget

Drive C:
cd \
mklink /j C:\MyJunction  D:\MyJunction

now - a program containing

boost::filesystem::path p(boost::filesystem::canonical("C:\\MyJunction"));

never returns (hangs forever). See also attached boost_junction_issue.cpp.

note: tested with libboost v1.71.0. But it is also reproducable with libboost v1.57.0.

GCC suggest-override warnings

Complete list of warnings when Boost 1.72 is built with GCC 7.5 with -Wsuggest-override added to cxxflags:

./boost/filesystem/exception.hpp:72:15: warning: ‘virtual const char* boost::filesystem::filesystem_error::what() const’ can be marked override [-Wsuggest-override]
libs/filesystem/src/codecvt_error_category.cpp:34:19: warning: ‘virtual const char* {anonymous}::codecvt_error_cat::name() const’ can be marked override [-Wsuggest-override]
libs/filesystem/src/codecvt_error_category.cpp:35:20: warning: ‘virtual std::__cxx11::string {anonymous}::codecvt_error_cat::message(int) const’ can be marked override [-Wsuggest-override]

Related: boostorg/config#253

space() function requires a directory path on Windows

I'm not sure whether this is a bug or a documentation issue.

On Windows, the filesystem::space call passes its parameter to GetDiskFreeSpaceEx, which is documented as requiring a directory.

The boost documentation makes no such requirement, and just talks about posix fstatvfs, which will accept a file.

So I think either the documentation needs updating to require a directory path, or the Windows code path needs to detect whether the parameter is a file and if so, strip the file name before calling GetDiskFreeSpaceEx.

Management of junction on Windows is broken and api is inconsistent

This bug was originally posted on boost-users mailing list on 06th November 2019 but since there is no answer I am reporting the bug here:
http://boost.2283326.n4.nabble.com/filesystem-Management-of-junction-on-Windows-is-broken-and-api-is-inconsistent-td4710249.html

I am bit disappointed to see that symlink and junctions are still not really properly handled on Windows with boost (and even with standard MS libs but that's another story.)
First my environment: Windows 10, VS2019 16.3.7, boost 1.71 (static, x64).
I would like to write a small c++ utility that does not use os dependent code to handle files so filesystem seems a good candidate.

Here is what I am doing on Windows:

cd "C:\Users\Vincent\AppData\Roaming\Apple Computer\MobileSync"
mklink /j Backup E:\Backup

So basically I am creating inside C:..\MobileSync a junction folder that points to my second harddisk E:\Backup

Then I am checking that Windows recognize it by entering dir command:

01/11/2019 23:48

.
01/11/2019 23:48 ..
01/11/2019 23:29 Backup [E:\Backup]
0 fichier(s) 0 octets
3 Rép(s) 59 393 630 208 octets libres

As you can see I have a junction targeting E:\Backup, so now let's see what happens with boost:

fs::path p(_T("C:\\Users\\Vincent\\AppData\\Roaming\\Apple Computer\\MobileSync\\Backup"));

sys::error_code ec;
if (fs::is_symlink(p))
{
     fs::path pathTarget = fs::read_symlink(p, ec);
}

When I display the content of pathTarget I have Backup...
I might not understand anything about junction but is it not supposed to return E:\Backup ??

When I look at source code I can see the following lines:

/********************** boost_1_71_0\libs\filesystem\src\operations.cpp **************************************/
symlink_path.assign(

static_cast<wchar_t*>(info.rdb.SymbolicLinkReparseBuffer.PathBuffer)
        + info.rdb.SymbolicLinkReparseBuffer.PrintNameOffset/sizeof(wchar_t),

static_cast<wchar_t*>(info.rdb.SymbolicLinkReparseBuffer.PathBuffer)
        + info.rdb.SymbolicLinkReparseBuffer.PrintNameOffset/sizeof(wchar_t)
        + info.rdb.SymbolicLinkReparseBuffer.PrintNameLength/sizeof(wchar_t));

This is wrong since we need to distinguish between a symlink and a junction folder so here is my contribution :

--- a/src/operations.cpp
+++ b/src/operations.cpp
@@ -201,6 +201,12 @@ typedef struct _REPARSE_DATA_BUFFER {
 # ifndef IO_REPARSE_TAG_SYMLINK
 #   define IO_REPARSE_TAG_SYMLINK (0xA000000CL)
 # endif
+# ifndef IO_REPARSE_TAG_MOUNT_POINT
+#   define IO_REPARSE_TAG_MOUNT_POINT (0xA0000003L)
+# endif
+# ifndef IO_REPARSE_TAG_DEDUP
+#   define IO_REPARSE_TAG_DEDUP (0x80000013L)
+# endif

 inline std::wstring wgetenv(const wchar_t* name)
 {
@@ -1661,12 +1667,24 @@ namespace detail
     if (!error(::DeviceIoControl(h.handle, FSCTL_GET_REPARSE_POINT,
           0, 0, info.buf, sizeof(info), &sz, 0) == 0 ? BOOST_ERRNO : 0, p, ec,
           "boost::filesystem::read_symlink" ))
+       if (info.rdb.ReparseTag == IO_REPARSE_TAG_SYMLINK)
+       {
       symlink_path.assign(

static_cast<wchar_t*>(info.rdb.SymbolicLinkReparseBuffer.PathBuffer)
         + info.rdb.SymbolicLinkReparseBuffer.PrintNameOffset/sizeof(wchar_t),

static_cast<wchar_t*>(info.rdb.SymbolicLinkReparseBuffer.PathBuffer)
         + info.rdb.SymbolicLinkReparseBuffer.PrintNameOffset/sizeof(wchar_t)
         + info.rdb.SymbolicLinkReparseBuffer.PrintNameLength/sizeof(wchar_t));
+       }
+       else if (info.rdb.ReparseTag == IO_REPARSE_TAG_MOUNT_POINT)
+       {
+          symlink_path.assign(
+ static_cast<wchar_t*>(info.rdb.MountPointReparseBuffer.PathBuffer)
+             + info.rdb.MountPointReparseBuffer.PrintNameOffset / sizeof(wchar_t),
+ static_cast<wchar_t*>(info.rdb.MountPointReparseBuffer.PathBuffer)
+             + info.rdb.MountPointReparseBuffer.PrintNameOffset / sizeof(wchar_t)
+             + info.rdb.MountPointReparseBuffer.PrintNameLength / sizeof(wchar_t));
+       }
 #     endif
     return symlink_path;
   } 

Test operations_test.cpp failed for not finding specified path in clang on Windows

Hi,
I found test operations_test.cpp failed for not finding specified path in clang on Windows when running function temp_directory_path_tests().
Below is the compiling and linking commands.

clang "..\..\..\libs\filesystem\src\utf8_codecvt_facet.cpp" -Foutf8_codecvt_facet.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "..\..\..\libs\filesystem\src\codecvt_error_category.cpp" -Focodecvt_error_category.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "..\..\..\libs\filesystem\src\portability.cpp" -Foportability.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "..\..\..\libs\filesystem\src\path_traits.cpp" -Fopath_traits.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "..\..\..\libs\filesystem\src\windows_file_codecvt.cpp" -Fowindows_file_codecvt.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "..\..\..\libs\filesystem\src\unique_path.cpp" -Founique_path.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "..\..\..\libs\filesystem\src\path.cpp" -Fopath.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "..\..\..\libs\filesystem\src\operations.cpp" -Fooperations.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
clang "operations_test.cpp" -Fooperations_test.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_FILESYSTEM_DYN_LINK=1 -I"..\..\.."
link /nologo /DEBUG /subsystem:console /out:"operations_test_clang.exe" operations_test.obj utf8_codecvt_facet.obj codecvt_error_category.obj portability.obj path_traits.obj windows_file_codecvt.obj unique_path.obj path.obj operations.obj

I got the following output when runing the executable file.

ksh-3.2$ ./operations_test_clang.exe
BOOST_WINDOWS_API is defined
...
temp_directory_path_tests...
 temp_directory_path() is "C:/Users/yuxianch/AppData/Local/Temp/3"

ERROR  ERROR  ERROR  ERROR  ERROR  ERROR  ERROR  ERROR  ERROR  ERROR  ERROR

****************************** std::exception *****************************
boost::filesystem::create_directory: The system cannot find the path specified: "op-test-e619-eefa\123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890#"
***************************************************************************

Can someone explain why this test failed in running temp_directory_path_tests()?
Thanks!

boost 1.68 fs::copy coredumped under OSX while fs::copy_file is okay.

system info : osx high sierra (10.13.6).

compiler : g++ ( Apple LLVM version 10.0.0 (clang-1000.10.44.4)).

compile cmd : g++ -std=c++17 -g -lboost_filesystem -lboost_system -o main main.cc

boost version: 1_68 .

reproduce code :

#include "boost/filesystem.hpp"

namespace fs = boost::filesystem;

int main(int argc, char** argv){

    fs::path _src("demo.cc");
    fs::path _dst("demo_dst.cc");

    //normal
    fs::copy_file(_src, _dst); 

    //core
    fs::copy(_src, _dst);
    return 0;
}

coredump info :

(lldb) target create "main"                                                                                             
Current executable set to 'main' (x86_64).                                                                              
(lldb) r                                                                                                                
Process 44221 launched: '/Users/arthur/test/main' (x86_64)                                                              
libboost_filesystem.dylib was compiled with optimization - stepping may behave oddly; variables may not be available.   
Process 44221 stopped                                                                                                   
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)                        
    frame #0: 0x00000001000a7435 libboost_filesystem.dylib`boost::filesystem::detail::copy(boost::filesystem::path const
&, boost::filesystem::path const&, boost::system::error_code*) [inlined] boost::system::error_code::operator bool(this=0
x0000000000000000) const at error_code.hpp:680 [opt]                                                                    
   677                                                                                                                  
   678        BOOST_SYSTEM_CONSTEXPR explicit operator bool() const BOOST_SYSTEM_NOEXCEPT  // true if error             
   679        {                                                                                                         
-> 680          return m_val != 0;                                                                                      
   681        }                                                                                                         
   682                                                                                                                  
   683  #else                                                                                                           
Target 0: (main) stopped.                                                                                               

I wonder if it was because fs::copy is deprecated in the recent boost release versions.

Quickbook no longer builds

The superproject builds (https://travis-ci.org/boostorg/boost/builds) have been broken, starting with https://travis-ci.org/boostorg/boost/builds/567103808; the error is

tools/quickbook/src/include_paths.cpp: In function ‘void quickbook::include_search_glob(std::set<quickbook::quickbook_path>&, const quickbook::quickbook_path&, std::string, quickbook::state&)’:
tools/quickbook/src/include_paths.cpp:160:14: error: ‘directory_iterator’ is not a member of ‘quickbook::fs’
         for (fs::directory_iterator dir_i(base_dir), dir_e; dir_i != dir_e;
              ^

Add some functionality like scandir

ITNOA

Hi,

Many times we need to scanning directory instead of iterating of directories to see all directory with specific order like alphabetically or based on modified time or …

So I think it is useful feature to boost::filesystem provide scan directories

thanks for great library

relative(path a_child, path a_base) fails if a_base contains symlinks on Windows

relative() calls weakly_canonical() passing a_base as argument, which in turn calls canonical(), which returns an empty path, and then relative() throws.
The message reads:
boost::filesystem::relative: The system cannot find the file specified: "T:\src\wxWidgets\interface\wx"

This wxWidgets is a symlink to wxWidgets-3.1.0.
If instead of "T:\src\wxWidgets\interface\wx" I call relative() with "T:\src\wxWidgets-3.1.0\interface\wx", the function succeeds.

operations_unit_test failures on Appveyor

When the tests are run from the Boost root with b2 libs/filesystem/test, the recursive iterator tests in operations_unit_test fail on Appveyor for some reason, I'm not sure why; current_path() is C:\projects\boost-root there. See f.ex. https://ci.appveyor.com/project/pdimov/filesystem/build/1.0.1-develop/job/xg5w0lancn3p8uj7#L809

When I run a similar configuration locally, I don't get this failure, but a recursive iterator from one level up from the Boost root enumerates half of the visible universe, so the test hangs for a very long time.

readdir_r is deprecated

While building 1.68.0 beta 1 on x86_64 linux with g++ 8.2.0, I get the following warnings:

gcc.compile.c++ bin.v2/libs/filesystem/build/gcc-8.2.0/release/link-static/threading-multi/operations.o

    "g++-8.2.0"   -I/softs/lin64-gcc-8.2.0/release/iconv/include -I/softs/lin64-gcc-8.2.0/release/gettext/include -I/softs/lin64-gcc-8.2.0/release/bzip2/include -I/softs/lin64-gcc-8.2.0/release/zlib/include -I/softs/lin64-gcc-8.2.0/release/jpeg-turbo/include -I/softs/lin64-gcc-8.2.0/release/xz/include -I/softs/lin64-gcc-8.2.0/release/tiff/include -I/softs/lin64-gcc-8.2.0/release/png/include -march=native -std=c++14 -O2 -DNDEBUG -m64 -pthread -O3 -finline-functions -Wno-inline -Wall  -DBOOST_ALL_NO_LIB=1 -DBOOST_ASIO_NO_DEPRECATED -DBOOST_FILESYSTEM_NO_DEPRECATED -DBOOST_FILESYSTEM_STATIC_LINK=1 -DBOOST_SYSTEM_STATIC_LINK=1 -DNDEBUG  -I"." -c -o "bin.v2/libs/filesystem/build/gcc-8.2.0/release/link-static/threading-multi/operations.o" "libs/filesystem/src/operations.cpp"

libs/filesystem/src/operations.cpp: Dans la fonction « int {anonyme}::readdir_r_simulator(DIR*, dirent*, dirent**) »:
libs/filesystem/src/operations.cpp:2126:18: warning: « int readdir_r(DIR*, dirent*, dirent**) » est obsolète [-Wdeprecated-declarations]
       { return ::readdir_r(dirp, entry, result); }
                  ^~~~~~~~~
In file included from /usr/include/features.h:423,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/x86_64-pc-linux-gnu/bits/os_defines.h:39,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/x86_64-pc-linux-gnu/bits/c++config.h:508,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/cstddef:49,
                 from ./boost/config/compiler/gcc.hpp:165,
                 from ./boost/config.hpp:39,
                 from ./boost/filesystem/operations.hpp:18,
                 from libs/filesystem/src/operations.cpp:68:
/usr/include/dirent.h:189:12: note: déclaré ici
 extern int __REDIRECT (readdir_r,
            ^~~~~~~~~~
libs/filesystem/src/operations.cpp:2126:47: warning: « int readdir_r(DIR*, dirent*, dirent**) » est obsolète [-Wdeprecated-declarations]
       { return ::readdir_r(dirp, entry, result); }
                                               ^
In file included from /usr/include/features.h:423,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/x86_64-pc-linux-gnu/bits/os_defines.h:39,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/x86_64-pc-linux-gnu/bits/c++config.h:508,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/cstddef:49,
                 from ./boost/config/compiler/gcc.hpp:165,
                 from ./boost/config.hpp:39,
                 from ./boost/filesystem/operations.hpp:18,
                 from libs/filesystem/src/operations.cpp:68:
/usr/include/dirent.h:189:12: note: déclaré ici
 extern int __REDIRECT (readdir_r,
            ^~~~~~~~~~
libs/filesystem/src/operations.cpp:2126:47: warning: « int readdir_r(DIR*, dirent*, dirent**) » est obsolète [-Wdeprecated-declarations]
       { return ::readdir_r(dirp, entry, result); }
                                               ^
In file included from /usr/include/features.h:423,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/x86_64-pc-linux-gnu/bits/os_defines.h:39,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/x86_64-pc-linux-gnu/bits/c++config.h:508,
                 from /softs/gcc-8.2.0/include/c++/8.2.0/cstddef:49,
                 from ./boost/config/compiler/gcc.hpp:165,
                 from ./boost/config.hpp:39,
                 from ./boost/filesystem/operations.hpp:18,
                 from libs/filesystem/src/operations.cpp:68:
/usr/include/dirent.h:189:12: note: déclaré ici
 extern int __REDIRECT (readdir_r,
            ^~~~~~~~~~

Possible memory leak in filesystem::copy()

Hi team,
Do you have any update on the ticket https://svn.boost.org/trac10/ticket/8007?
I'm seeing my app increases mem consumption exactly at the time I call filesystem::copy.

Every 12 hours, I copy 16 files each is about 1GB. After the copy process, RSS mem increases and stays flat for the next 12 hours. I expect RSS only increases during the copy process, but then will go back to the level before copying. I'm running Ubuntu 16.04 64bit, boost 1.66.

Thanks,
Trung

screen shot 2018-01-02 at 12 34 58

Can not iterate through directory entries on Android

Prerequisites

  • Android device with Android OS below 6.0, for example Samsung Galaxy S6 with Android 5.0.2. The issue should be reproducible on Android Emulator as well.
  • Android NDK with enabled Unified Headers, for example version r16b
  • Compiler: Clang (shouldn't be important)
  • STL: GNU STL (shouldn't be important)
  • Selected (minimum) Android API level: 14 (shouldn't be important)

How to reproduce

Compile the following sample code as Android executable.

#include <boost/filesystem.hpp>
#include <iostream>

int main() {
    namespace fs = boost::filesystem;

    fs::directory_iterator iter("/data/local/tmp");
    for (const fs::path& p: iter)
        std::cout << "Directory entry found: " << p.c_str() << std::endl;

    return 0;
}

Copy the resulting executable to the device and run it:

adb push libs/armeabi-v7a/sample-cmdline-app /data/local/tmp/
adb shell /data/local/tmp/sample-cmdline-app

Expected behavior

All directory entries are listed, for example:

Directory entry found: /data/local/tmp/sample-cmdline-app
Directory entry found: /data/local/tmp/some-another-file.txt

Actual behavior

boost::filesystem::filesystem_error is thrown on the second iteration of the loop:

Directory entry found: /data/local/tmp/sample-cmdline-app
terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
  what():  boost::filesystem::directory_iterator::operator++: Function not implemented: "/data/local/tmp"

UBSAN issue in 1.68.0

Using the Ubuntu Bionic docker container from boostorg/boost#184, this issue emerged:

'''
boost@a7d451ec0735:/boost/libs/filesystem/test$ UBSAN_OPTIONS=print_stacktrace=1 ../../../b2 toolset=gcc-7 cxxstd=03 cxxflags=-fno-omit-frame-pointer cxxflags=-fsanitize=undefined cxxflags=-fno-sanitize-recover=undefined linkflags=-fsanitize=undefined linkflags=-fno-sanitize-recover=undefined linkflags=-fuse-ld=gold variant=debug -q -a

...

testing.capture-output ../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.run
====== BEGIN OUTPUT ======
../../../libs/filesystem/src/operations.cpp:2346:28: runtime error: reference binding to null pointer of type 'struct error_code'
#0 0x7f11d4e00e67 in boost::filesystem::detail::directory_iterator_construct(boost::filesystem::directory_iterator&, boost::filesystem::path const&, boost::system::error_code*) ../../../libs/filesystem/src/operations.cpp:2346
#1 0x562749a68bba in boost::filesystem::directory_iterator::directory_iterator(boost::filesystem::path const&) ../../../boost/filesystem/operations.hpp:903
#2 0x562749a6a43b in boost::filesystem::recursive_directory_iterator::recursive_directory_iterator(boost::filesystem::path const&) ../../../boost/filesystem/operations.hpp:1158
#3 0x562749a64323 in cpp_main(int, char**) /boost/libs/filesystem/test/convenience_test.cpp:143
#4 0x562749a635ca in main ../../../boost/detail/lightweight_main.hpp:23
#5 0x7f11d357eb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
#6 0x562749a634c9 in _start (/boost/bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test+0x114c9)

EXIT STATUS: 1
====== END OUTPUT ======

LD_LIBRARY_PATH="/boost/bin.v2/libs/filesystem/build/gcc-7/debug/cxxstd-03-iso:/boost/bin.v2/libs/system/build/gcc-7/debug/cxxstd-03-iso:/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"

export LD_LIBRARY_PATH

status=0
if test $status -ne 0 ; then
    echo Skipping test execution due to testing.execute=off
    exit 0
fi
 "../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test"   > "../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.output" 2>&1 < /dev/null
status=$?
echo >> "../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.output"
echo EXIT STATUS: $status >> "../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.output"
if test $status -eq 0 ; then
    cp "../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.output" "../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.run"
fi
verbose=0
if test $status -ne 0 ; then
    verbose=1
fi
if test $verbose -eq 1 ; then
    echo ====== BEGIN OUTPUT ======
    cat "../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.output"
    echo ====== END OUTPUT ======
fi
exit $status

...failed testing.capture-output ../../../bin.v2/libs/filesystem/test/convenience_test.test/gcc-7/debug/cxxstd-03-iso/convenience_test.run...
...failed updating 1 target...
'''

Test failures on Cygwin g++

temp_directory_path_tests...
test\operations_test.cpp(1981): test 'p == tmp_path' failed in function 'void {anonymous}::temp_directory_path_tests()': '"op-test-674d-dc9b\123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890#"' != '"C:\Users\PETERD~1\AppData\Local\Temp"'
test\operations_test.cpp(1991): test 'tmp_path == fs::initial_path()' failed in function 'void {anonymous}::temp_directory_path_tests()': '"C:\Users\PETERD~1\AppData\Local\Temp"' != '"C:\Projects\boost-git\boost\libs\filesystem"'
Fallback test, temp_directory_path() returned "C:\Users\PETERD~1\AppData\Local\Temp"
test\operations_test.cpp(2095): test 'equivalent(test_temp_dir, ph)' failed in function 'void {anonymous}::temp_directory_path_tests()'
test\operations_test.cpp(2100): test 'equivalent(test_temp_dir, ph)' failed in function 'void {anonymous}::temp_directory_path_tests()'
test\operations_test.cpp(2107): test 'equivalent(test_temp_dir/L"Temp", ph)' failed in function 'void {anonymous}::temp_directory_path_tests()'
temp_directory_path() returned "C:\Users\PETERD~1\AppData\Local\Temp"
test\operations_test.cpp(2113): test 'equivalent(test_temp_dir/L"Temp", ph)' failed in function 'void {anonymous}::temp_directory_path_tests()'
temp_directory_path() returned "C:\Users\PETERD~1\AppData\Local\Temp"

It looks like guarded_env_var doesn't work.

recursive_directory_iterator::increment does not advance on error

I'm using the latest develop branch (will be 1.71)

From operations.cpp:

  // Invariant: On return, the top of the iterator stack is the next valid (possibly
  // end) iterator, regardless of whether or not an error is reported, and regardless of
  // whether any error is reported by exception or error code. In other words, progress
  // is always made so a loop on the iterator will always eventually terminate
  // regardless of errors.
  BOOST_FILESYSTEM_DECL
  void recur_dir_itr_imp::increment(system::error_code* ec)

This is not (always?) true as the iterator does not advance when an error is reported during push_directory.

Steps to reproduce

Consider this program (slightly modified simple_ls.cpp from examples)

recursive_ls.cpp

#define BOOST_FILESYSTEM_VERSION 3

//  As an example program, we don't want to use any deprecated features
#ifndef BOOST_FILESYSTEM_NO_DEPRECATED 
#  define BOOST_FILESYSTEM_NO_DEPRECATED
#endif
#ifndef BOOST_SYSTEM_NO_DEPRECATED 
#  define BOOST_SYSTEM_NO_DEPRECATED
#endif

#include "boost/filesystem/operations.hpp"
#include "boost/filesystem/path.hpp"
#include "boost/progress.hpp"
#include <iostream>

namespace fs = boost::filesystem;

int main(int argc, char* argv[])
{
  fs::path p(fs::current_path());

  if (argc > 1)
    p = fs::system_complete(argv[1]);
  else
    std::cout << "\nusage:   recursive_ls [path]" << std::endl;

  unsigned long file_count = 0;
  unsigned long dir_count = 0;
  unsigned long other_count = 0;
  unsigned long err_count = 0;

  if (!fs::exists(p))
  {
    std::cout << "\nNot found: " << p << std::endl;
    return 1;
  }

  if (fs::is_directory(p))
  {
    std::cout << "\nIn directory: " << p << "\n\n";
    fs::recursive_directory_iterator end_iter;
    for (fs::recursive_directory_iterator dir_itr(p);
          dir_itr != end_iter;)
    {
      try
      {
        if (fs::is_directory(dir_itr->status()))
        {
          ++dir_count;
          std::cout << dir_itr->path() << " [directory]\n";
        }
        else if (fs::is_regular_file(dir_itr->status()))
        {
          ++file_count;
          std::cout << dir_itr->path() << "\n";
        }
        else
        {
          ++other_count;
          std::cout << dir_itr->path() << " [other]\n";
        }

        boost::system::error_code ec;
        dir_itr.increment(ec);
        if (ec)
        {
            ++err_count;
            std::cout << "*" << dir_itr->path() << " *" << ec.message() << std::endl;
        }
      }
      catch (const std::exception & ex)
      {
        ++err_count;
        std::cout << dir_itr->path() << " " << ex.what() << std::endl;
      }
    }
    std::cout << "\n" << file_count << " files\n"
              << dir_count << " directories\n"
              << other_count << " others\n"
              << err_count << " errors\n";
  }
  else // must be a file
  {
    std::cout << "\nFound: " << p << "\n";    
  }
  return 0;
}

Now run the following commands (tested on linux):

mkdir testdir
chmod 000 testdir
mkdir otherdir
./recursive_ls

Actual result

usage:   recursive_ls [path]

In directory: "/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test"

"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" [directory]
*"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" *Permission denied
"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" [directory]
*"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" *Permission denied
"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" [directory]
*"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" *Permission denied
"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" [directory]
*"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" *Permission denied
"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" [directory]
*"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" *Permission denied

... infinite loop

Expected result

One of:

  1. Fullfill the invariant and make the iterator always advance. I can see that other tests expect that the for loop finishes without special handling or errors
  2. Drop the invariant and write into documentation that on errors, it.no_push() must be called in order to continue.

I'm not sure what is better.

The first would have the disadvantage that because the iterator has already advances, the current path would not correspond to the error code, like this:

"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/testdir" [directory]
*"/home/rodney/Projects/boost/boost/libs/filesystem/test/recursive_test/otherdir" *Permission denied

This is because testdir record is read and reported just fine, the error only happens once we try to recurse into that directory. So the contents are skipped and iterator now points to the next record in the parent directory.

The second variant, on the other hand, only works for errors connected to directory opens. What about other errors? Should another api be added to "acknowledge" the current error?

Many thanks for a wonderful library.

Handling of volume designators broken on windows

I noticed the problem when using canonical when I'm currently in a directory with a junction point.

To reproduce:

  • Create junction point: mklink /J C:/tmp/junction C:/tmp/real
  • Put an executable into C:/tmp/real which calls e.g. canonical("C:/tmp/real")
  • Execute once form within C:/tmp/junction and C:/tmp/real
  • Observe different results while stepping through boost source (see below)

What happens:

  • We are iterating over the path in
    for (path::iterator itr = source.begin(); itr != source.end(); ++itr)
  • The first entry is "C:"
  • This is passed into is_symlink in
    bool is_sym (is_symlink(detail::symlink_status(result, ec)));
    yielding true when in junction and false when in real!!!

Reason for that is the call to GetFileAttributesW in

DWORD attr(::GetFileAttributesW(p.c_str()));
. As described in e.g. https://stackoverflow.com/questions/38300808/why-does-getfileattributeswlc-return-file-attribute-reparse-point "C:" does not mean "the drive C" but "the current working directory on drive C" (https://docs.microsoft.com/de-de/windows/desktop/FileIO/naming-a-file)

If a file name begins with only a disk designator but not the backslash after the colon, it is interpreted as a relative path to the current directory on the drive with the specified letter.

This is a very serious issue as it returns wrong results in unexpected ways.

I'm not sure how to fix this. Maybe iteration should not return the "root_name" but the "root_path" first? Or the root_name on windows shall include the slash if given?

I'm afraid there was a mistake when defining the decomposition table (https://www.boost.org/doc/libs/1_68_0/libs/filesystem/doc/reference.html#path-decomposition-table): for c: vs c:\ (vs c:/?): the (back)slash after the colon has to be treated as part of the first element or the meaning is significantly altered. (I'm not 100% sure that C:\ is the same as C:/ but it seems like [GetFileAttributesW returns the same])

recursive_directory_iterator::increment fails assertion on corrupted file system

From operations.cpp

  // Invariant: On return, the top of the iterator stack is the next valid (possibly
  // end) iterator, regardless of whether or not an error is reported, and regardless of
  // whether any error is reported by exception or error code. In other words, progress
  // is always made so a loop on the iterator will always eventually terminate
  // regardless of errors.
  BOOST_FILESYSTEM_DECL
  void recur_dir_itr_imp::increment(system::error_code* ec)

This invariant is not honored on corrupted filesystems, when a nested directory cannot be traversed fully.
An example of such "corrupted" filesystem is Apple implementation of quarantine on Mac OS X High Sierra and higher.

Steps to reproduce

Consider this program:

#define BOOST_FILESYSTEM_VERSION 3

//  As an example program, we don't want to use any deprecated features
#ifndef BOOST_FILESYSTEM_NO_DEPRECATED 
#  define BOOST_FILESYSTEM_NO_DEPRECATED
#endif
#ifndef BOOST_SYSTEM_NO_DEPRECATED 
#  define BOOST_SYSTEM_NO_DEPRECATED
#endif

#include "boost/filesystem/operations.hpp"
#include "boost/filesystem/path.hpp"
#include "boost/progress.hpp"
#include <iostream>

namespace fs = boost::filesystem;

int main(int argc, char* argv[])
{
  fs::path p(fs::current_path());

  if (argc > 1)
    p = fs::system_complete(argv[1]);
  else
    std::cout << "\nusage:   recursive_ls [path]" << std::endl;

  unsigned long file_count = 0;
  unsigned long dir_count = 0;
  unsigned long other_count = 0;
  unsigned long err_count = 0;

  if (!fs::exists(p))
  {
    std::cout << "\nNot found: " << p << std::endl;
    return 1;
  }

  if (fs::is_directory(p))
  {
    boost::system::error_code ec;
    std::cout << "\nIn directory: " << p << "\n\n";
    fs::recursive_directory_iterator end_iter;
    for (fs::recursive_directory_iterator dir_itr(p);
          dir_itr != end_iter;)
    {
      try
      {
        if (fs::is_directory(dir_itr->status()))
        {
          ++dir_count;
          std::cout << dir_itr->path() << " [directory]\n";
        }
        else if (fs::is_regular_file(dir_itr->status()))
        {
          ++file_count;
          std::cout << dir_itr->path() << "\n";
        }
        else
        {
          ++other_count;
          std::cout << dir_itr->path() << " [other]\n";
        }

        boost::system::error_code ec;
        dir_itr.increment(ec);
        if (ec)
        {
            ++err_count;
            std::cout << "*" << dir_itr->path() << " *" << ec.message() << std::endl;
        }
      }
      catch (const std::exception & ex)
      {
        ++err_count;
        std::cout << dir_itr->path() << " " << ex.what() << std::endl;
      }
    }
    std::cout << "\n" << file_count << " files\n"
              << dir_count << " directories\n"
              << other_count << " others\n"
              << err_count << " errors\n";
  }
  else // must be a file
  {
    std::cout << "\nFound: " << p << "\n";    
  }
  return 0;
}


1. Real world example (Mac OS X High Sierra)

  1. Download terminal application from here: https://www.iterm2.com (In theory, any app will do)
  2. Do not install it!
  3. Just run the downloaded file from whenever it was downloaded to
  4. I repeat, do not let MacOS move it to /Applications or anywhere else
  5. Check that mount command gives you something like:
/Users/<user>/Downloads/iTerm.app on /private/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/317B211C-C40C-4662-8947-416AC32DD07B (nullfs, local, nodev, nosuid, read-only, nobrowse, mounted by iscan)
  1. Leave the app running
  2. ./recursive_ls /var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/ (ids may vary, but the path must end with AppTranslocation)

Output:


In directory: "/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/"

"/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/317B211C-C40C-4662-8947-416AC32DD07B" [directory]
"/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/317B211C-C40C-4662-8947-416AC32DD07B/d" [directory]
"/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/317B211C-C40C-4662-8947-416AC32DD07B/d/iTerm.app" [directory]

... shortened ...

"/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/317B211C-C40C-4662-8947-416AC32DD07B/d/iTerm.app/Contents/Frameworks/ColorPicker.framework/ColorPicker"
"/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/317B211C-C40C-4662-8947-416AC32DD07B/d/iTerm.app/Contents/Info.plist"
"/var/folders/ys/08961svn0qx6_1g9sg6j32sw0000gn/T/AppTranslocation/317B211C-C40C-4662-8947-416AC32DD07B/d/iTerm.app/Contents/PkgInfo"
Assertion failed: ((m_imp.get())&&("attempt to dereference end iterator")), function dereference, file ../../../boost/filesystem/operations.hpp, line 1013.
*Abort trap: 6

2. Artifical example (any linux or MAC with FUSE)

I have been able to reproduce this behaviour relatively closely with fusepy using the attached script.
memoryfs.py.txt

Python3 and pip is required

sudo -s                               # Switch to root
pip3 install fusepy
python3 memoryfs.py mountpoint/ &     # The file memoryfs.py must be downloaded to the current directory
mkdir -p mountpoint/dirxx/dir2
mkdir -p mountpoint/diryy/dir2
./recursive_ls

Output:


usage:   recursive_ls [path]

In directory: "/home/rodney/Projects/boost/libs/filesystem/test"

"/home/rodney/Projects/boost/libs/filesystem/test/mountpoint" [directory]
"/home/rodney/Projects/boost/libs/filesystem/test/mountpoint/dirxx" [directory]
*"/home/rodney/Projects/boost/libs/filesystem/test/mountpoint/dirxx" *Input/output error
"/home/rodney/Projects/boost/libs/filesystem/test/mountpoint/dirxx" [directory]
recursive_ls: ../../../boost/filesystem/operations.hpp:1013: boost::filesystem::directory_entry& boost::filesystem::directory_iterator::dereference() const: Assertion `(m_imp.get())&&("attempt to dereference end iterator")' failed.
Aborted

Analysis

In both cases, the opendir passes and readdir (or readdir_r) provides first few entries of the nested directory. After a few entries, readdir(_r) fails with an error.

The following condition gets activated:

void directory_iterator_increment(directory_iterator& it,
    system::error_code* ec)
	
	/* .... */
	
	 if (increment_ec)  // happens if filesystem is corrupt, such as on a damaged optical disc
        {
          boost::intrusive_ptr< detail::dir_itr_imp > imp;
          imp.swap(it.m_imp);
          path error_path(imp->dir_entry.path().parent_path());  // fix ticket #5900
          if (ec == 0)
            BOOST_FILESYSTEM_THROW(
              filesystem_error("boost::filesystem::directory_iterator::operator++",
                error_path,
                increment_ec));
          *ec = increment_ec;
          return;
        }

which is then handled here:

  void recur_dir_itr_imp::increment(system::error_code* ec)
  {
    /* ..... */

    //  Do the actual increment operation on the top iterator in the iterator
    //  stack, popping the stack if necessary, until either the stack is empty or a
    //  non-end iterator is reached.
    while (!m_stack.empty())
    {
      directory_iterator& it = m_stack.top();
      detail::directory_iterator_increment(it, ec);
      if (ec && *ec)     /// <<<<<----------------- here
        return;
      if (it != directory_iterator())
        break;

      m_stack.pop();
      --m_level;
    }

it equals directory_iterator() at the marked point, but we return too early to progress to another (possibly valid) directory.

Add creation_time API in operations.hpp

ITNOA

I think it is good to add API to provide creation time of path. with below signature

std::time_t creation_time(const path& p)

std::time_t creation_time(const path& p, system::error_code& ec) BOOST_NOEXCEPT

failes to build QNX 7; not supported system function "fchmodat"

Hi there,
I tried to build boost for QNX 7 (compiler qcc 5.4.0; under the hood GCC 5.4.0). the compilation stops in operations.cpp at line

if (::fchmodat(AT_FDCWD, p.c_str(), mode_cast(prms),

with the following error

error: '::fchmodat' has not been declared

QNX does not support the "fchmodat" function. For other systems like Solaris there is a preprocessor check if the function is supported. This preprocessor check misses QNX.
By adding the following line to the preprocessor check at line

&& !(defined(linux) || defined(__linux) || defined(__linux__)) \

&& !(defined(__QNX__) && (_NTO_VERSION <= 700)) \

I was able to compile for QNX 7 successfully.
The version check is deliberately included as future versions of QNX could add the "fchmodat" function (QNX adapts functionality slowly).

Best regards

boost/filesystem.hpp

the following statement will not work in a Win32 environment
filesize = boost::filesystem::file_size(p, ec);
value of p is: {m_pathname=L"logging"}
the function returns as value for ec:
{val_=2 failed_=true cat_=0x006d7688 {ManageServer.exe!boost::system::detail::system_error_category boost::system::detail::cat_holder<void>::sysyem_category_instance}{...}}

compile problems on Android ndk r16 beta 1

Hello, in attempt to build boost for android with more or less reasonable strict settings (-Werror) I uncovered some bugs.

/usr/local/opt/android-ndk/android-ndk-r16-beta1//sources/android/support/include/stdio.h:36:25: error: invalid token at start of a preprocessor expression
#if __USE_FILE_OFFSET64 && __ANDROID_API__ < __ANDROID_API_N__

This happens because filesystem defines a "harmless" macro:

libs/filesystem/src/operations.cpp:18:9: warning: '__USE_FILE_OFFSET64' macro redefined [-Wmacro-redefined]
#define __USE_FILE_OFFSET64 // but that is harmless on Windows and on POSIX
        ^

While this check was #if defined(__USE_FILE_OFFSET64) in previous android versions, it seems to have changed when they introduced unified headers. It's probably more "harmless" to define an integer value in boost.filesystem rather than persuade Google to fix their code.

This is using clang toolchain on Android NDK r16 beta1.

​Reference to user-config.jam for the build

Regression in recursive_directory_iterator

There seems to be a regression introduced in recursive_directory_iterator with the release of boost 1.70.

It can be fairly easily reproduced by running tut6b.cpp in the examples directory, which crashes for me with the following backtrace:

#0  0x000055555555caa0 in boost::intrusive_ptr<boost::filesystem::detail::dir_itr_imp>::get (this=0xfffffffffffffff8) at thirdparty/boost/boost_external-prefix/include/boost/smart_ptr/intrusive_ptr.hpp:181
#1  0x000055555555ba2c in boost::filesystem::directory_iterator::dereference (this=0xfffffffffffffff8) at thirdparty/boost/boost_external-prefix/include/boost/filesystem/operations.hpp:1014
#2  0x000055555555dd00 in boost::iterators::iterator_core_access::dereference<boost::filesystem::directory_iterator> (f=...) at thirdparty/boost/boost_external-prefix/include/boost/iterator/iterator_facade.hpp:550
#3  0x000055555555ce88 in boost::iterators::detail::iterator_facade_base<boost::filesystem::directory_iterator, boost::filesystem::directory_entry, boost::iterators::single_pass_traversal_tag, boost::filesystem::directory_entry&, long, false, false>::operator* (this=0xfffffffffffffff8)
    at thirdparty/boost/boost_external-prefix/include/boost/iterator/iterator_facade.hpp:656
#4  0x00005555555683b8 in boost::iterators::detail::iterator_facade_base<boost::filesystem::directory_iterator, boost::filesystem::directory_entry, boost::iterators::single_pass_traversal_tag, boost::filesystem::directory_entry&, long, false, false>::operator-> (this=0xfffffffffffffff8)
    at ./boost/iterator/iterator_facade.hpp:661
#5  0x00005555555666e3 in boost::filesystem::detail::recur_dir_itr_imp::push_directory (this=0x55555558be70, ec=...) at libs/filesystem/src/operations.cpp:2571
#6  0x00005555555668f4 in boost::filesystem::detail::recur_dir_itr_imp::increment (this=0x55555558be70, ec=0x0) at libs/filesystem/src/operations.cpp:2609
#7  0x000055555555be40 in boost::filesystem::recursive_directory_iterator::increment (this=0x7fffffffddd8) at thirdparty/boost/boost_external-prefix/include/boost/filesystem/operations.hpp:1345
#8  0x000055555555ddf2 in boost::iterators::iterator_core_access::increment<boost::filesystem::recursive_directory_iterator> (f=...) at thirdparty/boost/boost_external-prefix/include/boost/iterator/iterator_facade.hpp:556
#9  0x000055555555cfe6 in boost::iterators::detail::iterator_facade_base<boost::filesystem::recursive_directory_iterator, boost::filesystem::directory_entry, boost::iterators::single_pass_traversal_tag, boost::filesystem::directory_entry&, long, false, false>::operator++ (this=0x7fffffffddd8)
    at thirdparty/boost/boost_external-prefix/include/boost/iterator/iterator_facade.hpp:666
#10 0x0000555555559993 in main (argc=2, argv=0x7fffffffdf18) at ../tut6b.cpp:34

The same code works just fine with boost-1.69. The same issue is present in boost 1.71 as well as the master branch. The issue is not present on Windows.

It seems like commit 3a90694 introduced the regression.

I have tried to see if I could find the exact cause of the bug without luck.

Please let me know if you need any more information or if there's something more I should try.

Thanks a lot.

Windows tests not executed

The windows-only tests are not executed since it checks BOOST_PLATFORM against "Windows" but that definition is defined to "Win32" instead: https://github.com/boostorg/config/blob/c2b1df496ab856921316ec29ddfe7d11013594fb/include/boost/config/platform/win32.hpp#L13

This affects e.g.

if (platform == "Windows" && language_id == 0x0409) // English (United States)

but other files I found using grep include:

example/directory_symlink_parent_resolution.cpp
test/deprecated_test.cpp
test/operations_test.cpp
test/path_test.cpp
test/issues/recurse_dir_iter_5403.cpp

To avoid this I'd suggest using the Jamfile mechanism to determine whether to execute the platform specific tests if that provides checking against invalid platforms(/target-os)

boost/filesystem.hpp warning c4244 (MSVC2019)

Inside a C++ class I created the following member function.
VOID write_logfile(const tuple_logging& tl) { // write tuple to file wofs.open(p, std::ios_base::app); // results in a compiler warning: // warning C4244: ... // TODO: get rid of this warning wofs << boost::tuples::set_delimiter(L'#') << tl << L'\n'; wofs.close(); }
The relevant member variables are:
boost::filesystem::wofstream wofs;
The tuple, named tl, is constructed with std::wstring elements
The relevant includes are:
#include <boost/filesystem.hpp> #include <boost/tuple/tuple.hpp> #include <boost/tuple/tuple_io.hpp>
The compiler inside the MSVC2019 IDE gives me the following warning.
warning C4244: 'argument': conversion from 'const CharType' to 'const char', possible loss of data (and a lot more of this stuff) On the following statement.
wofs << boost::tuples::set_delimiter(L'#') << tl << L'\n';
Can someone help me to get rid of this warning?
`

Trac #7404 "filesystem::canonical fails on UNC paths on Windows" still an issue in 1.68

Original text:

The filesystem library function canonical(p, base) fails for all UNC paths on Windows because it gets an "invalid path" error when it calls symlink_status at line 816 of operations.cpp. Example: For the path "\server\share\file", the first time execution reaches line 816 of operations.cpp, it calls is_symlink(detail::symlink_status("server", ec)) and ec is set to the Windows "invalid path" system error code.

This is still an issue. Minimal program to reproduce:

#include <boost/filesystem.hpp>
#include <cstdio>

namespace fs = boost::filesystem;

int main(int argc, const char** argv)
{
	fs::path p(R"(\\localhost\c$)");
	boost::system::error_code ec;
	auto p2 = fs::canonical(p,ec);
	printf("%d", ec.value());
}

Only happens on Windows platforms, specifically because of the call to boost::filesystem::detail::symlink_status during filesystem::canonical

Executable permission bit on Windows

Probably nobody particularly cares (including me), but while answering a question on the Boost-Users list today I noticed a bug in the way that Boost.Filesystem reports the executable bit for files on Windows.

path ext = p.extension();
wchar_t const* q = ext.c_str();
if (equal_extension(q, L".exe", L".EXE")
|| equal_extension(q, L".com", L".COM")
|| equal_extension(q, L".bat", L".BAT")
|| equal_extension(q, L".cmd", L".CMD"))
prms |= fs::owner_exe | fs::group_exe | fs::others_exe;

This code will set the bit when the path extension matches one of these literal strings.

Firstly, it's performing a case-sensitive comparison; while it does explicitly match both all-uppercase and all-lowercase variants, it will incorrectly fail to match mixed-case variants. (Edit: no, never mind, this does work correctly, it was just unobvious.)

Secondly, the list is wrong -- both in being too short and too long. It's too short because it doesn't consider the other extensions from the PATHEXT environment variable which will be automatically appended when trying to find a command to run from the command line. It's too long because the batch file extensions cannot be directly run by CreateProcess but instead have to be run by ShellExecute -- but the latter can "run" any registered document files as well. It should probably pick one of these intended definitions and stick with it rather than being inconsistent with either.

No longer necessary to link against Boost system

Judging from the build file and my very limited knowledge of Boost build files, the filesystem library still links against Boost system, even though it is now a header-only library since Boost 1.69.0.

Please remove the dependency on Boost system, so that client code that links against Boost filesystem does not have to pull in the stub Boost system library. I am not 100% sure on the correct way to do it, otherwise I would have filed a pull request.

lexically_relative and lexically_poximate give unexpected result

/home/user/documents/file.txt relative to /home/user/. should be documents/file.txt.

c++17 gives the expected result.

#include <filesystem>
#include <iostream>

namespace fs = std::filesystem;

#include <boost/filesystem.hpp>

namespace bfs = boost::filesystem;

int main() {
    std::string hs = "/home/user/.";
    std::string ps = "/home/user/documents/file.txt";

    {
        fs::path h = hs;
        fs::path p = ps;
        std::cout << "relative: " << p.lexically_relative(h) << '\n'
                  << "proximate: " << p.lexically_proximate(h) << '\n';
    }

    {
        bfs::path h = hs;
        bfs::path p = ps;
        std::cout << "relative: " << p.lexically_relative(h) << '\n'
                  << "proximate: " << p.lexically_relative(h) << '\n';
    }

    return 0;
}

outputs

relative: "documents/file.txt"
proximate: "documents/file.txt"
relative: "../documents/file.txt"
proximate: "../documents/file.txt"

boost::filesystem::path diverged from std::filesystem::path

Hi,

i realized that the behaviour of boost::filesystem::path and std::filesystem::path are strongly different. For example boost::filesystem::path adds and returns very often ".". Two examples:
1.) boost::fs::path("foo/bar/").filename() = "." != std::fs::path("foo/bar/").filename() = ""
2.) boost::fs::path("foo/bar/").lexically_normal() = "foo/bar/." != std::fs::path("foo/bar/").lexically_normal() = "foo/bar"

Why boost::filesystem has this obsession with '.'. Especially in case of a normalization function, we should pick the shortest path representation.

Another question is, why in case of boost::fs::path("/").filename() the method returns "/"? The code handles explicitly this case. (boost::fs::path("/foo/").filename() == ".") Why?

Another question is, why boost::fs::path("").lexically_normal() is violating half of the C++ standard rules? Is their a reason that in case of boost::fs::path("foo/") a '.' will be append after normalization? or that "./foo" is still "./foo" and not "foo"?

I ask this questions, because i am thinking to change the code. But i am afraid, that this behaviour of this code is needed somewhere else.

Best regards,
Trafo

crashed in "boost::filesystem::path::~path()", glibc free error

os: ubuntu 16.04
g++ --version:

g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

boost version: 1.66/1.69 both have this problem, I didn't test the others

I built boost myself, using g++ 5.4, and following build, and then facing a crash same as crash, glibc free error

A simple code slice can reproduce this problem.

#include <iostream>
#include <vector>
#include <boost/filesystem.hpp>
#include <boost/filesystem/operations.hpp>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/xml_parser.hpp>
#include <sstream>
#include <locale>
#include <codecvt>

using namespace std;

std::wstring utf8_to_wstring (const std::string& str)
{
	std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
	return myconv.from_bytes(str);
}
std::wstring utf8_to_wstring (std::string && str)
{
	std::wstring_convert<std::codecvt_utf8<wchar_t>> myconv;
	return myconv.from_bytes(str);
}

int main(int argc, char *argv[]) 
{
	wstring pwd = utf8_to_wstring(boost::filesystem::initial_path<boost::filesystem::path>().string());
	// get xml
	wstring config_xml = pwd + L"/config.xml";
	if(!boost::filesystem::exists(config_xml)) {  // crash here
	}
}
g++ -std=c++11 1.cc -lboost_filesystem -lboost_system
./a.out

root@be387067c6b0:~ # ./a.out
[1]    45287 segmentation fault  ./a.out

gdb...
>>> bt
#0  __GI___libc_free (mem=0xe) at malloc.c:2951
#1  0x0000000000404872 in boost::filesystem::path::~path() ()
#2  0x00000000004033a0 in main ()
>>>

Also, valgrind will give some errors

root@be387067c6b0:~ # valgrind ./a.out
==45357== Memcheck, a memory error detector
==45357== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==45357== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==45357== Command: ./a.out
==45357==
==45357== Syscall param stat(file_name) contains uninitialised byte(s)
==45357==    at 0x56E0BE5: _xstat (xstat.c:35)
==45357==    by 0x4E43AD9: boost::filesystem::detail::status(boost::filesystem::path const&, boost::system::error_code*) (in /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.58.0)
==45357==    by 0x404935: boost::filesystem::exists(boost::filesystem::path const&) (in /root/a.out)
==45357==    by 0x403393: main (in /root/a.out)
==45357==
==45357== Syscall param stat(file_name) points to unaddressable byte(s)
==45357==    at 0x56E0BE5: _xstat (xstat.c:35)
==45357==    by 0x4E43AD9: boost::filesystem::detail::status(boost::filesystem::path const&, boost::system::error_code*) (in /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.58.0)
==45357==    by 0x404935: boost::filesystem::exists(boost::filesystem::path const&) (in /root/a.out)
==45357==    by 0x403393: main (in /root/a.out)
==45357==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==45357==
==45357== Conditional jump or move depends on uninitialised value(s)
==45357==    at 0x4048E2: boost::filesystem::exists(boost::filesystem::file_status) (in /root/a.out)
==45357==    by 0x404941: boost::filesystem::exists(boost::filesystem::path const&) (in /root/a.out)
==45357==    by 0x403393: main (in /root/a.out)
==45357==
==45357== Invalid free() / delete / delete[] / realloc()
==45357==    at 0x4C2F24B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==45357==    by 0x404871: boost::filesystem::path::~path() (in /root/a.out)
==45357==    by 0x40339F: main (in /root/a.out)
==45357==  Address 0xe is not stack'd, malloc'd or (recently) free'd
==45357==
==45357==
==45357== HEAP SUMMARY:
==45357==     in use at exit: 72,735 bytes in 2 blocks
==45357==   total heap usage: 141 allocs, 140 frees, 89,620 bytes allocated
==45357==
==45357== LEAK SUMMARY:
==45357==    definitely lost: 31 bytes in 1 blocks
==45357==    indirectly lost: 0 bytes in 0 blocks
==45357==      possibly lost: 0 bytes in 0 blocks
==45357==    still reachable: 72,704 bytes in 1 blocks
==45357==         suppressed: 0 bytes in 0 blocks
==45357== Rerun with --leak-check=full to see details of leaked memory
==45357==
==45357== For counts of detected and suppressed errors, rerun with: -v
==45357== Use --track-origins=yes to see where uninitialised values come from
==45357== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)
root@be387067c6b0:~ #

I would like to know how to workaround this.
I'm looking forward to the reply, thank you!

Support char16_t and char32_t sequences

Contrary to what is stated in the docs, source code does no seem to have support for utf16 an utf32 char sequences. At least specialization of path_traits for is_pathable is missing and I assume for convert as well. I am not sure, what the internal decisions are, maybe its just postponed?

jump or move on uninitialized value on linux

Upgraded from boost 1.67 to 1.69, started getting valgrind errors related to jump or move on uninitialized value.

Triggered by this line:
if (entry->d_type == DT_UNKNOWN) // filesystem does not supply d_type value
https://github.com/boostorg/filesystem/blob/boost-1.69.0/src/operations.cpp#L2163

From what I can tell, this was introduced with this commit: 48b8d75

I believe what is happening (on my system), BOOST_FILESYSTEM_STATUS_CACHE is defined, and yet ::readdir_r() is not being called anymore after 48b8d75 , which leaves the d_type member of the dirent struct uninitialized.

I have used the following patch to address this.

index 53dcdb7..0749f91 100644
--- a/libs/filesystem/src/operations.cpp
+++ b/libs/filesystem/src/operations.cpp
@@ -2144,6 +2144,9 @@ namespace
       return errno;
     std::strcpy(entry->d_name, p->d_name);
     *result = entry;
+#   ifdef BOOST_FILESYSTEM_STATUS_CACHE
+    entry->d_type = DT_UNKNOWN;
+#   endif
     return 0;
   }

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.