Boost Process
Home Page: https://www.boost.org/libs/process
Boost.process is a library for comfortable management of processes, released with boost 1.64.0.
| Branches | Linux / Windows | Code coverage | Matrix |
|---|---|---|---|
| Develop: |  |
 |
 |
| Master: |  |
 |
 |
Latest developer documentation
This C++11 library is the current result of a long attempt to get a boost.process library, which is going on since 2006.
Distributed under the Boost Software License, Version 1.0.
This library requires boost 1.64 with which it is released.
I just got started with Boost.process, and noticed some things in the documentation that I hope can be addressed.
About sync io it says:
The pipe will cause a deadlock if you try to read after nm exited
Doesn't this suggest the example above it has a race condition? Also, doesn't the pipe just give EOF when reading from it after the process exited?
About async IO
Passing an instance of boost::asio::io_service to the launching function automatically cause it to wait asynchronously for the exit, so no call of wait is needed.
What is waiting for the exit automaticaly? This suggests the child constructor does, but I suspect it is the ios.run() function that actually does the waiting?
To make it even easier, you can use std::future for asynchronous operations (you will still need to pass a reference to a boost::asio::io_service) to the launching function, unless you use bp::system or bp::async_system
It seems the parenthesis are messed up here?
Also, perhaps it would be good to mention the fact that passing the ios is needed for bp:child, but not for bp:system (I suppose because it an create its own ios and run it, since bp:system is blocking).
boost::asio::io_service ios;
std::vector<char> buf;
bp::child c(bp::search_path("g++"), "main.cpp", bp::std_out > boost::asio::buffer(buf), ios);
ios.run();
int result = c.exit_code();
This example suggests that the output is read into buf. Since buf does not get a particular size, this suggests that it is automatically resized (it is a vector after all). However, digging into asio, shows that these buffers are really simple, and just contain a reference to an existing buffer and size, without any support for resizing. In fact, running the above example leaves buf empty, so all output is actually discarded.
Defining buf with an initial size allows capturing output up to that size, but I couldn't find a way to actually get the amount of data put into the buffer (since the vector size is unaffected by writing to the buffer).
I'm not quite sure how this is intended to work, but with the current documentation, I got quite the wrong expectation from how this buffer stuff works. The next example on that page shows how to get an auto-resizing buffer by passing a std::future<std::string> (which also works for vector<char> it seems), but I wonder why a future is actually needed for that: Can it not just write to an existing string instead?
MacOS Sierra 10.12
MacOS High Sierra 10.13.6
In file included from ~/boost/boost/process.hpp:24:
In file included from ~/boost/boost/process/async_system.hpp:22:
In file included from ~/boost/boost/process/child.hpp:21:
In file included from ~/boost/boost/process/detail/child_decl.hpp:30:
/boost/boost/process/detail/posix/wait_for_exit.hpp:60:7: error: expected unqualified-id
::sigemptyset(&sigset);
^
/usr/include/signal.h:125:26: note: expanded from macro 'sigemptyset'
#define sigemptyset(set) (*(set) = 0, 0)
^
What am I doing wrong? Any ideas? Thanks in advance.
and few same errors....
PS: I am using boost 1.68.0 built from sources.
In 1.65, there was a race when using boost::process::child::wait_for. Compiling the attached example with C++17 and gcc 7.3 on linux (although other version and clang are affected too) sometimes produces the expected output
$ ./a.out foo
Timeout
Finished
And sometimes immediately without waiting
$ ./a.out foo
Exited with code 252
Finished
which I assume is caused by a race somewhere in the fork and pipe setup part.
Interestingly, the bug is gone in 1.69, but I didn't find a ticket for it so I'm still reporting. Mostly because the fix got it wrong, too. Although there is no more race, wait_for now sleeps for the given time. Independent of whether the child exits or not. To reproduce, change the Sleep timespan to like 3s in the testcase.
In https://www.boost.org/doc/libs/1_68_0/doc/html/boost_process/tutorial.html#boost_process.tutorial.error_handling's Error section:
The original code:
std::error_code ec;
bp::system c("g++ main.cpp", ec);I think it should be:
std::error_code ec;
bp::system("g++ main.cpp", ec);Deadlock may occur if you execute boost.process code in a multithreaded program
std::string and std::vector require allocate memory. When other thread lock this, child process mybe deadlock.
Hi,
8d2bd87 (i.e. upgrading from boost 1.68 to boost 1.69) has introduced a regression where an fd for one of the child process pipes is not closed when there's nothing to read from it.
This leads to fd exhaustion after a while, e.g. with the following piece of code:
bp::ipstream stdout;
bp::child diffProcess(bp::search_path("diff"),
"--unified=1",
static_cast<std::string>(tmpFile1),
static_cast<std::string>(tmpFile2),
bp::std_out > stdout);
diffProcess.wait();
This patch is a workaround for the single case that we ran into, but there are probably more cases where the fd is not closed either. Please check :)
diff --git a/include/boost/process/detail/posix/executor.hpp b/include/boost/process/detail/posix/executor.hpp
index 1390a58..8b86ed1 100644
--- a/include/boost/process/detail/posix/executor.hpp
+++ b/include/boost/process/detail/posix/executor.hpp
@@ -282,7 +282,10 @@ class executor
set_error(std::error_code(err, std::system_category()), "Error read pipe");
}
if (count == 0)
+ {
+ ::close(source);
return ;
+ }
std::error_code ec(data[0], std::system_category());
std::string msg(data[1], ' ');
Thank You & BR,
Ingmar
【Problem Description】
In the Linux operating system, after the child process is created, the wait_for(const std::chrono::duration<Rep, Period>& rel_time) function is called, and the rel_time is set too large and the child process does not exit within the set time. wait_for() The function will cause the CPU usage to be too high to reach 100%.
Wait_until() implements the use of do{}while(); is there any optimization space?
【Code location】
process/include/boost/process/detail/posix/wait_for_exit.hpp
template< class Clock, class Duration >
inline bool wait_until(
const child_handle &p,
int & exit_code,
const std::chrono::time_point<Clock, Duration>& time_out,
std::error_code & ec) noexcept
{
pid_t ret;
int status;
bool timed_out;
do
{
ret = ::waitpid(p.pid, &status, WNOHANG);
if (ret == 0)
{
timed_out = Clock::now() >= time_out;
if (timed_out)
return false;
}
}
while ((ret == 0) ||
(((ret == -1) && errno == EINTR) ||
((ret != -1) && !WIFEXITED(status) && !WIFSIGNALED(status))));
if (ret == -1)
ec = boost::process::detail::get_last_error();
else
{
ec.clear();
exit_code = status;
}
return true;
}
【scenes to be used】
...
pusher_ = std::make_sharedboost::process::child(exec_cmd_/*,pubsher_handler(this)/);
const auto& wait_ret = pusher_->wait_for(std::chrono::seconds(1000));
if (wait_ret)
{
int exit_code = pusher_->exit_code();
}
...
Should this work, or is there a bug in what I'm doing? With the latest code on github, the following program hangs:
std::string data = "some data to send to cat";
std::future<std::string> result;
boost::asio::io_service ios;
bp::pipe p;
bp::spawn("cat", bp::std_in < bp::buffer(data), bp::std_out > p, ios);
bp::spawn("cat", bp::std_in < p, bp::std_out > result, ios);
// run blocks forever.
ios.run();
std::cout << result.get() << std::endl;
This seems like something that ought to work. Also, these two other programs that are only a little different work as expected, making it seem like there might be a bug in the interaction between multiple processes participating in a pipeline and the use of async input:
// use echo to make data instead of bp::buffer()
std::future<std::string> result;
boost::asio::io_service ios;
bp::pipe p1,p2;
bp::spawn("echo -n some data to send to cat", bp::std_out > p1);
bp::spawn("cat", bp::std_in < p1, bp::std_out > p2);
bp::spawn("cat", bp::std_in < p2, bp::std_out > result, ios);
ios.run();
std::cout << result.get() << std::endl;
// When pipes aren't used everything also works as expected.
std::string data = "some data to send to cat";
std::future<std::string> result;
boost::asio::io_service ios;
bp::spawn("cat", bp::std_in < bp::buffer(data), bp::std_out > result, ios);
ios.run();
std::cout << result.get() << std::endl;
System: Debian sid, x86_64
Since 0938103, the following program:
#include <iostream>
#include <string>
#include <vector>
#include <boost/process.hpp>
int main() {
std::vector<std::string> args{"/bin/echo", "hello from child"};
boost::process::ipstream out;
auto begin = std::chrono::high_resolution_clock::now();
bool timeout_flag = false;
boost::process::child child(args);
std::error_code ec;
if (!child.wait_for(std::chrono::seconds{10}, ec)) {
timeout_flag = true;
child.terminate(ec);
}
std::cout << "after wait_for" << std::endl;
}
ends up taking 10 seconds to complete, even though the spawned process exits pretty much immediately... except when running under strace or gdb. When I attach gdb to an already running process, I can see that it is blocking in sigtimedwait.
We tried to build and run process test with visual studio latest release on Windows. Sparring_partner.exe crashed when execute 2 tests(bind_stdin_stdout and exit_code), could you please help take a look at this? Thank you!
Reproduce steps:
Sparring_partner.exe crashed when execute steps "..\out\x86rel\boost\bin.v2\libs\process\test\exit_code.test\msvc-14.1\release\threading-multi\exit_code.exe" "..\out\x86rel\boost\bin.v2\libs\process\test\msvc-14.1\release\threading-multi\sparring_partner.exe" > "..\out\x86rel\boost\bin.v2\libs\process\test\exit_code.test\msvc-14.1\release\threading-multi\exit_code.output" 2>&1”
It also crashed when run ‘bind_stdin_stdout’ test.
std_out/std_err keywords overloads accept std::vector<char> by implicit conversion to boost::filesystem::path.
This is completely surprising, because
std::vector<char> output;
bp::child(command, bp::std_out > output);
compiles and will be expected to do the equivalent of
std::future<std::vector<char>> output;
boost::asio::io_service io;
bp::child(command, bp::std_out > output, io);
However, hard to diagnose errors arise.
A secondary issue might be that if the
dup2call fails (which it does) that doesn't stop de child process from running. This has lead to me not diagnosing this exact error for 3 weeks¹, until I finally noticed my oversight²
¹ due to this SO answer: https://stackoverflow.com/questions/49139875/boostprocess-with-standard-output-redirection-fails-randomly-with-ubuntu-16/49140495
² due to making the same error in this SO answer https://stackoverflow.com/questions/49462992/how-to-test-an-instance-counter-by-asynchronous-run-of-a-boost-childprocess
std::vector<char> converts to path implicitly:
std::vector<char> processOutput;
boost::filesystem::path p = processOutput;
We might want to rule out implicit conversions by going through a dispatch template.
When an ios is passed to bp:child, the io_ref_context created from that is, AFAICS, intended to handle SIGCHLD and wait for the child pid. However, it seems this does not happen currently, and no meaningful exit code is subsequently reported by exit_code(). E.g. running the following example, shows 127 instead of the expected 0:
boost::asio::io_service ios;
std::vector<char> buf;
bp::child c("/bin/true", bp::std_out > boost::asio::buffer(buf), ios);
ios.run();
int result = c.exit_code();
std::cout << result;
Investigation on my actual case (which used bp::system and std::future, but should be identical in terms of this problem) showed that the io_ref_context::on_success() is run, but decides that there are no async handlers (e.g. on_exit) and there is no need to handle SIGCHLD. However, it seems that it should always wait for sigchld() in order to capture the exit status.
Removing this check fixes the problem for me and seems like the right fix, but I'm not sure if there are any side effects. I can prepare a PR if that's helpful.
process/include/boost/process/detail/posix/io_context_ref.hpp
Lines 89 to 90 in 35fda5a
It seems that the Windows implementation has the same problem, though I can't test that:
process/include/boost/process/detail/windows/io_context_ref.hpp
Lines 91 to 94 in 35fda5a
I'd like to ask to support non-inheriting file descriptors in child processes on POSIX systems, in particular, on Linux.
Rationale:
This is an often desired feature, that is already available in Windows via inherit_handles flag. Although it can be implemented by user as on_exec_setup handler, I believe, it is common and tricky enough to benefit from being part of Boost.Process out of the box.
Note that I'm not considering marking all file descriptors with CLOEXEC flags in the user's code because that is not practical. Most third party libraries that open file descriptors, including the C/C++ standard libraries (e.g. std::FILE, iostreams) do not set CLOEXEC flags and there is no way to force all file descriptors to automatically have that flag process-wide.
Possible implementation:
Since POSIX does not define an API to not inherit file descriptors on exec, one suggestion is to do iterate over file descriptors, available in /proc/self/fd, in the child process after fork and before exec, and closing them. Note that file descriptors 0, 1 and 2 need to be skipped from closing, as these are stdin, stdout and stderr. These fds are also available as STDIN_FILENO, STDOUT_FILENO and STDERR_FILENO constants, which could be used instead of integers.
This functionality should be made optional, controllable by a flag (inherit_handles makes a precedent here).
This request comes from this Boost.Log ticket: boostorg/log#83.
I want to terminate my child git process
The VC++2017 x86 compiler gives me this warning.
\boost\process\detail\windows\wait_group.hpp(55): warning C4244: '-=': conversion from '_Rep' to 'int', possible loss of data
Please fix it, I like to be without warnings :)
We tried to build and run process test for Boost. It will pop out a box and hang. Could you please help take a look at this? Thanks!
Reproduce steps:
Expected result:
No issue
Actual result:
Pop out a box(sparring_partner.exe has stopped working) and hang.
The following program hangs if the flush() call is uncommented, but runs to completion when left commented out.
int main() {
ipstream in;
opstream out;
spawn("cat", std_in < out, std_out > in);
int val = 0;
out << 1 << endl;
in >> val;
cout << "val: "<< val << endl;
// uncomment this and program hangs.
//out.flush();
out << 2 << endl;
in >> val;
cout << "val: "<< val << endl;
}
On my system, when a program launched via system() with an error_code segfaults, it raise an exception.
To me, it seems to be abnormal.
What do you think ?
namespace bp = boost::process;
BOOST_AUTO_TEST_CASE(stackful, *boost::unit_test::timeout(1500))
{
using boost::unit_test::framework::master_test_suite;
bool did_something_else = false;
boost::asio::io_service ios;
boost::process::async_pipe StdOutPipe(ios);
auto stackful =
[&](boost::asio::yield_context yield_)
{
int ret =
bp::async_system(
ios, yield_,boost::process::windows::hide,
"C:/Program Files/OpenSSH/bin/ssh.exe localhost "
,boost::process::std_out > StdOutPipe
);
};
boost::asio::spawn(ios, stackful);
ios.post([&] {did_something_else = true; });
ios.run();
BOOST_CHECK(did_something_else);
}program always crash at
file：win_iocp_io_service.ipp ：
function：size_t win_iocp_io_service::do_one(bool block, boost::system::error_code& ec)
line 390 : op->Internal = reinterpret_cast<ulong_ptr_t>(&result_ec.category());
OS：win10
boost version：boost_1_69_0
c++ compliler：vc2017 x86
openssh version：setupssh-7.9p1-1
help me if I did anything wrong;
thank a lot！
Hi All,
In the latest code of boost::process the function dup2() is called as follows, e.g.:
if (::dup2(sink, STDERR_FILENO) == -1) ....
Could it be an issue if interrupt had happened? Some people recommend the following way, e.g.:
while ((dup2(filedes[1], STDOUT_FILENO) == -1) && (errno == EINTR)) {}
( http://www.microhowto.info/howto/capture_the_output_of_a_child_process_in_c.html )
Hello! Is there any chance that process isolation features or process resource monitoring features will appear in boost.process? Would it be useful for this library to have such things, or is it possible to contribute for them? Some existing solutions are nsjail or isolate, or may be some others, like docker.
I'm developing automated testing system for computer science students competitions which requires constrainted sandbox launch of student programs. All mentioned solutions are either not clossplatform (nsjail) or don't have everything that is needed (docker).
I'm just wondering whether it's intentional that Boost.Process doesn't support Visual Studio 2013 due to use of noexcept rather than BOOST_NOEXCEPT? (Someone raised this in trac for VS2012, but it also applies to VS2013.)
I'm having test compilation failures on Windows on the develop branch:
multi_ref1.cpp
..\..\..\boost/asio/io_service.hpp(27): error C2371: 'boost::asio::io_service': redefinition; different basic types
..\..\..\boost/process/detail/traits/async.hpp(15): note: see declaration of 'boost::asio::io_service'
..\..\..\boost/process/detail/windows/io_service_ref.hpp(133): error C2664: 'boost::asio::windows::object_handle::object_handle(const boost::asio::windows::object_handle &)': cannot convert argument 1 from 'boost::asio::io_service' to 'boost::asio::io_context &'
..\..\..\boost/process/detail/windows/io_service_ref.hpp(145): error C2039: 'native': is not a member of 'boost::asio::windows::object_handle'
..\..\..\boost/asio/windows/object_handle.hpp(61): note: see declaration of 'boost::asio::windows::object_handle'
..\..\..\boost/process/detail/windows/asio_fwd.hpp(42): error C2371: 'boost::asio::windows::object_handle': redefinition; different basic types
..\..\..\boost/asio/windows/object_handle.hpp(61): note: see declaration of 'boost::asio::windows::object_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(50): error C2371: 'boost::asio::windows::stream_handle': redefinition; different basic types
..\..\..\boost/process/detail/windows/asio_fwd.hpp(34): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(146): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(147): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(179): error C2664: 'boost::asio::windows::stream_handle::stream_handle(const boost::asio::windows::stream_handle &)': cannot convert argument 1 from 'boost::asio::io_service' to 'boost::asio::io_context &'
..\..\..\boost/process/detail/windows/async_pipe.hpp(185): error C2664: 'boost::asio::windows::stream_handle::stream_handle(const boost::asio::windows::stream_handle &)': cannot convert argument 1 from 'boost::asio::io_service' to 'boost::asio::io_context &'
..\..\..\boost/process/detail/windows/async_pipe.hpp(195): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(196): error C3536: 'source_in': cannot be used before it is initialized
..\..\..\boost/process/detail/windows/async_pipe.hpp(196): error C2446: '==': no conversion from 'const boost::winapi::HANDLE_' to 'int'
..\..\..\boost/process/detail/windows/async_pipe.hpp(196): note: There is no context in which this conversion is possible
..\..\..\boost/process/detail/windows/async_pipe.hpp(201): error C2664: 'BOOL DuplicateHandle(HANDLE,HANDLE,HANDLE,LPHANDLE,DWORD,BOOL,DWORD)': cannot convert argument 2 from 'int' to 'HANDLE'
..\..\..\boost/process/detail/windows/async_pipe.hpp(201): note: Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
..\..\..\boost/process/detail/windows/async_pipe.hpp(204): error C2440: '<function-style-cast>': cannot convert from 'initializer list' to 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(204): note: No constructor could take the source type, or constructor overload resolution was ambiguous
..\..\..\boost/process/detail/windows/async_pipe.hpp(211): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(212): error C3536: 'sink_in': cannot be used before it is initialized
..\..\..\boost/process/detail/windows/async_pipe.hpp(212): error C2446: '==': no conversion from 'const boost::winapi::HANDLE_' to 'int'
..\..\..\boost/process/detail/windows/async_pipe.hpp(212): note: There is no context in which this conversion is possible
..\..\..\boost/process/detail/windows/async_pipe.hpp(217): error C2664: 'BOOL DuplicateHandle(HANDLE,HANDLE,HANDLE,LPHANDLE,DWORD,BOOL,DWORD)': cannot convert argument 2 from 'int' to 'HANDLE'
..\..\..\boost/process/detail/windows/async_pipe.hpp(217): note: Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
..\..\..\boost/process/detail/windows/async_pipe.hpp(220): error C2440: '<function-style-cast>': cannot convert from 'initializer list' to 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(220): note: No constructor could take the source type, or constructor overload resolution was ambiguous
..\..\..\boost/process/detail/windows/async_pipe.hpp(236): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(237): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(239): error C3536: 'source_in': cannot be used before it is initialized
..\..\..\boost/process/detail/windows/async_pipe.hpp(239): error C2446: '==': no conversion from 'const boost::winapi::HANDLE_' to 'int'
..\..\..\boost/process/detail/windows/async_pipe.hpp(239): note: There is no context in which this conversion is possible
..\..\..\boost/process/detail/windows/async_pipe.hpp(244): error C2664: 'BOOL DuplicateHandle(HANDLE,HANDLE,HANDLE,LPHANDLE,DWORD,BOOL,DWORD)': cannot convert argument 2 from 'int' to 'HANDLE'
..\..\..\boost/process/detail/windows/async_pipe.hpp(244): note: Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
..\..\..\boost/process/detail/windows/async_pipe.hpp(247): error C3536: 'sink_in': cannot be used before it is initialized
..\..\..\boost/process/detail/windows/async_pipe.hpp(247): error C2446: '==': no conversion from 'const boost::winapi::HANDLE_' to 'int'
..\..\..\boost/process/detail/windows/async_pipe.hpp(247): note: There is no context in which this conversion is possible
..\..\..\boost/process/detail/windows/async_pipe.hpp(252): error C2664: 'BOOL DuplicateHandle(HANDLE,HANDLE,HANDLE,LPHANDLE,DWORD,BOOL,DWORD)': cannot convert argument 2 from 'int' to 'HANDLE'
..\..\..\boost/process/detail/windows/async_pipe.hpp(252): note: Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
..\..\..\boost/process/detail/windows/async_pipe.hpp(263): error C2664: 'boost::asio::windows::stream_handle::stream_handle(const boost::asio::windows::stream_handle &)': cannot convert argument 1 from 'boost::asio::io_service' to 'boost::asio::io_context &'
..\..\..\boost/process/detail/windows/async_pipe.hpp(302): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(305): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(307): error C2660: 'DuplicateHandle': function does not take 6 arguments
..\..\..\boost/process/detail/windows/async_pipe.hpp(310): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(313): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(315): error C2660: 'DuplicateHandle': function does not take 6 arguments
..\..\..\boost/process/detail/windows/async_pipe.hpp(328): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/detail/windows/async_pipe.hpp(331): error C2039: 'native': is not a member of 'boost::asio::windows::stream_handle'
..\..\..\boost/asio/windows/stream_handle.hpp(51): note: see declaration of 'boost::asio::windows::stream_handle'
..\..\..\boost/process/async_system.hpp(44): error C2143: syntax error: missing ';' before '<'
..\..\..\boost/process/async_system.hpp(93): note: see reference to class template instantiation 'boost::process::detail::async_system_handler<ExitHandler>' being compiled
..\..\..\boost/process/async_system.hpp(44): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
..\..\..\boost/process/async_system.hpp(45): error C2238: unexpected token(s) preceding ';'
call "C:\Users\Lastique\AppData\Local\Temp\b2_msvc_14.1_vcvars32_.cmd" >nul
cl /Zm800 -nologo @"..\..\..\bin.v2\libs\process\test\msvc-14.1\release\threadapi-win32\threading-multi\multi_ref1.obj.rsp"
It looks like at least initially the errors are caused by recent changes in Boost.Asio (e.g. boost::asio::io_service is no longer a class).
It would be very helpful to have an async_wait() method in child. In Unix it can be done asynchronous waiting for the SIGCHLD signal. I'm guessing there is something similar in Windows.
Currently it uses hardcoded limited set of them: "", ".exe", ".com", ".bat", but it is not correct.
Boost.Process depends on Boost.Filesystem, mainly because of boost::filesystem::path.
Now that Filesystem is in the standard (almost?) I think it would be nice to have the option for Boost.Process to compile against std's filesystem (#include<experimental/filesystem>) instead of #include<boost/filesystem.hpp>.
description:
I'm trying to write synchronous to a pipe.
boost::asio::io_context context;
boost::process::async_pipe pipe{context};
auto buffer = boost::asio::buffer(data, size);
boost::asio::write(pipe, buffer);
but this does not compile, with the error:
error: no matching function for call to ‘boost::process::detail::posix::async_pipe::write_some(boost::asio::const_buffers_1, boost::system::error_code&)’
However;
pipe.write_some(buffer);
DOES compile, but
boost::system::error_code error;
pipe.write_some(buffer, error);
not.
I'm not finding the write_some(buffer, error); method in the documentation:
http://www.boost.org/doc/libs/develop/doc/html/boost/process/async_pipe.html#idp32419296-bb
So it seems it doesn't comply to the SyncWriteStream requirements? http://www.boost.org/doc/libs/1_66_0/doc/html/boost_asio/reference/SyncWriteStream.html
Is that on purpose?
The following hpp files both use the same name "env" which can cause comiple warning/error.
//\boost\process\environment.hpp
explicit entry(string_type &&name, environment_t & env)
//\boost\process\env.hpp
constexpr boost::process::detail::env_ env{};
This probably has to do with the order I happen to include headers and compile my source files, after a refractoring of my program. I remember getting the same error about a year ago. Instead of trying to figure out a working ordering, it was easier this time to simply rename all the conflicting argument names in environment.hpp from "env" to "env_". If the argument name doesn't matter, you might want to do something similar, in case this happens to other people also.
details:
\boost_1_67_0\boost\process\environment.hpp(97): error C2220: warning treated as error - no 'object' file generated
\boost_1_67_0\boost\process\environment.hpp(97): warning C4459: declaration of 'env' hides global declaration
\boost_1_67_0\boost\process\env.hpp(497): note: see declaration of 'boost::process::env'
./boost/process/detail/posix/sigchld_service.hpp: In member function ‘void boost::process::detail::posix::sigchld_service::_handle_signal(const boost::system::error_code&)’:
./boost/process/detail/posix/sigchld_service.hpp:91:29: error: no matching function for call to ‘std::vector<std::pair<int, std::function<void(int, std::error_code)> > >::erase(__gnu_cxx::__normal_iterator<const std::pair<int, std::function<void(int, std::error_code)> >*, std::vector<std::pair<int, std::function<void(int, std::error_code)> > > >&)’
_receivers.erase(itr);
^
See https://travis-ci.org/boostorg/boost/jobs/291234390
Apologies for enabling issues.
when building against musl libc. that assertion failed.
because musl contain correction for mips.
details here:
http://git.musl-libc.org/cgit/musl/commit/include/sys/wait.h?id=41c632824c08ac2c9eea43b30d1b3515dd910df6
so that assertion should be fixed.
the status code is opaque implementation detail, isn't it? that's why we need those test macro, so this is not reliable and portable at the beginning, iiuc.
As i can not find the supported platforms and compilers in the documentation, i am not sure if it is the reason why i cant compile the intro.cpp from the examples folder or if it is a bug in boost::process.
My platform:
I get the following error:
1> intro.cpp
1>d:\entwicklung\3rdparty\boost\1.65.1\boost\process\detail\config.hpp(65): error C3646: "noexcept": Unbekannter Überschreibungsspezifizierer
1>d:\entwicklung\3rdparty\boost\1.65.1\boost\process\detail\config.hpp(72): error C2660: 'boost::process::process_error::process_error': Funktion akzeptiert keine 2 Argumente
1>d:\entwicklung\3rdparty\boost\1.65.1\boost\process\detail\config.hpp(77): error C2440: '<function-style-cast>': 'std::error_code' kann nicht in 'boost::process::process_error' konvertiert werden
I am on
$ git log -n1
commit 4075371 (HEAD, tag: boost-1.65.1, tag: boost-1.65.0, tag: boost-1.64.0)
Merge: 477a60e d5d2c7b
Author: Klemens Morgenstern [email protected]
Date: Wed Apr 5 11:32:27 2017 +0200
Merge pull request #3 from boostorg/develop
extension adding fix for windows
Documentation says
The following expressions are valid, with `value` being either a C-String or
a `std::basic_string` with `char` or `wchar_t` or a `boost::filesystem::path`.
However there is no operator()(const boost::filesystem::path &) overload. exe = boost::filesystem::path() fails with no viable overloaded '=' error. By looking on similar code in start_dir I think there should be explicit overload for boost::filesystem::path.
#define BOOST_ASIO_NO_DEPRECATED 1
#include <boost/process.hpp>
int main() {}
Error:
boost/process/detail/windows/async_pipe.hpp:102:39: error: no member named 'get_io_context' in 'boost::asio::windows::stream_handle'
_sink = handle_type(_sink.get_io_context());
I have compiled boost 1.68 with VS2013 and when I try to use it I have the following errors:
in include\boost\process\detail\config.hpp L64 there is the c++ noexcept keyword but VS2013 do not support it. Why it doesn't use the BOOST_NOEXCEPT define instead ?
include\boost/process/detail/config.hpp(72): error C2660: 'boost::process::process_error::process_error' : function does not take 2 arguments
After building Boost 1.67.0 and running the test suite, I am running into an intermittent test failure for the multithreaded_async_pipe test case from test/async_pipe.cpp, where a SIGABRT is raised.
For reference, I am building Boost with the mock tool, using the spec file from https://github.com/clearlinux-pkgs/boost/blob/cbefce45d9e125337d0545ca4957c761d7fd58ec/boost.spec, modified for 1.67.0.
When the test fails, the output from the async_pipe test binary is:
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
what(): assign: Bad file descriptor
Running 3 test cases...
unknown location(0): fatal error: in "multithreaded_async_pipe": signal: SIGABRT (application abort requested)
../libs/process/test/async_pipe.cpp(67): last checkpoint: "multithreaded_async_pipe" test entry
*** 1 failure is detected in the test module "Master Test Suite"
EXIT STATUS: 201
Running the test binary through GDB, I captured the backtraces for the crashing thread and the other threads. If you would like me to post the backtraces from the other threads, let me know.
Backtrace from crashing thread:
#0 0x00007ffff748f247 in raise () from /usr/lib64/libc.so.6
#1 0x00007ffff7490ba1 in abort () from /usr/lib64/libc.so.6
#2 0x00007ffff78749fd in __gnu_cxx::__verbose_terminate_handler ()
#3 0x00007ffff78726d6 in __cxxabiv1::__terminate (handler=<optimized out>)
at ../../../../gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:47
#4 0x00007ffff7872721 in std::terminate ()
at ../../../../gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:57
#5 0x00007ffff78a7b1e in std::execute_native_thread_routine (__p=0x749420)
at ../../../../../gcc-7.3.0/libstdc++-v3/src/c++11/thread.cc:91
#6 0x00007ffff7b7a66a in ?? () from /usr/lib64/libpthread.so.0
#7 0x00007ffff756fb3f in clone () from /usr/lib64/libc.so.6
If you require further information about the build environment or other details, I can provide them.
child::exit_code() evaluates WIFEXITED and WIFSIGNALED. Depending on that child::exit_code() returns WEXITSTATUS or WTERMSIG (see boost::process::detail::posix::eval_exit_status()).
For code calling child::exit_code() it is not possible to determine if the returned value of 9 means that the forked process returned normally with exit-code 9 or the forked process was killed by SIGKILL (9 on Linux).
Because of https://github.com/boostorg/process/blob/develop/include/boost/process/detail/posix/executor.hpp#L35 boost::process fails to build with uClibc.
While at some point it was added (https://git.uclibc.org/uClibc/commit/?id=0eb30761a26c46aaf555464114851202ae9c27bd), that was after the latest release (0.9.33.2).
Compilation with musl breaks on either (or both) line 21 and 22 in detail/posix/is_running.hpp:
// Use the "stopped" state (WIFSTOPPED) to indicate "not terminated".
// This bit arrangement of status codes is not guaranteed by POSIX, but (according to comments in
// the glibc <bits/waitstatus.h> header) is the same across systems in practice.
constexpr int still_active = 0x017f;
static_assert(!WIFEXITED(still_active), "Expected still_active to not indicate WIFEXITED");
static_assert(!WIFSIGNALED(still_active), "Expected still_active to not indicate WIFSIGNALED");Excerpted from musl's mailing list:
What they're testing for, based on the name still_active, seems to be
that they have a fake status value they can use that doesn't evaluate
true for either of these WIF* macros. And their assertion has
uncovered a bug in boost, because 127 is a valid signal number (on
mips).
See https://ci.appveyor.com/project/boostorg/boost/build/1.0.4226-develop
C:\projects\boost\boost/process/async.hpp(48): fatal error C1083: Cannot open include file: 'boost/process/detail/windows/io_context_ref.hpp': No such file or directory
In case if child received SIGABRT WIFEXITED() may be false and WEXITSTATUS() will result in 0.
proposed change:
// from
inline int eval_exit_status(int code)
{
return WEXITSTATUS(code);
}
// to
inline int eval_exit_status(int code)
{
if (WIFEXITED(code))
{
return WEXITSTATUS(code);
}
else if (WIFSIGNALED(code))
{
return WTERMSIG(code);
}
else if (WIFSTOPPED(code))
{
return WSTOPSIG(code);
}
else
{
return code;
}
}
Hey There. Getting a ' pp hides previous declaration' compiler error in:
for (const boost::filesystem::path & pp : path)
{
auto p = pp / filename;
for (boost::filesystem::path ext : extensions)
{
boost::filesystem::path pp = p;
At the bottom off file: boost\process\detail\windows\search_path.hpp
Is this supposed to be a differently named variable, or is that line supposed to set the value of the previously defined?
Cheers
When including boost process header, warning pop up (-Wpedantic)
There are semicolons present after the method definition (definition, not declaration)
In file included from /home/ldco/boost/include/boost/process/detail/on_exit.hpp:12:0,
from /home/ldco/boost/include/boost/process/async.hpp:33,
from /home/ldco/boost/include/boost/process.hpp:23,
from /home/ldco/Projects/Teaching/Teaching/Boost/Process/Process1/src/main.cpp:7:
/home/ldco/boost/include/boost/process/detail/posix/on_exit.hpp:30:6: warning: extra ‘;’ [-Wpedantic]
};
^
In file included from /home/ldco/boost/include/boost/process/async.hpp:42:0,
from /home/ldco/boost/include/boost/process.hpp:23,
from /home/ldco/Projects/Teaching/Teaching/Boost/Process/Process1/src/main.cpp:7:
/home/ldco/boost/include/boost/process/detail/posix/io_context_ref.hpp:67:6: warning: extra ‘;’ [-Wpedantic]
};
^
I have fixed this in ; #71
With the recent addition of the WIFSIGNALED macro use the is_running function is consistently returning false do to differences in the behavior of this macro on Solaris.
First off the static assert fails. The code 0177 results in WIFSIGNALED returning true rather than false. I commented that out to see if things would run.
This is where it gets strange. After the successful execution a child the status code is 0177, which when checked in is_running results in a WIFSIGNALED being true and the function returning false. A check of the child's exit code, using WTERMSIG since WIFSIGNALED is true, results in the value 0177 (127), which appears to not be a valid signal code on Solaris.
Looking at wait.h on Solaris it defines these macros quite differently than say MacOS.
On Solaris:
#define WIFEXITED(stat) ((int)((stat)&0xFF) == 0)
#define WIFSIGNALED(stat) ((int)((stat)&0xFF) > 0 && \
(int)((stat)&0xFF00) == 0)
#define WIFSTOPPED(stat) ((int)((stat)&0xFF) == 0177 && \
(int)((stat)&0xFF00) != 0)
#define WEXITSTATUS(stat) ((int)(((stat)>>8)&0xFF))
#define WTERMSIG(stat) ((int)((stat)&0x7F))
#define WSTOPSIG(stat) ((int)(((stat)>>8)&0xFF))
From this we can clearly see why we get the values we do on Solaris when code is 0177, since (int)((stat)&0xFF) > 0 is true and so is (int)((stat)&0xFF00) == 0.
On MacOS:
#define _WSTATUS(x) (_W_INT(x) & 0177)
#define _WSTOPPED 0177
#define WSTOPSIG(x) (_W_INT(x) >> 8)
#define WIFCONTINUED(x) (_WSTATUS(x) == _WSTOPPED && WSTOPSIG(x) == 0x13)
#define WIFSTOPPED(x) (_WSTATUS(x) == _WSTOPPED && WSTOPSIG(x) != 0x13)
#define WIFEXITED(x) (_WSTATUS(x) == 0)
#define WIFSIGNALED(x) (_WSTATUS(x) != _WSTOPPED && _WSTATUS(x) != 0)
#define WTERMSIG(x) (_WSTATUS(x))
Clearly we can see that the code 0177 case is handled in the WIFSIGNALED and would result in the expected false.
It almost feels like the Solaris implementation is missing the stopped case in it's &signaled check.
Not being an expert in this particular area I hesitate to make any changes other then to revert back to prior implementations that only used the WIFEXITED macro but would like to have an answer for how to upgrade in the future.
Any thoughts on how to proceed?
https://github.com/boostorg/process/blob/develop/include/boost/process/io.hpp#L241 is very handy, but it would be even more handy if there would be an overload supporting a DynamicBuffer (https://www.boost.org/doc/libs/1_66_0/doc/html/boost_asio/reference/DynamicBuffer.html) as argument.
std::vector<std::string> read_outline(std::string & file)
{
bp::ipstream is; //reading pipe-stream
bp::child c(bp::search_path("nm"), file, bp::std_out > is);
std::vector<std::string> data;
std::string line;
while (c.running() && std::getline(is, line) && !line.empty())
data.push_back(line);
c.wait();
return data;
}
What if nm exit after c.runing() return true?
I faced with the following warnings when compiling with clang-6.0 -Wall -Wextra:
include/boost/process/detail/posix/sigchld_service.hpp:98:31: warning:
lambda capture 'ec' is not used [-Wunused-lambda-capture]
_strand.post([ec]{});
boost/process/detail/posix/async_pipe.hpp:235:54: warning: unused
parameter 'p' [-Wunused-parameter]
async_pipe& async_pipe::operator=(const async_pipe & p)
boost/process/detail/posix/async_in.hpp:63:56: warning: unused
parameter 'ec' [-Wunused-parameter]
[pipe](const boost::system::error_code&ec, std::size_t size){});
include/boost/process/detail/posix/async_in.hpp:63:72: warning: unused
parameter 'size' [-Wunused-parameter]
[pipe](const boost::system::error_code&ec, std::size_t size){});
include/boost/process/detail/posix/async_in.hpp:42:38: warning: unused
parameter 'exec' [-Wunused-parameter]
inline void on_success(Executor &exec)
include/boost/process/detail/posix/async_out.hpp:64:70: warning: unused
parameter 'size' [-Wunused-parameter]
[pipe](const boost::system::error_code&, std::size_t size){});
include/boost/process/detail/posix/async_out.hpp:60:38: warning: unused
parameter 'exec' [-Wunused-parameter]
inline void on_success(Executor &exec)
include/boost/process/detail/posix/async_out.hpp:121:90: warning: unused
parameter 'size' [-Wunused-parameter]
[pipe, buffer, promise](const boost::system::error_code& ec, std::size_t size)
include/boost/process/detail/posix/async_out.hpp:113:38: warning: unused
parameter 'exec' [-Wunused-parameter]
inline void on_success(Executor &exec)
include/boost/process/async_system.hpp:85:27: warning: unused parameter
'exit_code' [-Wunused-parameter]
return [](int exit_code, const std::error_code & ec){};
include/boost/process/async_system.hpp:85:27: warning: unused parameter
'ec' [-Wunused-parameter]
return [](int exit_code, const std::error_code & ec){};
include/boost/process/detail/posix/group_handle.hpp:62:47: warning: unused
parameter 'ec' [-Wunused-parameter]
bool has(handle_t proc, std::error_code & ec) noexcept
1>c:\api\boost_1_67_0_vc1914\boost\process\detail\windows\search_path.hpp(63): error C2220: warning treated as error - no 'object' file generated
1>c:\api\boost_1_67_0_vc1914\boost\process\detail\windows\search_path.hpp(63): warning C4456: declaration of 'pp' hides previous local declaration
_MSC_VER: 1914
#define BOOST_NO_ANSI_APIS 1
#include <boost/process.hpp>
int main() {}
Fails to compile with:
boost/process/detail/windows/locale.hpp(44): error C3861: 'AreFileApisANSI': identifier not found
I upgraded from "boost_1_65_1_vc141" to "boost_1_67_0_vc1914" and my program started crashing in release mode. I never saw this in the old version. It took me a day or two, but I narrowed down the error to this toy example. It crashes the same.
#include <boost/process.hpp>
int main()
{
boost::process::group group_; //In Debug mode it crashes here in boost\process\detail\windows\job_workaround.hpp : query_information_job_object : "Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention."
} //In Release mode it crashes here: "Unhandled exception at 0x00A74ACB in crash_test.exe: Stack cookie instrumentation code detected a stack-based buffer overrun."
//settings&versions:
//boost version: boost_1_67_0
//vs version: Microsoft Visual C++ 2017, _MSC_VER 1914
//Default Debug mode command line settings: "/permissive- /GS /analyze- /W3 /Zc:wchar_t /I"C:\API\boost_1_67_0_vc1914" /ZI /Gm- /Od /sdl /Fd"Debug\vc141.pdb" /Zc:inline /fp:precise /D "_SCL_SECURE_NO_WARNINGS" /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_UNICODE" /D "UNICODE" /errorReport:prompt /WX- /Zc:forScope /RTC1 /Gd /Oy- /MDd /FC /Fa"Debug\" /EHsc /nologo /Fo"Debug\" /Fp"Debug\crash_test.pch" /diagnostics:classic "
//Default Release mode command line settings: "/permissive- /GS /GL /analyze- /W3 /Gy /Zc:wchar_t /I"C:\API\boost_1_67_0_vc1914" /Zi /Gm- /O2 /sdl /Fd"Release\vc141.pdb" /Zc:inline /fp:precise /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_UNICODE" /D "UNICODE" /errorReport:prompt /WX- /Zc:forScope /Gd /Oy- /Oi /MD /FC /Fa"Release\" /EHsc /nologo /Fo"Release\" /Fp"Release\crash_test.pch" /diagnostics:classic "
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.