taocpp/taopq

C++ client library for PostgreSQL

taopq's Issues

Iterative bind (for an insert with dynamic number of columns)

I have a use case where I'd like to store tables with a dynamic number of columns. The use case is from analysis measurements, where users can configure the actually computed measurements with a simple on/off selection. I receive the measurements as a vector of vectors which represent the columns.

It seems not directly possible to use the parameter binding with such dynamic data, or is it? In my naiive understanding it seems I could loop over the columns and bind value by value, iteratively. But taopq seems not to support this.

Am I even on the right track? How to work with a dynamic set of columns?

Clean cmake failing (visual studio 2017)

As of the latest commit in master (haven't tried earlier ones)

CMake Error at W:/Program Files (x86)/Microsoft Visual Studio/2017/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.12/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find PostgreSQL (missing: PostgreSQL_LIBRARY
PostgreSQL_INCLUDE_DIR PostgreSQL_TYPE_INCLUDE_DIR) W:/Program Files (x86)/Microsoft Visual Studio/2017/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.12/Modules/FindPackageHandleStandardArgs.cmake 137

Took a look in taopq-master\taopq-master\cmake\taopqConfig.cmake.in

and it seems like it specifically needs PostgreSQL 9.6.9. Is that the case? I've got postgresql 12 installed.

Either way, figured I'd try the conan route, and the package isn't found in the remote.

taocpp-taopq/20200215: Not found in local cache, looking in remotes...
taocpp-taopq/20200215: Trying with 'conan-center'...
ERROR: Unable to find 'taocpp-taopq/20200215' in remotes

I've got the libpq libraries built, but not sure how to proceed,

Building with Visual Studio?

Dear developers, I very much cherish the moderns C++ postgreSQL interface. However it seems currently a bit hard to user with MSVC. Are you interested in supporting this platform? From my experience it can be slightly tedious work, but often it improves the overall code quality.
If this is relevant for you I could try to make an Azure DevOps CI setup that provides continuous builds for your Github repository?

Unable to use set_notification_handler

Hello,

I am trying to set a notification handler to listen for events. I have set up multiple channels and I have a notification handler who's signature is:

(XXXXX::XXXX is namespace blanked out)
void XXXXX::XXXX::Connection_Notification(const tao::pq::notification &message)

When I try to execute:

conn->set_notification_handler(Connection_Notification);

I am getting an error , "no instance of overload function tao::pq::connection::set_notification_handler matches the argument list (void(const tao::pq::notification &message))"

Any ideas what is wrong with the function signature I am providing? Thanks in advance.

Building without Postgres installed?

Hi,

Pardon my noobness, I'm trying to write a C++ postgres pipeline using taopq (the Python one is way too slow).

I saw that PostgreSQL itself is a dependency in CMakeLists.txt. However, it seems that libpq itself is much smaller -
https://github.com/postgres/postgres/tree/master/src/backend/libpq

I built it by the instructions here, to receive a lipq.so.5.12

I also saw in the makefile a line that includes pg_config. Does this mean I still need postgres installed on the machine on which I'm using taopq? Is it somehow possible to avoid it, if I already have libpq.so?

Thanks,
Eli

BYTEA columns results wrong with tablewriter

I think that BYTEA colums are not written correctly to the database when using a table writer.

I created a small example, mostly copied from the bytea tests:

    auto conn = create_db_connection()
    conn->execute("DROP TABLE IF EXISTS test_table;");
    conn->execute("CREATE TABLE test_table (  buffer_bytes BYTEA NOT NULL);");

    const unsigned char bdata[] = { 'v', 255, 0, 'a', 1, 'b', 0 }; 
    
    conn->execute( "INSERT INTO test_table VALUES ( $1 )", tao::pq::to_binary_view(bdata) );

    tao::pq::table_writer tw(conn->direct(), "COPY test_table ( buffer_bytes ) FROM STDIN");
    tw.insert(tao::pq::to_binary_view(bdata));   
    tw.commit();
    
    const auto result_0 = conn->execute( "SELECT * FROM test_table" )[ 0 ][ 0 ].as< std::basic_string<unsigned char > >();
    const auto result_1 = conn->execute( "SELECT * FROM test_table" )[ 1 ][ 0 ].as< std::basic_string<unsigned char > >();
    fmt::print("result_0 = {}\n", fmt::join(result_0, ", ")); // result_0 = 118, 255, 0, 97, 1, 98, 0
    fmt::print("result_1 = {}\n", fmt::join(result_1, ", ")); // result_1 = 125, 48, 48, 48, 48, 54, 55, 48, 49, 54, 56, 48, 48

I I write { 'v', 255, 0, 'a', 1, 'b', 0 } to the db and using an INSERT statement I get 118, 255, 0, 97, 1, 98, 0 which is correct (?).
If i write the same data using a table writer i get 125, 48, 48, 48, 48, 54, 55, 48, 49, 54, 56, 48, 48 .

I tried changing the column type to TEXT and insert two strings instead, this seems work.

Am I doing something wrong?

I'm using a621cbb

Compile error with MSYS2 and GCC

Compile command:

git clone https://github.com/taocpp/postgres.git taocpp_pg
cd taocpp_pg
mkdir build&&cd build
cmake ../ -G "MSYS Makefiles"
make

Output:

Scanning dependencies of target taocpp-postgres
[  3%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/connection.cpp.obj
[  6%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/connection_pool.cpp.obj
[ 10%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/field.cpp.obj
[ 13%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/result.cpp.obj
[ 17%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/result_traits.cpp.obj
[ 20%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/row.cpp.obj
[ 24%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/table_writer.cpp.obj
[ 27%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/postgres/transaction.cpp.obj
[ 31%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/utility/demangle.cpp.obj
[ 34%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/utility/getenv.cpp.obj
[ 37%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/utility/printf.cpp.obj
[ 41%] Building CXX object CMakeFiles/taocpp-postgres.dir/src/lib/utility/strtox.cpp.obj
[ 44%] Linking CXX static library libtaocpp-postgres.a
[ 44%] Built target taocpp-postgres
Scanning dependencies of target transaction
[ 48%] Building CXX object src/test/postgres/CMakeFiles/transaction.dir/transaction.cpp.obj
[ 51%] Linking CXX executable transaction.exe
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x18): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIavE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x18): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x28): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIhvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x28): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x38): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIsvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x38): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x48): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsItvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x48): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x58): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIivE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x58): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x68): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIjvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x68): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x78): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIlvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x78): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x88): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsImvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x88): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0x98): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIxvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0x98): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0xa8): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIyvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0xa8): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0xb8): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIfvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0xb8): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0xc8): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIdvE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0xc8): first defined here
../../../libtaocpp-postgres.a(transaction.cpp.obj):transaction.cpp:(.rdata+0xd8): multiple definition of `.weak._ZN3tao8postgres16parameter_traitsIevE6formatE._ZNSt9exceptionC2Ev'
../../../libtaocpp-postgres.a(connection.cpp.obj):connection.cpp:(.rdata+0xd8): first defined here
collect2.exe: error: ld returned 1 exit status
make[2]: *** [src/test/postgres/CMakeFiles/transaction.dir/build.make:98：src/test/postgres/transaction.exe] 错误 1
make[1]: *** [CMakeFiles/Makefile2:137：src/test/postgres/CMakeFiles/transaction.dir/all] 错误 2
make: *** [Makefile:141：all] 错误 2

It seems that the static variable "format" at there is not defined in the cpp file.

Support for binary storage of numerical values?

I'm under the impression that taopq does currently not support binary storage of numerical values, is that correct? This less commonly known mode of libpq allows to store and retrieve numerical values directly as a series of bytes (albeit in network storage order). So for example, an int2 can be stored as the two bytes (low byte, high byte) of the short value. The main benefits are:

• a higher numerical precision due to the lack of limited string representation of floating point values,
• faster storage due to the lack of string conversion, and
• lower storage and bandwidth requirement due to the lack of string representation.

Could this be a possible extension for taopq?

[GCC][Release] Compilation failed because of assert statement

**Compiler : gcc (GCC) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.

Errors :**
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = char; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:99:35: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = float; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:128:67: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = double; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:156:81: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = long double; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:231:82: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = signed char; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:67:14: required from ‘typename std::enable_if<std::is_signed<_Tp>::value>::type check(const string&) [with T = signed char; typename std::enable_if<std::is_signed<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:113:36: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = unsigned char; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:81:14: required from ‘typename std::enable_if<std::is_unsigned<_Tp>::value>::type check(const string&) [with T = unsigned char; typename std::enable_if<std::is_unsigned<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:114:38: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = short int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:67:14: required from ‘typename std::enable_if<std::is_signed<_Tp>::value>::type check(const string&) [with T = short int; typename std::enable_if<std::is_signed<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:115:30: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = short unsigned int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:81:14: required from ‘typename std::enable_if<std::is_unsigned<_Tp>::value>::type check(const string&) [with T = short unsigned int; typename std::enable_if<std::is_unsigned<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:116:39: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:67:14: required from ‘typename std::enable_if<std::is_signed<_Tp>::value>::type check(const string&) [with T = int; typename std::enable_if<std::is_signed<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:117:28: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = unsigned int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:81:14: required from ‘typename std::enable_if<std::is_unsigned<_Tp>::value>::type check(const string&) [with T = unsigned int; typename std::enable_if<std::is_unsigned<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:118:33: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = long int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:67:14: required from ‘typename std::enable_if<std::is_signed<_Tp>::value>::type check(const string&) [with T = long int; typename std::enable_if<std::is_signed<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:119:29: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = long unsigned int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:81:14: required from ‘typename std::enable_if<std::is_unsigned<_Tp>::value>::type check(const string&) [with T = long unsigned int; typename std::enable_if<std::is_unsigned<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:120:38: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = long long int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:67:14: required from ‘typename std::enable_if<std::is_signed<_Tp>::value>::type check(const string&) [with T = long long int; typename std::enable_if<std::is_signed<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:121:34: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]
/postgres/src/test/postgres/basic_datatypes.cpp: In instantiation of ‘void check(const string&, const T&) [with T = long long unsigned int; std::__cxx11::string = std::__cxx11::basic_string]’:
/postgres/src/test/postgres/basic_datatypes.cpp:81:14: required from ‘typename std::enable_if<std::is_unsigned<_Tp>::value>::type check(const string&) [with T = long long unsigned int; typename std::enable_if<std::is_unsigned<_Tp>::value>::type = void; std::__cxx11::string = std::__cxx11::basic_string]’
/postgres/src/test/postgres/basic_datatypes.cpp:122:43: required from here
/postgres/src/test/postgres/basic_datatypes.cpp:58:18: error: unused variable ‘v’ [-Werror=unused-variable]

I am not a big fan of assert statement because it provide two different projects based on compilation settings.

question: calling execute on a threadpool

can I call execute on a threadpool?

ex.

const auto conn = tao::pq::connection_pool::create("dbname=postgres user=postgres password=postgres");
conn->execute("CREATE TABLE IF NOT EXISTS fcu_data (VALUE TEXT);");

instead of
conn->connection()->execute("CREATE TABLE IF NOT EXISTS fcu_data (VALUE TEXT);");
?

Broken Windows build on CI

For documentation proposal.

Related to actions/runner-images#4243

The current approach is installing PostgreSQL 13 by choco, and start the service via PowerShell.

For some reason, it fails to start. I tried to reproduce the same error, but it works for me and I don't have a second Windows machine to validate it.

Another possible approaches:

• Use another package manager (Conan, vcpkg) and generate all needed files (cmake)
• Build postgresql from source and store in some place (bin and libs)
• More suggestions?

Aggregate Support optimization

how about using pfr to implement this feature

What are the template arguments for transaction?

I've just updated to latest master branch, and found that the following code does not compile anymore:

std::shared_ptr<tao::pq::transaction> = conn->transaction();

It seems that the transaction class now uses template arguments. I've checked the docs, but its not explicitly mentioned, and in the code I find now mostly this kind of usage: const auto tr = conn->transaction();.

Could you kindly help what the template arguments for transaction are?

Better ways to detect constraint violations, etc

Currently you just get a runtime error. Compare this to the exceptions provided by libpqxx: https://libpqxx.readthedocs.io/en/7.6.0/a00247.html.

I will try to do this myself, but I don't know much about PostgreSQL so it may be difficult.

akrzemi1/optional.hpp clashes with experimental/optional

Hello Daniel,

it looks that the tao/postgres header clashes with experimental/optional.

// t.cc
#include <experimental/optional>
#include <tao/postgres.hpp>
int main() {}

using

g++ -std=c++14 -Itao/postgres/include t.cc

results in:

In file included from tao/postgres/include/tao/optional/optional.hpp:14:0,
             from tao/postgres/include/tao/postgres/result.hpp:19,
             from tao/postgres/include/tao/postgres/connection.hpp:12,
             from tao/postgres/include/tao/postgres.hpp:9,
             from t.cc:3:
tao/postgres/include/tao/optional/akrzemi1/optional.hpp:183:13: error: ‘optional’ is not a class template
   class optional< T& >;
         ^~~~~~~~

and a lot more....

The library itself compiles fine in this environment.

Using g++ 6.3.0 with libstdc++ 6.3 on Debian 9.1.

Any fix or workaround available?

Support for LISTEN/NOTIFY ?

Are there any plans to support PostgreSQL LISTEN/NOTIFY ?

fatal error: unresolved external symbol during connection create

I am new to c++
recently downloaded taopq for database connection to an existing cpp code
I am using windows and vs2019
so, I have built taopq using cmakelist in vs2019
then included the tao/pq.hpp in my code
from the introduction part of github page (https://github.com/taocpp/taopq), I tried to make a connection in my code
const auto conn = tao::pq::connection::create("host=, port=, dbname=, user:");
whenever I try to build this, get the following error
error LNK2001: unresolved external symbol "public: static class std::shared_ptr __cdecl tao::pq::connection::create(class std::basic_string<char,struct std::char_traits,class std::allocator > const &)" (?create@connection@pq@tao@@sa?AV?$shared_ptr@Vconnection@pq@tao@@@std@@aebv?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@5@@z)
1>D:\Lahari\work\EQ\EQ C++ db\x64\Release\EQ.exe : fatal error LNK1120: 1 unresolved externals
1>Done building project "EQ.vcxproj" -- FAILED.
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========
any help??

Unpacking Rows into structs

I was just looking at some code using the official postgres C++ library, and we have code like:

using PersonTuple = std::tuple<std::string, std::string, int>;
struct Person {
  std::string firstName;
  std::string lastName;
  int dob;
};

Person readFromDB() {
  PersonTuple pt = readRowFromDB();
  return Person{pt.get<0>(), pt.get<1>(), pt.get<2>()};
}

This is bad enough when there are only 3 columns/fields to fill, but in production code we have tables with 10-20 columns.

I would love if there was a clever way of unpacking a tuple into a struct. The best I can think of is to use std::tie:

Person readFromDB() {
  PersonTuple pt = readRowFromDB();
  Person p;
  std::tie(p.firstName, p.lastName, p.dob) = pt;
  return p;
}

If you could make a suggestion in the documentation on how to write such unpacking, I think that would be a help to all users of taopq. Even better would be to have a function that does the unpacking for you, but that's something I can't figure out yet. Possibly some code using Antony Polyukhin's reflection library from Boost: https://apolukhin.github.io/pfr_non_boost/

CMake - ExternalProject usage as an example

Hi,

I like your library, but it's problematic to get it installed. I am playing a lot with getting cmakes ExternalProject_Add to download it,compile it and link/include it.

DO you have any such setup? Could you share it as an example of how to "install" this?

Specializing Parameter traits

Hi,
Im trying to specialize paramter_traits for an absl::CivilDay type because I want to try out the new struct to row magi :)

So far i more or less copied the string specialization an specialized it for absl::CivilDay

template<>
   struct tao::pq::parameter_traits< absl::CivilDay >
      : parameter_traits< std::string_view >
   {
       explicit parameter_traits( const absl::CivilDay v ) noexcept
         : parameter_traits<std::string_view>( absl::FormatCivilTime(v) ){}
      
};

FormatCivilTime will return a string. I think this works if the postgres column type is TEXT. My problem is that I'm using a DATE column type and get an error:

C++ exception with description "ERROR:  column "local_date" is of type date but expression is of type bytea
LINE 1: INSERT INTO settings VALUES ($1, $2, $3, $4, $5, $6, $7, $8,...

I tried with all the types defined in oid.hpp but none of them work. Do i need to find the oid for DATE type?

Failed to cooperate with vcpkg in CMake

I have a CMakeLists.txt like this:

cmake_minimum_required(VERSION 3.21.4)

set(CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED TRUE)

# init dependencies
include(FetchContent)

# init vcpkg
FetchContent_Declare(
    vcpkg
    GIT_REPOSITORY  https://github.com/microsoft/vcpkg.git
    GIT_TAG         5bdb9d60123df20c247a1654e6a1def51d6c5140
    GIT_PROGRESS    TRUE
)
FetchContent_MakeAvailable(vcpkg)
execute_process(
    COMMAND sh ./bootstrap-vcpkg.sh --disableMetrics
    WORKING_DIRECTORY ${vcpkg_SOURCE_DIR}
    RESULTS_VARIABLE vcpkg_BUILD_RESULT
    COMMAND_ERROR_IS_FATAL ANY
)
set(CMAKE_TOOLCHAIN_FILE ${vcpkg_SOURCE_DIR}/scripts/buildsystems/vcpkg.cmake CACHE STRING "Vcpkg toolchain file")

project(xxx CXX)

# dep: taoPQ
execute_process(
    COMMAND ./vcpkg install libpq --clean-after-build
    WORKING_DIRECTORY ${vcpkg_SOURCE_DIR}
    COMMAND_ERROR_IS_FATAL ANY
)
find_package(PostgreSQL REQUIRED)

FetchContent_Declare(
   taopq
   GIT_REPOSITORY   https://github.com/taocpp/taopq
   GIT_TAG          8d9285f76726e312282ac2c1de70b3338b2c701d
   GIT_PROGRESS     TRUE
)
FetchContent_MakeAvailable(taopq)

...

But it failed with:

[cmake] -- Configuring done
[cmake] CMake Error in build/_deps/taopq-src/CMakeLists.txt:
[cmake]   Target "taopq" INTERFACE_INCLUDE_DIRECTORIES property contains path:
[cmake] 
[cmake]     "/home/iyang/xxx/build/_deps/vcpkg-src/installed/x64-linux/include"
[cmake] 
[cmake]   which is prefixed in the build directory.
[cmake] 
[cmake] 
[cmake] CMake Error in build/_deps/taopq-src/CMakeLists.txt:
[cmake]   Target "taopq" INTERFACE_INCLUDE_DIRECTORIES property contains path:
[cmake] 
[cmake]     "/home/iyang/xxx/build/_deps/vcpkg-src/installed/x64-linux/include"
[cmake] 
[cmake]   which is prefixed in the build directory.Target "taopq"
[cmake]   INTERFACE_INCLUDE_DIRECTORIES property contains path:
[cmake] 
[cmake]     "/home/iyang/xxx/build/_deps/vcpkg-src/installed/x64-linux/include"
[cmake] 
[cmake]   which is prefixed in the source directory.
[cmake] 
[cmake] 
[cmake] CMake Error in build/_deps/taopq-src/CMakeLists.txt:
[cmake]   Target "taopq" INTERFACE_INCLUDE_DIRECTORIES property contains path:
[cmake] 
[cmake]     "/home/iyang/xxx/build/_deps/vcpkg-src/installed/x64-linux/include/postgresql/server"
[cmake] 
[cmake]   which is prefixed in the build directory.
[cmake] 
[cmake] 
[cmake] CMake Error in build/_deps/taopq-src/CMakeLists.txt:
[cmake]   Target "taopq" INTERFACE_INCLUDE_DIRECTORIES property contains path:
[cmake] 
[cmake]     "/home/iyang/xxx/build/_deps/vcpkg-src/installed/x64-linux/include/postgresql/server"
[cmake] 
[cmake]   which is prefixed in the build directory.Target "taopq"
[cmake]   INTERFACE_INCLUDE_DIRECTORIES property contains path:
[cmake] 
[cmake]     "/home/iyang/xxx/build/_deps/vcpkg-src/installed/x64-linux/include/postgresql/server"
[cmake] 
[cmake]   which is prefixed in the source directory.
[cmake] 
[cmake] 
[cmake] -- Generating done

After I applied the following patch, CMake can be successfully configured.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 308c235..df1fc1a 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -92,8 +92,8 @@ target_include_directories(taopq
   PUBLIC
     $<INSTALL_INTERFACE:include>
     $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
-    ${PostgreSQL_INCLUDE_DIRS}
   PRIVATE
+    ${PostgreSQL_INCLUDE_DIRS}
     ${CMAKE_CURRENT_SOURCE_DIR}/src
 )

So I think maybe the change can directly happen on the code to avoid this patch ?

BTW, taoPQ is a very wonderful thing, THANKS for your libraries!

CI is broken

As you can see, the current CI status is not okay.

I want to check and fix it later.

Reference to "Table of Content" in README.md is not valid

README.md links to doc/TOC.md on [line 63] (

taopq/README.md

Line 63 in b170878

* [Table of Content](doc/TOC.md)

) which is not currently present in the tree.

The rest of the links under "Documentation" are to valid locations.

Missing byteswap.h on MSVC and macOS

On MSVC and macOS I get build errors due to missing include byteswap.h, i.e.

C:\builds\taopq\include\tao/pq/internal/parameter_binary_traits.hpp(13): fatal error C1083: Cannot open include file: 'byteswap.h': No such file or directory

The docs only mention this header on glibc systems. Should I look for a drop-in replacement for other platforms?

I have some problems to compile and use the incomplete methods

First of all, greetings and thanks for your excellent work.

I have some drawbacks when using the methods declared in each class.

For example, this error appears in null.hpp "Error 1 error C2470: 'final': looks like a function definition, but there is no parameter list; skipping apparent body", I do not have much expertise in the language so I do not understand the reason.

Also in the file "integer_sequence.hpp", in the line of code:
template <typename T, T ... Ns>
These three points after the T, what do they mean and how can I adapt the methods in each file?
I would really appreciate your help

Shared library Windows build: exporting symbols, is it acceptable?

Dear developers, thanks again for the very great work.

I have seen that the shared library build does not export any symbols on Microsoft Windows. With the help of cmake its nowadays quite easy to add explicit symbol visibility. When using explicit symbol visibility its also possible to default to hidden symbols on Un*x. Therefore the whole library becomes slightly more encapsulated.

I can provide such a pull request if you are interested? Please give me a heads up first, before I go through the effort to add the changes.

The change adds a new header postgres_export.h that is auto-generated by cmake. The header works in combination with an (automatic) cmake build-time definition that ensures that symbols become visible. In CMakeLists.txt, the required change is small:

include (GenerateExportHeader)
generate_export_header (postgres)
install (FILES ${CMAKE_CURRENT_BINARY_DIR}/postgres_export.h DESTINATION include)

The new header postgres_export.h needs to be included in all publicly visible source files. The header defines a macro POSTGRES_EXPORT that encapsulates all the "magic" required for symbol visibility on all platforms. An example for a fully visible class:

class POSTGRES_EXPORT connection final
         : public std::enable_shared_from_this< connection >

And a visible method:

POSTGRES_EXPORT std::string printf( const char* format, ... );

I can make all the required changes and provide a PR if you deem it interesting? Without such a change, a Windows shared library build can either only export all or no symbols.

Common base class for timeout_reached, network_error, sql_error

Do you think that it makes sense to have a common base class for timeout_reached, network_error and sql_error? The base class could be called common_error, taopq_error or other similar name. This would allow catching all these exception types with a single catch block.

If you think that such a base class makes sense, then I can provide a PR for it.

XMake packaging?

Hello, I was wondering if you would accept to package this library on the xmake build system ?

That would make xmake-based development easier.

Thanks!

Vector doesn't seem to conform to has_reserve

taopq/include/tao/pq/result.hpp

Line 151 in 40d2876

if constexpr( internal::has_reserve< T > ) {

using result_t = std::tuple<int, int>;
auto rows = resp.vector<result_t>();

resulting vector's size != it's capacity

bazel build target

Can we define a bazel build target for this project? It helps software built using bazel to adopt this library easily.

Reference: https://bazel.build/start/cpp

Stable releases?

Thank you for your work on this project!

It would be very helpful for us if you had releases, release branches, or some kind of versioning so we aren't all pulling directly from main which right now looks like the active development branch.

https://semver.org/

tests require ${PostgreSQL_INCLUDE_DIRS}

With the latest master I can not build the tests anymore, due to errors:

FAILED: src/test/pq/CMakeFiles/basic_datatypes.dir/basic_datatypes.cpp.o 
/data/Tools/lib/ccache/c++   -I/home/user/Source/taopq/include -ggdb3 -fno-omit-frame-pointer -O1 -m64 -march=nehalem -mtune=haswell  -DDEBUG -g -fPIE   -pedantic -Wall -Wextra -Wshadow -Werror -std=c++1z -MD -MT src/test/pq/CMakeFiles/basic_datatypes.dir/basic_datatypes.cpp.o -MF src/test/pq/CMakeFiles/basic_datatypes.dir/basic_datatypes.cpp.o.d -o src/test/pq/CMakeFiles/basic_datatypes.dir/basic_datatypes.cpp.o -c /home/user/Source/taopq/src/test/pq/basic_datatypes.cpp
In file included from /home/user/Source/taopq/include/tao/pq/parameter_traits.hpp:7:0,
                 from /home/user/Source/taopq/include/tao/pq/transaction.hpp:14,
                 from /home/user/Source/taopq/include/tao/pq/connection.hpp:13,
                 from /home/user/Source/taopq/src/test/pq/basic_datatypes.cpp:11:
/home/user/Source/taopq/include/tao/pq/internal/parameter_binary_traits.hpp:16:10: fatal error: libpq-fe.h: No such file or directory
 #include <libpq-fe.h>
          ^~~~~~~~~~~~
compilation terminated.

I'm not certain what the best fix is, therefore no PR attached. Should the tests just add ${PostgreSQL_INCLUDE_DIRS}?

can't get optional string

auto res = db->execute("...").at(0);
auto opt = res.optional<std::string>(3);

/usr/local/include/tao/pq/result_traits.hpp:23:7: error: static_assert failed due to requirement 'internal::dependent_false<std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>' "data type T not registered as taopq result type"
      static_assert( internal::dependent_false< T >, "data type T not registered as taopq result type" );
      ^              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/tao/pq/result_traits.hpp:32:66: note: in instantiation of template class 'tao::pq::result_traits<std::__1::optional<std::__1::basic_string<char>>>' requested here
   inline constexpr std::size_t result_traits_size< T, decltype( result_traits< T >::size ) > = result_traits< T >::size;
                                                                 ^
/usr/local/include/tao/pq/row.hpp:206:24: note: during template argument deduction for variable template partial specialization 'result_traits_size<T, decltype(result_traits<T>::size)>' [with T = std::__1::optional<std::__1::basic_string<char>>]
         if constexpr( result_traits_size< T > == 0 ) {
                       ^
/usr/local/include/tao/pq/row.hpp:229:17: note: in instantiation of function template specialization 'tao::pq::row::get<std::__1::optional<std::__1::basic_string<char>>>' requested here
         return get< std::optional< T > >( column );
                ^
/usr/local/include/tao/pq/row.hpp:219:40: error: no member named 'from' in 'tao::pq::result_traits<std::__1::optional<std::__1::basic_string<char>>>'
            return result_traits< T >::from( get( column ) );
                                       ^
/usr/local/include/tao/pq/row.hpp:229:17: note: in instantiation of function template specialization 'tao::pq::row::get<std::__1::optional<std::__1::basic_string<char>>>' requested here
         return get< std::optional< T > >( column );

Support Conan package

What?
Provide Conan package support for this project

Why?
Distribute taocpp::postgres over Bintray as a package

How?

• Provide Conan recipe
• Provide Conan test package
• Provide Conan build (CPT)
• Update Travis CI to build and upload Conan package
• Update Appveyor to build Conan package

Support arrays (as parameters and as fields)

Will be great if we could write something like:

conn->execute( "CREATE TABLE example ( tarray TEXT[] NOT NULL )" );
std::vector<std::string> args{"1", "2", "val3"};
conn->execute( "INSERT INTO  example (tarray) VALUES($1) ", args);
conn->execute( "INSERT INTO  example (tarray) VALUES($1) ", {"a", "b", "c"});

auto res = conn->execute( "SELECT tarray FROM example");
for (const auto &row : res) {
   for (const auto &val : row[0].as<std::vector<std::string>>()) 
    std::cout << val << std::endl;
}

/// multidimensional arrays will be even better
conn->execute( "CREATE TABLE examplem ( tarray TEXT[][] NOT NULL )" );
std::vector<std::vector<std::string>> args{{"1", "2", "3"}. {"4", "5"}};
conn->execute( "INSERT INTO  examplem (tarray) VALUES($1) ", args);
conn->execute( "INSERT INTO  examplem (tarray) VALUES($1) ", {{"a1", "b1", "c1"}, {"a2", "b2"});

auto res = conn->execute( "SELECT tarray FROM examplem");
for (const auto &row : res) {
   for (const auto &array : row[0].as<tao::pq::array_field>()) 
     for (const auto &val : array.as<std::vector<std::string>>()) 
        std::cout << val << std::endl;
}

Support largeobjects

largeobjects are very useful for storing any data into a database.

Error: undefined reference to 'WSAPoll', '__imp_WSAGetLastError'

Hello everyone! I've got a problem while using this library. It may not even be related to it, but i hope, you can help me. While trying to connect to the database I get an error.

in function tao::pq::connection::wait(bool, std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long long, std::ratio<1ll, 1000000000ll> > >)':
.../connection.cpp:271: undefined reference to 'WSAPoll'
.../connection.cpp:293: undefined reference to '__imp_WSAGetLastError'
collect2.exe: error: ld returned 1 exit status`

(connection.cpp is a file of this library after building. Route: cmake-build-debug/_deps/taocpp-taopq-src/src/lib/pq/connection.cpp)

A part of my test code that interacts with db is only dbConnect() function yet. It has no try-catch blocks but i think it doesn't matter. This part of code i took from the docs of this library.

void dbConnect () {
const auto conn = tao::pq::connection::create(DB_URL);
// DB_URL is a const of a std::string type that contains a url of real database that i have access to
const auto users = conn->execute( "SELECT * FROM users");
for( const auto& row : users ) {
std::cout << row[ "username" ].as< std::string >() << " is " << row[ "name" ].as< std::string >() << "\n";
}
}

I've also tried to insert into a connection string something like "host=... db=... ....", but I always catch the same error(

I googled what is the problem, but i cant find answers. Maybe you know how to solve it. I hope, you can help me. Thanks!

Test failed on macOS

11.4 (20F71)
Found PostgreSQL: /usr/local/lib/libpq.dylib (found version "13.3")
AppleClang 12.0.5.12050022

cmake -GNinja . \
        -DCMAKE_BUILD_TYPE=Debug \
        -DCMAKE_CXX_STANDARD=17 \
        -DCMAKE_CXX_STANDARD_REQUIRED=ON && sudo ninja install

basic_datatypes.cpp:294:37: error: magnitude of floating-point constant too large for type 'long double'; maximum is 1.7976931348623157E+308 [-Werror,-Wliteral-range]
   check< long double >( "NUMERIC", 1e1001L );

when the number of struct members is less than the number of columns in the table

For example

// an aggregate
struct user
{
   std::string name;
};

template<>
inline constexpr bool tao::pq::is_aggregate< user > = true;

int main()
{
   // open a connection to the database
   const auto conn = tao::pq::connection::create( "dbname=template1" );

   // execute statements
   conn->execute( "DROP TABLE IF EXISTS users" );
   conn->execute( "CREATE TABLE users ( name TEXT PRIMARY KEY, age INTEGER NOT NULL, planet TEXT NOT NULL )" );

   // execute previously prepared statements
   conn->execute( "insert_user", user{ "R. Daneel Olivaw", 19230, "Earth" } );

   // query and convert data even user only has on field
   for( const user u : conn->execute( "SELECT * FROM users" ) ) {
      std::cout << u.name;
   }
}

Is it possible to convert to a struct with fewer members than the number of table columns?

Batch Insert values

Is it possible to make a statement that will insert N values into a table?

I would like to be able to do something like

taopq/src/test/pq/traits.cpp

Line 56 in e1e5cc0

TEST_EXECUTE( connection->execute( "INSERT INTO tao_parameter_test VALUES ( $1, $2, $3, $4 )", example::user( 6 ) ) );

but with multiple values at once?

For two rows the sql statement would look like

INSERT INTO tao_parameter_test VALUES 
                               ( $1, $2, $3, $4 ),
                               ( $5, $6, $7, $8 );

I know I can type it manually but for performance reasonse i would like to insert ~500 values at once which would be a lot of typing :)

Thanks for putting up the library. I find it very easy to use.

best Søren

error: ‘null’ is not a member of ‘tao::pq::result_traits<int>’

I am getting these errors when I build! I have done a fresh clone. -- Thanks Kevin!

include/tao/pq/row.hpp: In instantiation of ‘T tao::pq::row::get(std::size_t) const [with T = int; std::size_t = long unsigned int]’:
include/tao/pq/row.hpp:299:40: required from ‘T tao::pq::field::as() const [with T = int]’
src/test/pq/result.cpp:156:10: required from here
include/tao/pq/row.hpp:224:50: error: ‘null’ is not a member of ‘tao::pq::result_traits’
return result_traits< T >::null();

include/tao/pq/row.hpp:224:50: required from ‘T tao::pq::row::get(std::size_t) const [with T = std::tuple; std::size_t = long unsigned int]’
include/tao/pq/row.hpp:247:29: required from ‘T tao::pq::row::as() const [with T = std::tuple]’
include/tao/pq/result.hpp:242:40: required from ‘T tao::pq::result::as() const [with T = std::tuple]’
include/tao/pq/result.hpp:263:43: required from ‘auto tao::pq::result::tuple() const [with Ts = {int}]’
src/test/pq/result.cpp:52:4: required from here
include/tao/pq/result_traits_tuple.hpp:24:55: error: ‘null’ is not a member of ‘tao::pq::result_traits’
return std::tuple< T >( result_traits< T >::null() );