uwhustle / hustle Goto Github PK
View Code? Open in Web Editor NEWIn-memory, columnar, arrow-based database.
License: Apache License 2.0
In-memory, columnar, arrow-based database.
License: Apache License 2.0
Can we loop the column type then over rows? If the lambda function is not inlined, we may pay a huge cost here.
Originally posted by @GindaChen in #105 (comment)
As mentioned in #99, we need to clean up the build script such that building in Ubuntu and macOS, with different language, can be easily done.
Major goals include:
brew
dependency on macOS if the compiler is g++. Almost all tools in brew
are built with clang, and if we need to use g++ we need to build most of them under the cmake toolchain. This is pretty doable../build.sh --cpp20
should give us a full build on C++20. Suggestions are welcomed.Not all of my comments follow the Doxygen style. Comments also need to be added to the .cpp files.
I am going to write my comments here so we have a history of the conversation.
Sending a shared pointer
https://github.com/UWQuickstep/hustle/blob/f2fa7582b2314ce0956aa61ded1ef6b36f3bcd92/src/main.cpp#L28
What is happening in this case, namely that the Block object contained in the shared pointer is deallocated before the receiver receives it, is the expected behavior. A shared pointer is alive while some in scope piece of code has a "name" for that shared pointer. (https://abseil.io/tips/55)
In the send/receive code when you write the value of the shared pointer to the queue and before the receiver receives it the reference counter goes to 0 and the Block object is deallocated.
There is no way for nng to know that what you are sending is a shared pointer, it never gets that information, bytes are sent.
How is this communication system going to be used? What data are we going to send? Who is going to own the data we send/receive? (maybe these questions will help you/us resolve this problem)
Raw Pointer Implementation
https://github.com/UWQuickstep/hustle/blob/44511ebceb4d399a2fa8687d9d0ca1a6218edf7a/src/main.cpp#L12
I am seeing that the pointer changes, but there is no additional call to the Block constructor (right?). Can you verify with Valgrind or another memory profiler that the memory usage is doubled? This is indeed very weird.
This issue extends #57 to make sure HashAggregate
delivers the following:
std::unordered_map
hash tables to phmap
(ideally) for better performance.AggregateStrategy
to flip between HashAggregate
and Aggregate
. We can also introduce this toggle in ssb_workload.cc
and ssb_workload_lip.cc
to compare the performance.arrow_compute_wrapper
and add some helpers to optimize the array construction.std::
and arrow::
.Finish()
.auto
Reference links
Sample Manual construction of the operator plan ->
hustle/src/benchmark/ssb_workload_lip.cc
Line 734 in df14c96
When installing hustle on macOS with the following procedure (in README.md):
brew
./install_requirements.sh
and ./install_arrow.sh
. After the step, I see hustle/arrow
directory.mkdir build && cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE ..
.This failes with the following log:
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
CMake Error at src/FindArrow.cmake:122 (message):
Could not find the Arrow library. Looked for headers in , and for libs in
Call Stack (most recent call first):
src/CMakeLists.txt:21 (find_package)
Question
cmake -v
on my machine is 3.18.2
instead of 3.15
. I think we didn't specify the cmake version in Brewfile
. Shall we stick to that cmake version, or whatever an upper version is okay?/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G "CodeBlocks - Unix Makefiles" /Users/ginda/Documents/GitHub/hustle
-- The C compiler identification is AppleClang 11.0.3.11030032
-- The CXX compiler identification is AppleClang 11.0.3.11030032
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found PythonInterp: /usr/bin/python (found version "2.7.16")
-- Looking for C++ include unistd.h
-- Looking for C++ include unistd.h - found
-- Looking for C++ include stdint.h
-- Looking for C++ include stdint.h - found
-- Looking for C++ include inttypes.h
-- Looking for C++ include inttypes.h - found
-- Looking for C++ include sys/types.h
-- Looking for C++ include sys/types.h - found
-- Looking for C++ include sys/stat.h
-- Looking for C++ include sys/stat.h - found
-- Looking for C++ include fnmatch.h
-- Looking for C++ include fnmatch.h - found
-- Looking for C++ include stddef.h
-- Looking for C++ include stddef.h - found
-- Check size of uint32_t
-- Check size of uint32_t - done
-- Looking for strtoll
-- Looking for strtoll - found
-- Check size of pthread_rwlock_t
-- Check size of pthread_rwlock_t - done
-- Performing Test COMPILER_HAS_WNO_UNUSED_LOCAL_TYPEDEF
-- Performing Test COMPILER_HAS_WNO_UNUSED_LOCAL_TYPEDEF - Success
-- Looking for C++ include dlfcn.h
-- Looking for C++ include dlfcn.h - found
-- Looking for C++ include execinfo.h
-- Looking for C++ include execinfo.h - found
-- Looking for C++ include glob.h
-- Looking for C++ include glob.h - found
-- Looking for C++ include libunwind.h
-- Looking for C++ include libunwind.h - found
-- Looking for C++ include memory.h
-- Looking for C++ include memory.h - found
-- Looking for C++ include pwd.h
-- Looking for C++ include pwd.h - found
-- Looking for C++ include stdlib.h
-- Looking for C++ include stdlib.h - found
-- Looking for C++ include strings.h
-- Looking for C++ include strings.h - found
-- Looking for C++ include string.h
-- Looking for C++ include string.h - found
-- Looking for C++ include syscall.h
-- Looking for C++ include syscall.h - not found
-- Looking for C++ include syslog.h
-- Looking for C++ include syslog.h - found
-- Looking for C++ include sys/syscall.h
-- Looking for C++ include sys/syscall.h - found
-- Looking for C++ include sys/time.h
-- Looking for C++ include sys/time.h - found
-- Looking for C++ include sys/ucontext.h
-- Looking for C++ include sys/ucontext.h - found
-- Looking for C++ include sys/utsname.h
-- Looking for C++ include sys/utsname.h - found
-- Looking for C++ include ucontext.h
-- Looking for C++ include ucontext.h - not found
-- Looking for C++ include unwind.h
-- Looking for C++ include unwind.h - found
-- Looking for fcntl
-- Looking for fcntl - found
-- Looking for sigalstack
-- Looking for sigalstack - not found
-- Check size of void*
-- Check size of void* - done
-- Performing Test HAVE___BUILTIN_EXPECT
-- Performing Test HAVE___BUILTIN_EXPECT - Success
-- Performing Test HAVE___ATTRIBUTE__
-- Performing Test HAVE___ATTRIBUTE__ - Success
-- Performing Test HAVE___SYNC_VAL_COMPARE_AND_SWAP
-- Performing Test HAVE___SYNC_VAL_COMPARE_AND_SWAP - Success
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[REG_EIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[REG_RIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.sc_ip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.uc_regs->gregs[PT_NIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[R15]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.arm_pc
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.mc_eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.mc_rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.__gregs[_REG_EIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.__gregs[_REG_RIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->ss.eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->__ss.__eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->ss.rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->__ss.__rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Success
-- Found program counter field in ucontext_t: uc_mcontext->__ss.__rip
CMake Warning (dev) at /Applications/CLion.app/Contents/bin/cmake/mac/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:272 (message):
The package name passed to `find_package_handle_standard_args` (PkgConfig)
does not match the name of the calling package (Arrow). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
/Applications/CLion.app/Contents/bin/cmake/mac/share/cmake-3.17/Modules/FindPkgConfig.cmake:45 (find_package_handle_standard_args)
src/FindArrow.cmake:10 (include)
src/CMakeLists.txt:21 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
CMake Error at src/FindArrow.cmake:122 (message):
Could not find the Arrow library. Looked for headers in , and for libs in
Call Stack (most recent call first):
src/CMakeLists.txt:21 (find_package)
-- Configuring incomplete, errors occurred!
See also "/Users/ginda/Documents/GitHub/hustle/cmake-build-debug/CMakeFiles/CMakeOutput.log".
See also "/Users/ginda/Documents/GitHub/hustle/cmake-build-debug/CMakeFiles/CMakeError.log".
[Failed to reload]
The new hash aggregate operator introduced in #57 can handle most of the queries in our SSB
workload.
Except for q11
- it can undeterministically throw the following error:
Assertion failed: (PolicyTraits::apply(FindElement{*this}, *iterator_at(i)) == iterator_at(i) && "constructed value does not match the lookup key"), function emplace_at, file /Users/ginda/Documents/GitHub/hustle/src/utils/parallel_hashmap/phmap.h, line 1999.
[1] 51407 abort ./hustle_src_benchmark_main
and when it doesn't throw the error, the result seems matching the output using normal aggregate.
This issue is about adding Small Materialize Aggregates (SMAs) on all numeric columns and then using these SMAs to "skip" blocks during query processing. The initial scope can be limited to:
Note, we keep SMAs in a memory structure outside the Arrow data block. If you want to look at the Quickstep code,
We want to design a "memoization" cache (MC) structure, so just before we start to run a select operation on a block, we:
foo
. The payload could be in a protobuf format, but please feel free to make a design decision on this aspect.foo > 100
, for blockid = 13
, and the min-max SMA for foo is [-14, 100)
, then we know that we can potentially skip block 13. To confirm that our metadata is current, we check the checksum for block 13 (which should be stored with the block and updated on any change).Thus, the functional cache can be an independent way to "accelerate" operators. It is an asynchronous data structure, so it can be built/updated lazily. For example, if new blocks are added to a table, then some blocks may have SMAs in the MC and others may not. For the latter, we have to scan the relevant columns in the block, but for hits in the MC, we may be able to skip scanning the block all together.
For now, the "answer" in the MC is a min-max SMA, but the "answer" in the 4-tuple could be a bloom filter. (Our design of the MC should leave room for this extension).
Ideally, we can use the MC to do more, and also store query expression results for pairs for block. So, in the 4-tuple, the third part could be more general than a column "expression." It can be generalized to be a relational algebraic expression (in a canonical data structure representation in the code), such as: σfoo > 100
.
We can generalize the expression in the MC even further to be a canonical representation for any relational algebra. With that extension, we can encode a complex function like πR.foo, S.bar [(σR.foo > 100) ⋈R.foobar = S.foobar (S)]
. Now, when evaluating this expression between an R block, r101, and an S block, s202, if an answer is present in the MC (and the checksums for both blocks match), we can just return the answer from the MC. No need to scan the data in the block(s).
More design is clearly needed for this more extensive use of MC, but it can be thought of as a "learning" layer and we can optimize what is being cached (the MC data structure likely has a limited memory budget) based on some utility function that is a combination of how frequently this entry is used and how expensive it is to compute. MC is a buffer pool for relational algebraic expression results at the block (pairs, triple, ...) level. This MC extension could be the Spring'21 784 project for the Hustle team members.
Perhaps @srsuryadev can help start this work and lead the design, and then pull in @mprammer2 in a pair-programming style on certain aspects. For example, @srsuryadev could design and implement the approach above with a min-max SMA as the "answer", and @mprammer2 could add the bloom filter. The 784 group could build the full-fledged MC feature.
Consider Gflags: https://gflags.github.io/gflags/
In general, a more centralized approach to setting default behavior may be relevant.
Currently, hustle resolver expects, the predicates from the same table are grouped to construct the individual predicate tree for each select operator for the table.
Example,
SELECT c_city, s_city, d_year, SUM(lo_revenue) AS revenue
FROM customer, lineorder, supplier, ddate
WHERE lo_custkey = c_custkey
AND lo_suppkey = s_suppkey
AND lo_orderdate = d_datekey
AND (c_city = 'UNITED KI1' OR c_city = 'UNITED KI5')
AND (s_city = 'UNITED KI1' OR s_city = 'UNITED KI5')
AND d_yearmonth = 'Dec1997'
GROUP BY c_city, s_city, d_year
ORDER BY d_year, revenue;
Make the resolver more generic to handle the case, even when the predicates are not grouped explicitly in the query.
Example,
SELECT c_city, s_city, d_year, SUM(lo_revenue) AS revenue
FROM customer, lineorder, supplier, ddate
WHERE lo_custkey = c_custkey
AND lo_suppkey = s_suppkey
AND lo_orderdate = d_datekey
AND c_city = 'UNITED KI1' OR c_city = 'UNITED KI5'
AND s_city = 'UNITED KI1' OR s_city = 'UNITED KI5'
AND d_yearmonth = 'Dec1997'
GROUP BY c_city, s_city, d_year
ORDER BY d_year, revenue;
Note: The paranthesis are not used in the query which is used by the sqlite parser to generate a subtree for that table alone.
The operator code uses switch blocks in several places to properly handle different data types. We should have a type system that replaces conditionals with polymorphism (also see this repo) or we could use templates.
List of issues that need to be fixed for making the Hustle wider SQL query range.
We don't have an agreed-upon way to handle errors. We should have a separate infrastructure to handle errors, so that we can use throw
to catch the stack-trace of an error.
The new hash aggregate operator introduced in #57 can handle most of the queries in our SSB
workload.
Except for q11
- it can undeterministically throw the following error:
Assertion failed: (PolicyTraits::apply(FindElement{*this}, *iterator_at(i)) == iterator_at(i) && "constructed value does not match the lookup key"), function emplace_at, file /Users/ginda/Documents/GitHub/hustle/src/utils/parallel_hashmap/phmap.h, line 1999.
[1] 51407 abort ./hustle_src_benchmark_main
and when it doesn't throw the error, the result seems matching the output using normal aggregate.
Got the following error due to an update to the ssb-dbgen:
> sh ./scripts/ssb/gen_benchmark_data.sh 1
...
CMake Error at CMakeLists.txt:163 (set_property):
C_STANDARD property may not be appended.
CMake Error at CMakeLists.txt:163 (set_property):
C_STANDARD property may not be appended.
@srsuryadev I think we can either
Google released a library for returning and handling error messages some time ago. Now that hustle is growing maybe such a solution would be beneficial to make the code cleaner and handling/propagation of errors easier. We had discussed this in the past, but never went through with it (this library was not available at the time).
You can find more information here: https://abseil.io/docs/cpp/guides/status
What do you think? Would such a convention for error handling play nice with arrow?
As mentioned ini #94, C++20 gives us promising solution to metaprogram easily and without loss of performance. To explore this possibility, we need to bump Hustle to CPP20.
I'm experimenting inside the following system:
set(CMAKE_CXX_STANDARD 20)
The following problems needs to be addressed:
Cereal.hpp
complaints at compile time.sqlite_utils.h:38
complaints at compile time.arrow
linker error. Solved by upgrading to latest arrow ini git repo. (Haven't used the release yet)absl
compiler error: abseil/abseil-cpp#722sqlite3
: Undefined references.The dump fiile libarrow.a.dump.zip does not contains the mangled name that compiler generates. For example,
"__ZNK5arrow6Status8ToStringB5cxx11Ev", referenced from:
__Z15evaluate_statusRKN5arrow6StatusEPKci in libhustle_src_storage.a(util.cc.o)
is produced by the compiler, but the closest we got is
U __ZNK5arrow6Status8ToStringEv
The complete compiler error g++-10
====================[ Build | hustle_select_test | Debug-GPP17 ]================
/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --build /Users/gindachen/Project/hustle/cmake-build-debug-gpp17 --target hustle_select_test -- -j 9
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[REG_EIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[REG_RIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.sc_ip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.uc_regs->gregs[PT_NIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[R15]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.arm_pc
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.mc_eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.mc_rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.__gregs[_REG_EIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.__gregs[_REG_RIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->ss.eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->__ss.__eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->ss.rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->__ss.__rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Success
-- Found program counter field in ucontext_t: uc_mcontext->__ss.__rip
-- Arrow SO and ABI version: 300
-- Arrow full SO version: 300.0.0
-- Found the Arrow core library: /usr/local/lib/libarrow.dylib
-- Found the Arrow Python library: ARROW_PYTHON_LIB_PATH-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/gindachen/Project/hustle/cmake-build-debug-gpp17
[ 1%] Built target absl_log_severity
[ 3%] Built target hustle_src_scheduler_SchedulerTypedefs
[ 3%] Built target hustle_src_utils_meta_FunctionTraits
[ 3%] Built target gflags_nothreads-static
[ 5%] Built target absl_spinlock_wait
[ 7%] Built target gtest
[ 7%] Built target absl_exponential_biased
[ 9%] Built target hustle_src_utils_Macros
[ 13%] Built target glog
[ 15%] Built target hustle_src_utils_ThreadSafeQueue
[ 15%] Built target hustle_src_utils_ParallelHashmap
[ 17%] Built target hustle_src_utils_Config
[ 17%] Built target hustle_src_utils_ContainerUtil
[ 17%] Built target absl_city
[ 19%] Built target absl_civil_time
[ 19%] Built target hustle_src_type
[ 19%] Built target absl_int128
[ 23%] Built target absl_time_zone
[ 27%] Built target hustle_src_scheduler_WorkerMessage
[ 27%] Built target hustle_src_threading_ThreadUtil
[ 29%] Built target hustle_src_scheduler_SchedulerMessage
[ 29%] Built target hustle_src_threading_Mutex
[ 31%] Built target hustle_src_scheduler_SchedulerFlags
[ 33%] Built target hustle_src_scheduler_Node
[ 35%] Built target hustle_src_utils_SyncStream
[ 35%] Built target hustle_src_scheduler_SchedulerInterface
[ 37%] Built target hustle_src_scheduler_TaskName
[ 37%] Built target hustle_src_utils_EventProfiler
[ 37%] Built target gtest_main
[ 39%] Built target gmock
[ 39%] Built target absl_raw_logging_internal
[ 39%] Built target hustle_src_scheduler_TaskDescription
[ 43%] Built target absl_bad_optional_access
[ 43%] Built target absl_throw_delegate
[ 47%] Built target absl_base
[ 47%] Built target absl_bad_variant_access
[ 50%] Built target absl_strings_internal
[ 52%] Built target absl_debugging_internal
[ 54%] Built target hustle_src_scheduler_Task
[ 54%] Built target absl_malloc_internal
[ 54%] Built target absl_demangle_internal
[ 56%] Built target absl_stacktrace
[ 64%] Built target absl_strings
[ 64%] Built target hustle_src_scheduler_Worker
[ 64%] Built target hustle_src_optimizer_ExecutionPlan
[ 64%] Built target absl_graphcycles_internal
[ 68%] Built target absl_symbolize
[ 70%] Built target absl_cord
[ 70%] Built target absl_hash
[ 72%] Built target absl_time
[ 72%] Built target hustle_src_scheduler_Scheduler
[ 76%] Built target absl_synchronization
[ 78%] Built target hustle_src_utils_ArrowComputeWrappers
[ 80%] Built target absl_hashtablez_sampler
[ 80%] Built target absl_raw_hash_set
[ 88%] Built target hustle_src_operators
[ 92%] Built target hustle_src_storage
[ 94%] Built target hustle_src_resolver
[ 96%] Built target hustleDB
[ 98%] Built target hustle_src_catalog
[100%] Built target sqlite_utils
[100%] Built target sqlite3
[100%] Linking CXX executable hustle_select_test
ld: warning: dylib (/usr/local/lib/libconfig++.dylib) was built for newer macOS version (11.0) than being linked (10.15)
ld: warning: dylib (/usr/local/lib/libarrow.dylib) was built for newer macOS version (11.0) than being linked (10.15)
ld: warning: dylib (/usr/local/gcc-10.1.0/lib/libstdc++.dylib) was built for newer macOS version (10.16) than being linked (10.15)
Undefined symbols for architecture x86_64:
"__ZN5arrow11RecordBatch4MakeESt10shared_ptrINS_6SchemaEExSt6vectorIS1_INS_9ArrayDataEESaIS6_EE", referenced from:
__ZN6hustle7storage5Block11get_recordsEv in libhustle_src_storage.a(util.cc.o)
__Z17copy_record_batchRKSt10shared_ptrIN5arrow11RecordBatchEE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow12ArrayBuilder6FinishEPSt10shared_ptrINS_5ArrayEE", referenced from:
__ZN39SelectTestFixture_SingleSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN36SelectTestFixture_AndSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN35SelectTestFixture_OrSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN49SelectTestFixture_SingleSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
__ZN46SelectTestFixture_AndSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
__ZZN6hustle7Context13apply_indicesEPNS_4TaskEN5arrow5DatumES4_S4_RS4_ENKUlS2_E_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow12ChunkedArrayC1ESt6vectorISt10shared_ptrINS_5ArrayEESaIS4_EE", referenced from:
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in select_test.cc.o
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_storage.a(util.cc.o)
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_operators.a(select.cc.o)
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_operators.a(lazy_table.cc.o)
__ZN5arrow12ChunkedArrayC1ESt10shared_ptrINS_5ArrayEE in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow12StringScalarC1ENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN9__gnu_cxx13new_allocatorIN5arrow12StringScalarEE9constructIS2_JRA3_KcEEEvPT_DpOT0_ in select_test.cc.o
"__ZN5arrow13SchemaBuilder8AddFieldERKSt10shared_ptrINS_5FieldEE", referenced from:
__Z11make_schemaRKN6hustle7catalog11TableSchemaE in libhustle_src_storage.a(util.cc.o)
__ZN6hustle9operators14OperatorResult11materializeERKSt6vectorINS0_15ColumnReferenceESaIS3_EEb in libhustle_src_operators.a(operator_result.cc.o)
"__ZN5arrow15MakeArrayOfNullERKSt10shared_ptrINS_8DataTypeEExPNS_10MemoryPoolE", referenced from:
__Z21array_to_chunkedarraySt10shared_ptrIN5arrow5ArrayEEi in libhustle_src_storage.a(util.cc.o)
__ZN6hustle7Context26apply_indices_internal_strERKSt10shared_ptrIN5arrow12ChunkedArrayEERKS1_INS2_5ArrayEESA_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context23apply_indices_internal2IxEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_SF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context22apply_indices_internalIxEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context23apply_indices_internal2IjEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_SF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context22apply_indices_internalIjEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow18RecordBatchBuilder4MakeERKSt10shared_ptrINS_6SchemaEEPNS_10MemoryPoolEPSt10unique_ptrIS0_St14default_deleteIS0_EE", referenced from:
__Z18read_from_csv_filePKcSt10shared_ptrIN5arrow6SchemaEEib in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow2io12ReadableFile4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPNS_10MemoryPoolE", referenced from:
__Z14read_from_filePKcbS0_ in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow2io16FileOutputStream4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb", referenced from:
__Z13write_to_filePKcRN6hustle7storage7DBTableE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow3ipc13NewFileWriterEPNS_2io12OutputStreamERKSt10shared_ptrINS_6SchemaEERKNS0_15IpcWriteOptionsERKS4_IKNS_16KeyValueMetadataEE", referenced from:
__Z13write_to_filePKcRN6hustle7storage7DBTableE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow3ipc21RecordBatchFileReader4OpenERKSt10shared_ptrINS_2io16RandomAccessFileEERKNS0_14IpcReadOptionsE", referenced from:
__Z14read_from_filePKcbS0_ in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow4util6detail19StringStreamWrapper3strB5cxx11Ev", referenced from:
__ZN5arrow4util13StringBuilderIJRA46_KcRxRA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in select_test.cc.o
__ZN5arrow4util13StringBuilderIJRA36_KcRxRA19_S2_S5_RA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in select_test.cc.o
__ZN5arrow4util13StringBuilderIJRA50_KcxRA22_S2_RxEEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in select_test.cc.o
__ZN5arrow4util13StringBuilderIJRA46_KcRxRA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow4util13StringBuilderIJRA36_KcRxRA19_S2_S5_RA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow4util13StringBuilderIJRA24_KcEEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_storage.a(sma.cc.o)
"__ZN5arrow5DatumC1ERKSt10shared_ptrINS_5ArrayEE", referenced from:
__ZN6hustle9operators6Select6FilterERKSt10shared_ptrINS_7storage5BlockEERKS2_INS0_9PredicateEE in libhustle_src_operators.a(select.cc.o)
__ZN5arrow5DatumC1INS_12BooleanArrayEvEERKSt10shared_ptrIT_E in libhustle_src_operators.a(select.cc.o)
__ZZZN6hustle7Context5matchEPNS_4TaskERKN5arrow5DatumES6_ENKUlS2_E_clES2_ENKUlvE_clEv in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context18apply_filter_blockERKSt10shared_ptrIN5arrow5ArrayEES6_RSt6vectorIS4_SaIS4_EE in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZZN6hustle7Context12apply_filterEPNS_4TaskERKN5arrow5DatumES6_RS4_ENKUlS2_E_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7storage3SmaC2ERKSt10shared_ptrIN5arrow5ArrayEE in libhustle_src_storage.a(sma.cc.o)
__ZN6hustle7storage3SmaC1ERKSt10shared_ptrIN5arrow5ArrayEE in libhustle_src_storage.a(sma.cc.o)
...
"__ZN5arrow5DatumC1ESt10shared_ptrINS_12ChunkedArrayEE", referenced from:
__ZN9__gnu_cxx13new_allocatorIN6hustle9operators9LazyTableEE9constructIS3_JRSt10shared_ptrINS1_7storage7DBTableEERS6_IN5arrow12ChunkedArrayEENSB_5DatumESF_EEEvPT_DpOT0_ in libhustle_src_operators.a(select.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEi in libhustle_src_operators.a(lazy_table.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEPNS_4TaskEiRN5arrow5DatumE in libhustle_src_operators.a(lazy_table.cc.o)
__ZZN6hustle7Context13apply_indicesEPNS_4TaskEN5arrow5DatumES4_S4_RS4_ENKUlS2_E0_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow5fieldENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS_8DataTypeEEbS6_IKNS_16KeyValueMetadataEE", referenced from:
__ZN17SelectTestFixture5SetUpEv in select_test.cc.o
__Z11make_schemaRKN6hustle7catalog11TableSchemaE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow6StatusC1ENS_10StatusCodeERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN5arrow6Status8FromArgsIJRA46_KcRxRA2_S2_EEES0_NS_10StatusCodeEDpOT_ in select_test.cc.o
__ZN5arrow6Status8FromArgsIJRA36_KcRxRA19_S2_S5_RA2_S2_EEES0_NS_10StatusCodeEDpOT_ in select_test.cc.o
__ZN5arrow6Status8FromArgsIJRA50_KcxRA22_S2_RxEEES0_NS_10StatusCodeEDpOT_ in select_test.cc.o
__ZN5arrow6Status8FromArgsIJRA46_KcRxRA2_S2_EEES0_NS_10StatusCodeEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow6Status8FromArgsIJRA36_KcRxRA19_S2_S5_RA2_S2_EEES0_NS_10StatusCodeEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow6Status8FromArgsIJRA24_KcEEES0_NS_10StatusCodeEDpOT_ in libhustle_src_storage.a(sma.cc.o)
"__ZN5arrow6schemaESt6vectorISt10shared_ptrINS_5FieldEESaIS3_EES1_IKNS_16KeyValueMetadataEE", referenced from:
__ZN17SelectTestFixture5SetUpEv in select_test.cc.o
"__ZN5arrow9ArrayData4MakeERKSt10shared_ptrINS_8DataTypeEExSt6vectorIS1_INS_6BufferEESaIS8_EExx", referenced from:
__ZN5arrow14NumericBuilderINS_10DoubleTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in select_test.cc.o
__ZN5arrow14NumericBuilderINS_9Int64TypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in select_test.cc.o
__ZN5arrow17BaseBinaryBuilderINS_10BinaryTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in select_test.cc.o
__Z17copy_record_batchRKSt10shared_ptrIN5arrow11RecordBatchEE in libhustle_src_storage.a(util.cc.o)
__ZN6hustle7storage5BlockC2EiRKSt10shared_ptrIN5arrow6SchemaEEib in libhustle_src_storage.a(block.cc.o)
__ZN6hustle7storage5BlockC1EiRKSt10shared_ptrIN5arrow6SchemaEEib in libhustle_src_storage.a(block.cc.o)
__ZN6hustle7storage5BlockC2EiSt10shared_ptrIN5arrow11RecordBatchEEib in libhustle_src_storage.a(block.cc.o)
...
"__ZN5arrow9MakeArrayERKSt10shared_ptrINS_9ArrayDataEE", referenced from:
__ZNK6hustle7storage5Block10get_columnEi in select_test.cc.o
__ZNK6hustle7storage5Block18get_column_by_nameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(select.cc.o)
__ZNK6hustle7storage5Block10get_columnEi in libhustle_src_storage.a(ma_block.cc.o)
__ZN6hustle7storage5Block9out_blockEPvPFiS2_iPPcS4_E in libhustle_src_storage.a(block.cc.o)
__ZN6hustle7storage5Block5printEv in libhustle_src_storage.a(block.cc.o)
__ZNK6hustle7storage5Block16get_valid_columnEv in libhustle_src_storage.a(table.cc.o)
__ZNK6hustle7storage5Block10get_columnEi in libhustle_src_operators.a(lazy_table.cc.o)
...
"__ZNK5arrow5Array6EqualsERKSt10shared_ptrIS0_ERKNS_12EqualOptionsE", referenced from:
__ZN39SelectTestFixture_SingleSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN36SelectTestFixture_AndSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN35SelectTestFixture_OrSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN49SelectTestFixture_SingleSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
__ZN46SelectTestFixture_AndSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
"__ZNK5arrow6Schema11field_namesB5cxx11Ev", referenced from:
__ZN6hustle7storage5Block9out_blockEPvPFiS2_iPPcS4_E in libhustle_src_storage.a(block.cc.o)
"__ZNK5arrow6Schema13GetFieldIndexERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZNK6hustle7storage5Block18get_column_by_nameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(select.cc.o)
__ZN6hustle7storage21MetadataAttachedBlock14SearchMetadataERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKN5arrow5DatumENSA_7compute15CompareOperatorE in libhustle_src_operators.a(select.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(lazy_table.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEPNS_4TaskENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN5arrow5DatumE in libhustle_src_operators.a(lazy_table.cc.o)
"__ZNK5arrow6Schema14GetFieldByNameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN6hustle9operators14OperatorResult11materializeERKSt6vectorINS0_15ColumnReferenceESaIS3_EEb in libhustle_src_operators.a(operator_result.cc.o)
"__ZNK5arrow6Status8ToStringB5cxx11Ev", referenced from:
__Z15evaluate_statusRKN5arrow6StatusEPKci in libhustle_src_storage.a(util.cc.o)
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make[3]: *** [src/operators/hustle_select_test] Error 1
make[2]: *** [src/operators/CMakeFiles/hustle_select_test.dir/all] Error 2
make[1]: *** [src/operators/CMakeFiles/hustle_select_test.dir/rule] Error 2
make: *** [hustle_select_test] Error 2
[ 93%] Linking CXX executable hustle_util_test
../../third_party/sqlite3/libsqlite3.a(sqlite3.c.o): In function `sqlite3LeaveMutexAndCloseZombie':
sqlite3.c:(.text+0x4ba12): undefined reference to `hustle_memlog_free'
../../third_party/sqlite3/libsqlite3.a(sqlite3.c.o): In function `sqlite3_open':
sqlite3.c:(.text+0x6804c): undefined reference to `hustle_memlog_initialize'
../../third_party/sqlite3/libsqlite3.a(sqlite3.c.o): In function `sqlite3_open_v2':
sqlite3.c:(.text+0x68085): undefined reference to `hustle_memlog_initialize'
../../third_party/sqlite3/libsqlite3.a(sqlite3.c.o): In function `sqlite3VdbeHalt':
sqlite3.c:(.text+0x6fe8b): undefined reference to `hustle_memlog_clear'
sqlite3.c:(.text+0x7086c): undefined reference to `hustle_memlog_update_db'
../../third_party/sqlite3/libsqlite3.a(sqlite3.c.o): In function `sqlite3VdbeExec':
sqlite3.c:(.text+0xa8134): undefined reference to `memlog_add_column_change'
sqlite3.c:(.text+0xa903f): undefined reference to `hustle_memlog_create_record'
sqlite3.c:(.text+0xa905d): undefined reference to `hustle_memlog_insert_record'
sqlite3.c:(.text+0xa94c0): undefined reference to `memlog_remove_table_mapping'
sqlite3.c:(.text+0xaa82f): undefined reference to `hustle_memlog_create_record'
sqlite3.c:(.text+0xaa842): undefined reference to `hustle_memlog_insert_record'
sqlite3.c:(.text+0xab217): undefined reference to `hustle_memlog_create_record'
sqlite3.c:(.text+0xab22a): undefined reference to `hustle_memlog_insert_record'
sqlite3.c:(.text+0xad2c3): undefined reference to `hustle_memlog_update_db'
../../third_party/sqlite3/libsqlite3.a(sqlite3.c.o): In function `sqlite3_exec':
sqlite3.c:(.text+0xb255a): undefined reference to `resolveSelect'
../../third_party/sqlite3/libsqlite3.a(sqlite3.c.o): In function `sqlite3EndTable':
sqlite3.c:(.text+0xd0026): undefined reference to `memlog_add_table_mapping'
collect2: error: ld returned 1 exit status
src/storage/CMakeFiles/hustle_util_test.dir/build.make:156: recipe for target 'src/storage/hustle_util_test' failed
make[2]: *** [src/storage/hustle_util_test] Error 1
CMakeFiles/Makefile2:5177: recipe for target 'src/storage/CMakeFiles/hustle_util_test.dir/all' failed
make[1]: *** [src/storage/CMakeFiles/hustle_util_test.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Solved logs
In file included from /Users/gindachen/Projects/hustle/src/utils/sqlite_utils.cc:18:
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:29:29: error: 'string' in namespace 'std' does not name a type
29 | void load_tables(const std::string &sqlitePath, std::vector<std::string> tables);
| ^~~~~~
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:23:1: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
22 | #include <vector>
+++ |+#include <string>
23 |
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:29:66: error: 'string' is not a member of 'std'
29 | void load_tables(const std::string &sqlitePath, std::vector<std::string> tables);
| ^~~~~~
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:29:66: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:29:72: error: template argument 1 is invalid
29 | void load_tables(const std::string &sqlitePath, std::vector<std::string> tables);
| ^
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:29:72: error: template argument 2 is invalid
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:33:38: error: 'string' in namespace 'std' does not name a type
33 | bool execute_sqlite_query(const std::string &sqlitePath,
| ^~~~~~
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:33:33: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
33 | bool execute_sqlite_query(const std::string &sqlitePath,
| ^~~
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:34:38: error: 'string' in namespace 'std' does not name a type
34 | const std::string &sql);
| ^~~~~~
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:34:33: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
34 | const std::string &sql);
| ^~~
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:38:6: error: 'string' in namespace 'std' does not name a type
38 | std::string execute_sqlite_result(const std::string &sqlitePath,
| ^~~~~~
/Users/gindachen/Projects/hustle/src/utils/sqlite_utils.h:38:1: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
38 | std::string execute_sqlite_result(const std::string &sqlitePath,
| ^~~
Undefined symbols for architecture x86_64:
"__ZN5arrow10MapBuilderC1EPNS_10MemoryPoolERKSt10shared_ptrINS_12ArrayBuilderEES7_b", referenced from:
__ZSt12construct_atIN5arrow10MapBuilderEJPNS0_10MemoryPoolERSt10shared_ptrINS0_12ArrayBuilderEES7_EEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPS9_DpOSA_ in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow10dictionaryERKSt10shared_ptrINS_8DataTypeEES4_b", referenced from:
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
__ZNK5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_9Int32TypeEE4typeEv in libhustle_src_type.a(type_helper.cc.o)
__ZNK5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_8Int8TypeEE4typeEv in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow10large_listERKSt10shared_ptrINS_8DataTypeEE", referenced from:
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow11RecordBatch4MakeESt10shared_ptrINS_6SchemaEExSt6vectorIS1_INS_9ArrayDataEESaIS6_EE", referenced from:
__ZN6hustle7storage5Block11get_recordsEv in libhustle_src_storage.a(util.cc.o)
__Z17copy_record_batchRKSt10shared_ptrIN5arrow11RecordBatchEE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow11dense_unionESt6vectorISt10shared_ptrINS_5FieldEESaIS3_EES0_IaSaIaEE", referenced from:
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow12ArrayBuilder6FinishEPSt10shared_ptrINS_5ArrayEE", referenced from:
__ZN38HashAggregateTestFixture_MeanTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN37HashAggregateTestFixture_SumTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN39HashAggregateTestFixture_CountTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN48HashAggregateTestFixture_SumWithGroupByTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN55HashAggregateTestFixture_SumWithGroupByOrderByTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN6hustle9operators13HashAggregate6FinishEPNS_4TaskE in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZZN6hustle7Context13apply_indicesEPNS_4TaskEN5arrow5DatumES4_S4_RS4_ENKUlS2_E_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
...
"__ZN5arrow12ChunkedArrayC1ESt6vectorISt10shared_ptrINS_5ArrayEESaIS4_EE", referenced from:
__ZSt12construct_atIN5arrow12ChunkedArrayEJRSt6vectorISt10shared_ptrINS0_5ArrayEESaIS5_EEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPSA_DpOSB_ in hash_aggregate_test.cc.o
__ZSt12construct_atIN5arrow12ChunkedArrayEJRSt6vectorISt10shared_ptrINS0_5ArrayEESaIS5_EEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPSA_DpOSB_ in libhustle_src_storage.a(util.cc.o)
__ZSt12construct_atIN5arrow12ChunkedArrayEJRSt6vectorISt10shared_ptrINS0_5ArrayEESaIS5_EEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPSA_DpOSB_ in libhustle_src_operators.a(lazy_table.cc.o)
__ZN5arrow12ChunkedArrayC1ESt10shared_ptrINS_5ArrayEE in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZSt12construct_atIN5arrow12ChunkedArrayEJRSt6vectorISt10shared_ptrINS0_5ArrayEESaIS5_EEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPSA_DpOSB_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow12sparse_unionESt6vectorISt10shared_ptrINS_5FieldEESaIS3_EES0_IaSaIaEE", referenced from:
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow13SchemaBuilder8AddFieldERKSt10shared_ptrINS_5FieldEE", referenced from:
__Z11make_schemaRKN6hustle7catalog11TableSchemaE in libhustle_src_storage.a(util.cc.o)
__ZN6hustle9operators13HashAggregate12OutputSchemaENS0_15AggregateKernelERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN6hustle9operators14OperatorResult11materializeERKSt6vectorINS0_15ColumnReferenceESaIS3_EEb in libhustle_src_operators.a(operator_result.cc.o)
"__ZN5arrow13SchemaBuilder9AddFieldsERKSt6vectorISt10shared_ptrINS_5FieldEESaIS4_EE", referenced from:
__ZN6hustle9operators13HashAggregate12OutputSchemaENS0_15AggregateKernelERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(hash_aggregate.cc.o)
"__ZN5arrow13StructBuilderC1ERKSt10shared_ptrINS_8DataTypeEEPNS_10MemoryPoolESt6vectorIS1_INS_12ArrayBuilderEESaISA_EE", referenced from:
__ZSt12construct_atIN5arrow13StructBuilderEJRSt10shared_ptrINS0_8DataTypeEEPNS0_10MemoryPoolESt6vectorIS2_INS0_12ArrayBuilderEESaISA_EEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPSE_DpOSF_ in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZSt12construct_atIN5arrow13StructBuilderEJRSt10shared_ptrINS0_10StructTypeEEPNS0_10MemoryPoolERSt6vectorIS2_INS0_12ArrayBuilderEESaISA_EEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPSF_DpOSG_ in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow15MakeArrayOfNullERKSt10shared_ptrINS_8DataTypeEExPNS_10MemoryPoolE", referenced from:
__Z21array_to_chunkedarraySt10shared_ptrIN5arrow5ArrayEEi in libhustle_src_storage.a(util.cc.o)
__ZN6hustle7Context26apply_indices_internal_strERKSt10shared_ptrIN5arrow12ChunkedArrayEERKS1_INS2_5ArrayEESA_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context23apply_indices_internal2IxEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_SF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context22apply_indices_internalIxEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context23apply_indices_internal2IjEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_SF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context22apply_indices_internalIjEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow15fixed_size_listERKSt10shared_ptrINS_8DataTypeEEi", referenced from:
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow17BasicUnionBuilder14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE", referenced from:
__ZTVN5arrow18SparseUnionBuilderE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow17BasicUnionBuilderC2EPNS_10MemoryPoolERKSt6vectorISt10shared_ptrINS_12ArrayBuilderEESaIS6_EERKS4_INS_8DataTypeEE", referenced from:
__ZN5arrow18SparseUnionBuilderC1EPNS_10MemoryPoolERKSt6vectorISt10shared_ptrINS_12ArrayBuilderEESaIS6_EERKS4_INS_8DataTypeEE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow17Decimal128BuilderC1ERKSt10shared_ptrINS_8DataTypeEEPNS_10MemoryPoolE", referenced from:
__ZSt12construct_atIN5arrow17Decimal128BuilderEJRSt10shared_ptrINS0_8DataTypeEEPNS0_10MemoryPoolEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPS9_DpOSA_ in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow18AdaptiveIntBuilder14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE", referenced from:
__ZN5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_9Int32TypeEE20FinishWithDictOffsetExPSt10shared_ptrINS_9ArrayDataEES8_ in libhustle_src_type.a(type_helper.cc.o)
__ZN5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_8Int8TypeEE20FinishWithDictOffsetExPSt10shared_ptrINS_9ArrayDataEES8_ in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow18RecordBatchBuilder4MakeERKSt10shared_ptrINS_6SchemaEEPNS_10MemoryPoolEPSt10unique_ptrIS0_St14default_deleteIS0_EE", referenced from:
__Z18read_from_csv_filePKcSt10shared_ptrIN5arrow6SchemaEEib in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow20FixedSizeListBuilderC1EPNS_10MemoryPoolERKSt10shared_ptrINS_12ArrayBuilderEERKS3_INS_8DataTypeEE", referenced from:
__ZSt12construct_atIN5arrow20FixedSizeListBuilderEJPNS0_10MemoryPoolERSt10shared_ptrINS0_12ArrayBuilderEERS4_INS0_8DataTypeEEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPSC_DpOSD_ in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow22FixedSizeBinaryBuilderC1ERKSt10shared_ptrINS_8DataTypeEEPNS_10MemoryPoolE", referenced from:
__ZSt12construct_atIN5arrow22FixedSizeBinaryBuilderEJRSt10shared_ptrINS0_8DataTypeEEPNS0_10MemoryPoolEEEDTgsnwcvPvLi0E_T_pispcl7declvalIT0_EEEEPS9_DpOSA_ in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow2io12ReadableFile4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPNS_10MemoryPoolE", referenced from:
__Z14read_from_filePKcbS0_ in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow2io16FileOutputStream4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb", referenced from:
__Z13write_to_filePKcRN6hustle7storage7DBTableE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow3ipc13NewFileWriterEPNS_2io12OutputStreamERKSt10shared_ptrINS_6SchemaEERKNS0_15IpcWriteOptionsERKS4_IKNS_16KeyValueMetadataEE", referenced from:
__Z13write_to_filePKcRN6hustle7storage7DBTableE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow3ipc21RecordBatchFileReader4OpenERKSt10shared_ptrINS_2io16RandomAccessFileEERKNS0_14IpcReadOptionsE", referenced from:
__Z14read_from_filePKcbS0_ in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow3mapESt10shared_ptrINS_8DataTypeEES2_b", referenced from:
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow4listERKSt10shared_ptrINS_8DataTypeEE", referenced from:
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow4util6detail19StringStreamWrapper3strB5cxx11Ev", referenced from:
__ZN5arrow4util13StringBuilderIJRA46_KcRxRA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow4util13StringBuilderIJRA36_KcRxRA19_S2_S5_RA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow4util13StringBuilderIJRA32_KcxRA14_S2_RxEEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow4util13StringBuilderIJRA50_KcxRA22_S2_RxEEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow4util13StringBuilderIJRA46_KcRxRA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN5arrow4util13StringBuilderIJRA36_KcRxRA19_S2_S5_RA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN5arrow4util13StringBuilderIJRA32_KcxRA14_S2_RxEEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_operators.a(hash_aggregate.cc.o)
...
"__ZN5arrow5DatumC1ERKSt10shared_ptrINS_5ArrayEE", referenced from:
__ZNO5arrow6ResultISt10shared_ptrINS_5ArrayEEE5ValueINS_5DatumEvEENS_6StatusEPT_ in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN6hustle9operators10Expression8EvaluateEPNS_4TaskEi in libhustle_src_operators.a(expression.cc.o)
__ZN6hustle9operators10Expression12ExecuteBlockIN5arrow12BooleanArrayEbEENS3_5DatumEiSt10shared_ptrINS3_5ArrayEES8_S8_ in libhustle_src_operators.a(expression.cc.o)
__ZN6hustle9operators10Expression12ExecuteBlockIN5arrow12NumericArrayINS3_8Int8TypeEEEaEENS3_5DatumEiSt10shared_ptrINS3_5ArrayEESA_SA_ in libhustle_src_operators.a(expression.cc.o)
__ZN6hustle9operators10Expression12ExecuteBlockIN5arrow12NumericArrayINS3_9Int16TypeEEEsEENS3_5DatumEiSt10shared_ptrINS3_5ArrayEESA_SA_ in libhustle_src_operators.a(expression.cc.o)
__ZN6hustle9operators10Expression12ExecuteBlockIN5arrow12NumericArrayINS3_9Int32TypeEEEiEENS3_5DatumEiSt10shared_ptrINS3_5ArrayEESA_SA_ in libhustle_src_operators.a(expression.cc.o)
__ZN6hustle9operators10Expression12ExecuteBlockIN5arrow12NumericArrayINS3_9Int64TypeEEExEENS3_5DatumEiSt10shared_ptrINS3_5ArrayEESA_SA_ in libhustle_src_operators.a(expression.cc.o)
...
"__ZN5arrow5DatumC1ESt10shared_ptrINS_12ChunkedArrayEE", referenced from:
__ZN6hustle9operators9LazyTable17MaterializeColumnEi in libhustle_src_operators.a(lazy_table.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEPNS_4TaskEiRN5arrow5DatumE in libhustle_src_operators.a(lazy_table.cc.o)
__ZZN6hustle7Context13apply_indicesEPNS_4TaskEN5arrow5DatumES4_S4_RS4_ENKUlS2_E0_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow5fieldENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS_8DataTypeEEbS6_IKNS_16KeyValueMetadataEE", referenced from:
__ZN24HashAggregateTestFixture5SetUpEv in hash_aggregate_test.cc.o
__Z11make_schemaRKN6hustle7catalog11TableSchemaE in libhustle_src_storage.a(util.cc.o)
__ZN6hustle9operators13HashAggregate12OutputSchemaENS0_15AggregateKernelERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow6StatusC1ENS_10StatusCodeERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN5arrow6Status8FromArgsIJRA46_KcRxRA2_S2_EEES0_NS_10StatusCodeEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow6Status8FromArgsIJRA36_KcRxRA19_S2_S5_RA2_S2_EEES0_NS_10StatusCodeEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow6Status8FromArgsIJRA32_KcxRA14_S2_RxEEES0_NS_10StatusCodeEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow6Status8FromArgsIJRA50_KcxRA22_S2_RxEEES0_NS_10StatusCodeEDpOT_ in hash_aggregate_test.cc.o
__ZN5arrow6Status8FromArgsIJRA46_KcRxRA2_S2_EEES0_NS_10StatusCodeEDpOT_ in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN5arrow6Status8FromArgsIJRA36_KcRxRA19_S2_S5_RA2_S2_EEES0_NS_10StatusCodeEDpOT_ in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN5arrow6Status8FromArgsIJRA32_KcxRA14_S2_RxEEES0_NS_10StatusCodeEDpOT_ in libhustle_src_operators.a(hash_aggregate.cc.o)
...
"__ZN5arrow6schemaESt6vectorISt10shared_ptrINS_5FieldEESaIS3_EES1_IKNS_16KeyValueMetadataEE", referenced from:
__ZN24HashAggregateTestFixture5SetUpEv in hash_aggregate_test.cc.o
"__ZN5arrow7struct_ERKSt6vectorISt10shared_ptrINS_5FieldEESaIS3_EE", referenced from:
__ZN6hustle9operators13HashAggregate10InitializeEPNS_4TaskE in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN6hustle7details10TestFieldsEN5arrow4Type4typeE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow8internal14DieWithMessageERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN5arrow6ResultISt10shared_ptrINS_12ArrayBuilderEEEC1ERKNS_6StatusE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow8internal19DictionaryMemoTable12GetArrayDataExPSt10shared_ptrINS_9ArrayDataEE", referenced from:
__ZN5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_9Int32TypeEE20FinishWithDictOffsetExPSt10shared_ptrINS_9ArrayDataEES8_ in libhustle_src_type.a(type_helper.cc.o)
__ZN5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_8Int8TypeEE20FinishWithDictOffsetExPSt10shared_ptrINS_9ArrayDataEES8_ in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow8internal19DictionaryMemoTableC1EPNS_10MemoryPoolERKSt10shared_ptrINS_8DataTypeEE", referenced from:
__ZN5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_8Int8TypeEEC2IS3_EENSt9enable_ifIXntsrSt10is_base_ofINS_19FixedSizeBinaryTypeET_E5valueERKSt10shared_ptrINS_8DataTypeEEE4typeEPNS_10MemoryPoolE in libhustle_src_type.a(type_helper.cc.o)
__ZN5arrow8internal21DictionaryBuilderBaseINS_18AdaptiveIntBuilderENS_9Int32TypeEEC2IS3_EENSt9enable_ifIXntsrSt10is_base_ofINS_19FixedSizeBinaryTypeET_E5valueERKSt10shared_ptrINS_8DataTypeEEE4typeEPNS_10MemoryPoolE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow9ArrayData4MakeERKSt10shared_ptrINS_8DataTypeEExSt6vectorIS1_INS_6BufferEESaIS8_EES6_IS1_IS0_ESaISB_EExx", referenced from:
__ZN5arrow15BaseListBuilderINS_13LargeListTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in libhustle_src_type.a(type_helper.cc.o)
__ZN5arrow15BaseListBuilderINS_8ListTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in libhustle_src_type.a(type_helper.cc.o)
"__ZN5arrow9ArrayData4MakeERKSt10shared_ptrINS_8DataTypeEExSt6vectorIS1_INS_6BufferEESaIS8_EExx", referenced from:
__ZN5arrow14NumericBuilderINS_10DoubleTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in hash_aggregate_test.cc.o
__ZN5arrow14NumericBuilderINS_9Int64TypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in hash_aggregate_test.cc.o
__ZN5arrow17BaseBinaryBuilderINS_10BinaryTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in hash_aggregate_test.cc.o
__Z17copy_record_batchRKSt10shared_ptrIN5arrow11RecordBatchEE in libhustle_src_storage.a(util.cc.o)
__ZN5arrow14NumericBuilderINS_10DoubleTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN5arrow14NumericBuilderINS_9Int64TypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN5arrow17BaseBinaryBuilderINS_10BinaryTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in libhustle_src_operators.a(hash_aggregate.cc.o)
...
"__ZN5arrow9MakeArrayERKSt10shared_ptrINS_9ArrayDataEE", referenced from:
__ZNK6hustle7storage5Block10get_columnEi in hash_aggregate_test.cc.o
__ZN6hustle7storage5Block9out_blockEPvPFiS2_iPPcS4_E in libhustle_src_storage.a(block.cc.o)
__ZN6hustle7storage5Block5printEv in libhustle_src_storage.a(block.cc.o)
__ZNK6hustle7storage5Block16get_valid_columnEv in libhustle_src_storage.a(table.cc.o)
__ZNK6hustle7storage5Block10get_columnEi in libhustle_src_operators.a(lazy_table.cc.o)
__ZN6hustle7Context26apply_indices_internal_strERKSt10shared_ptrIN5arrow12ChunkedArrayEERKS1_INS2_5ArrayEESA_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context23apply_indices_internal2IxEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_SF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
...
"__ZNK5arrow5Array6EqualsERKSt10shared_ptrIS0_ERKNS_12EqualOptionsE", referenced from:
__ZN38HashAggregateTestFixture_MeanTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN37HashAggregateTestFixture_SumTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN39HashAggregateTestFixture_CountTest_Test8TestBodyEv in hash_aggregate_test.cc.o
__ZN55HashAggregateTestFixture_SumWithGroupByOrderByTest_Test8TestBodyEv in hash_aggregate_test.cc.o
"__ZNK5arrow5Field8WithTypeERKSt10shared_ptrINS_8DataTypeEE", referenced from:
__ZN5arrow15BaseListBuilderINS_8ListTypeEEC2EPNS_10MemoryPoolERKSt10shared_ptrINS_12ArrayBuilderEERKS5_INS_8DataTypeEE in libhustle_src_type.a(type_helper.cc.o)
__ZN5arrow15BaseListBuilderINS_13LargeListTypeEEC2EPNS_10MemoryPoolERKSt10shared_ptrINS_12ArrayBuilderEERKS5_INS_8DataTypeEE in libhustle_src_type.a(type_helper.cc.o)
__ZNK5arrow15BaseListBuilderINS_13LargeListTypeEE4typeEv in libhustle_src_type.a(type_helper.cc.o)
__ZNK5arrow15BaseListBuilderINS_8ListTypeEE4typeEv in libhustle_src_type.a(type_helper.cc.o)
"__ZNK5arrow6Schema11field_namesB5cxx11Ev", referenced from:
__ZN6hustle7storage5Block9out_blockEPvPFiS2_iPPcS4_E in libhustle_src_storage.a(block.cc.o)
"__ZNK5arrow6Schema13GetFieldIndexERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN6hustle9operators9LazyTable17MaterializeColumnENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(lazy_table.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEPNS_4TaskENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN5arrow5DatumE in libhustle_src_operators.a(lazy_table.cc.o)
"__ZNK5arrow6Schema14GetFieldByNameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN6hustle9operators13HashAggregate10InitializeEPNS_4TaskE in libhustle_src_operators.a(hash_aggregate.cc.o)
__ZN6hustle9operators14OperatorResult11materializeERKSt6vectorINS0_15ColumnReferenceESaIS3_EEb in libhustle_src_operators.a(operator_result.cc.o)
"__ZNK5arrow6Status8ToStringB5cxx11Ev", referenced from:
__Z15evaluate_statusRKN5arrow6StatusEPKci in libhustle_src_storage.a(util.cc.o)
__ZN5arrow6ResultISt10shared_ptrINS_12ArrayBuilderEEEC1ERKNS_6StatusE in libhustle_src_type.a(type_helper.cc.o)
"__ZNK5arrow8DataType18ComputeFingerprintB5cxx11Ev", referenced from:
__ZTVN5arrow12BaseListTypeE in libhustle_src_type.a(type_helper.cc.o)
__ZTVN5arrow10NestedTypeE in libhustle_src_type.a(type_helper.cc.o)
"__ZNK5arrow8DataType26ComputeMetadataFingerprintB5cxx11Ev", referenced from:
__ZTVN5arrow12BaseListTypeE in libhustle_src_type.a(type_helper.cc.o)
__ZTVN5arrow10NestedTypeE in libhustle_src_type.a(type_helper.cc.o)
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make[3]: *** [src/operators/hustle_hash_aggregate_test] Error 1
make[2]: *** [src/operators/CMakeFiles/hustle_hash_aggregate_test.dir/all] Error 2
make[1]: *** [src/operators/CMakeFiles/hustle_hash_aggregate_test.dir/rule] Error 2
make: *** [hustle_hash_aggregate_test] Error 2
Select operator takes in arrow's table as an input and produces a filtered arrow table as an output. It should be able to support multiple predicates connected through ands and ors. The predicates could be on fields of different data types. Gandiva's expression tree evaluation should be used.
您好,我在https://stackoverflow.com/questions/58478390/blink-tree-insert-operation看到您回答了一个blink tree的问题。我不明白的是,为什么在parent层向右移动的时候,需要先拿右边的锁,然后再释放左边的锁。PostgreSQL实现blink tree的时候,没有采用这种chain lock的方式,而是直接先放左边的,然后拿右边的锁。我的邮箱是mengqingzhong.gm[at]gmail.com,期待您的回复。
The current metadata-attached block implementation is somewhat clunky overall, and may cause issues if further block extensions are implemented. Look into an interface-based system to make the overall design significantly more compartmentalized.
There is a problem with compiling the Catalog. I the following errors from linking sqlite3:
https://app.circleci.com/pipelines/github/UWQuickstep/hustle/12/workflows/4eae7dc5-21cd-4d7a-a00f-771f0d09917b/jobs/13. (The build report is from CircleCI)
Any ideas @lichengxihuang?
support custom data types (i.e. IP addrress was the running example at the time)
@johnchronis Im actually pretty confident that we can use the arrow::Extention
type in arrow to implement IP Address. In fact, this mysterious Extention
type is designed for user-defined type. The extention type can, in itself, defines its relationship with other types (ideally, of course, using what we have proposed ini #94).
In fact, arrow had a test IPv4
type in their code base:
https://github.com/apache/arrow/blob/2d140c3eeecca3ff7823edc8c9562ebd6a1c336a/cpp/src/arrow/array/array_view_test.cc#L55-L78
We can open up an issue to address user-defined type - will need to experiment a little bit on this.
Originally posted by @GindaChen in #28 (comment)
Currently, in the select operator, we do multithreading between different batches (i.e a set of blocks). In this task, we will experiment and see if we are getting speed-up by doing much fine multithreaded execution.
Ref:
Comparison operator call,
hustle/src/operators/select.cc
Line 95 in df14c96
hustle/src/operators/select.cc
Line 55 in df14c96
As the first step towards a multi-threaded implementation of Hustle operators, I will make selections multi-threaded. This will set the stage for multi-threaded implementations of the other operators. I'm seeing if we can reuse any code from the mini-version of Quickstep that UW-Madison submitted for the SIGMOD 2018 contest.
Per-table counters that record the effectiveness of metadata units. Must be thread-safe.
Procedure to reproduce
/usr/local/Cellar/gcc/10.2.0_4/bin/g++-10
git clone https://github.com/apache/arrow.git
cd arrow
git checkout apache-arrow-3.0.0
mkdir -p cpp/release
cd cpp/release
cmake -DARROW_COMPUTE=ON \
-DARROW_DEPENDENCY_SOURCE=AUTO \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_C_COMPILER=/path/to/gcc-10.1 \ # <- Your gcc10 Compiler path
-DCMAKE_CXX_COMPILER=/path/to/g++-10.1 \ # <- Your g++10 Compiler path
-DCMAKE_CXX_STANDARD_REQUIRED=ON \
-DCMAKE_CXX_STANDARD=17 ..
make -j install
-DCMAKE_BUILD_TYPE=Debug
-DCMAKE_C_COMPILER=gcc-10.1 # <- Your gcc10 Compiler path
-DCMAKE_CXX_COMPILER=g++-10.1 # <- Your g++10 Compiler path
-DCMAKE_CXX_STANDARD_REQUIRED=ON
-DCMAKE_CXX_STANDARD=17
Error Description
The dump fiile libarrow.a.dump.zip does not contains the mangled name that compiler generates. For example,
"__ZNK5arrow6Status8ToStringB5cxx11Ev", referenced from:
__Z15evaluate_statusRKN5arrow6StatusEPKci in libhustle_src_storage.a(util.cc.o)
is produced by the compiler, but the closest we got is
U __ZNK5arrow6Status8ToStringEv
To compare, the one built using g++ (first one) has an infix B5cxx11
that the second one (in libarrow) does not. libarrow
is bulit by clang (and will fail if built by g++), and is possible that it does not carry with some information in the ABI.
# Hustle-referenced symbol
__ZNK5arrow6Status8ToStringB5cxx11Ev
# Arrow symbol, produced by the dump
__ZNK5arrow6Status8ToStringEv
Complete Dump
libarrow.a dump file. Produced by
nm libarrow.a > ~/libarrow.a.dump
====================[ Build | hustle_select_test | Debug-GPP17 ]================
/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --build /Users/gindachen/Project/hustle/cmake-build-debug-gpp17 --target hustle_select_test -- -j 9
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[REG_EIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[REG_RIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.sc_ip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.uc_regs->gregs[PT_NIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.gregs[R15]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.arm_pc
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.mc_eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.mc_rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.__gregs[_REG_EIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext.__gregs[_REG_RIP]
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->ss.eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->__ss.__eip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->ss.rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Failed
-- Checking program counter fetch from ucontext_t member: uc_mcontext->__ss.__rip
-- Performing Test PC_FROM_UCONTEXT_COMPILES
-- Performing Test PC_FROM_UCONTEXT_COMPILES - Success
-- Found program counter field in ucontext_t: uc_mcontext->__ss.__rip
-- Arrow SO and ABI version: 300
-- Arrow full SO version: 300.0.0
-- Found the Arrow core library: /usr/local/lib/libarrow.dylib
-- Found the Arrow Python library: ARROW_PYTHON_LIB_PATH-NOTFOUND
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/gindachen/Project/hustle/cmake-build-debug-gpp17
[ 1%] Built target absl_log_severity
[ 3%] Built target hustle_src_scheduler_SchedulerTypedefs
[ 3%] Built target hustle_src_utils_meta_FunctionTraits
[ 3%] Built target gflags_nothreads-static
[ 5%] Built target absl_spinlock_wait
[ 7%] Built target gtest
[ 7%] Built target absl_exponential_biased
[ 9%] Built target hustle_src_utils_Macros
[ 13%] Built target glog
[ 15%] Built target hustle_src_utils_ThreadSafeQueue
[ 15%] Built target hustle_src_utils_ParallelHashmap
[ 17%] Built target hustle_src_utils_Config
[ 17%] Built target hustle_src_utils_ContainerUtil
[ 17%] Built target absl_city
[ 19%] Built target absl_civil_time
[ 19%] Built target hustle_src_type
[ 19%] Built target absl_int128
[ 23%] Built target absl_time_zone
[ 27%] Built target hustle_src_scheduler_WorkerMessage
[ 27%] Built target hustle_src_threading_ThreadUtil
[ 29%] Built target hustle_src_scheduler_SchedulerMessage
[ 29%] Built target hustle_src_threading_Mutex
[ 31%] Built target hustle_src_scheduler_SchedulerFlags
[ 33%] Built target hustle_src_scheduler_Node
[ 35%] Built target hustle_src_utils_SyncStream
[ 35%] Built target hustle_src_scheduler_SchedulerInterface
[ 37%] Built target hustle_src_scheduler_TaskName
[ 37%] Built target hustle_src_utils_EventProfiler
[ 37%] Built target gtest_main
[ 39%] Built target gmock
[ 39%] Built target absl_raw_logging_internal
[ 39%] Built target hustle_src_scheduler_TaskDescription
[ 43%] Built target absl_bad_optional_access
[ 43%] Built target absl_throw_delegate
[ 47%] Built target absl_base
[ 47%] Built target absl_bad_variant_access
[ 50%] Built target absl_strings_internal
[ 52%] Built target absl_debugging_internal
[ 54%] Built target hustle_src_scheduler_Task
[ 54%] Built target absl_malloc_internal
[ 54%] Built target absl_demangle_internal
[ 56%] Built target absl_stacktrace
[ 64%] Built target absl_strings
[ 64%] Built target hustle_src_scheduler_Worker
[ 64%] Built target hustle_src_optimizer_ExecutionPlan
[ 64%] Built target absl_graphcycles_internal
[ 68%] Built target absl_symbolize
[ 70%] Built target absl_cord
[ 70%] Built target absl_hash
[ 72%] Built target absl_time
[ 72%] Built target hustle_src_scheduler_Scheduler
[ 76%] Built target absl_synchronization
[ 78%] Built target hustle_src_utils_ArrowComputeWrappers
[ 80%] Built target absl_hashtablez_sampler
[ 80%] Built target absl_raw_hash_set
[ 88%] Built target hustle_src_operators
[ 92%] Built target hustle_src_storage
[ 94%] Built target hustle_src_resolver
[ 96%] Built target hustleDB
[ 98%] Built target hustle_src_catalog
[100%] Built target sqlite_utils
[100%] Built target sqlite3
[100%] Linking CXX executable hustle_select_test
ld: warning: dylib (/usr/local/lib/libconfig++.dylib) was built for newer macOS version (11.0) than being linked (10.15)
ld: warning: dylib (/usr/local/lib/libarrow.dylib) was built for newer macOS version (11.0) than being linked (10.15)
ld: warning: dylib (/usr/local/gcc-10.1.0/lib/libstdc++.dylib) was built for newer macOS version (10.16) than being linked (10.15)
Undefined symbols for architecture x86_64:
"__ZN5arrow11RecordBatch4MakeESt10shared_ptrINS_6SchemaEExSt6vectorIS1_INS_9ArrayDataEESaIS6_EE", referenced from:
__ZN6hustle7storage5Block11get_recordsEv in libhustle_src_storage.a(util.cc.o)
__Z17copy_record_batchRKSt10shared_ptrIN5arrow11RecordBatchEE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow12ArrayBuilder6FinishEPSt10shared_ptrINS_5ArrayEE", referenced from:
__ZN39SelectTestFixture_SingleSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN36SelectTestFixture_AndSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN35SelectTestFixture_OrSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN49SelectTestFixture_SingleSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
__ZN46SelectTestFixture_AndSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
__ZZN6hustle7Context13apply_indicesEPNS_4TaskEN5arrow5DatumES4_S4_RS4_ENKUlS2_E_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow12ChunkedArrayC1ESt6vectorISt10shared_ptrINS_5ArrayEESaIS4_EE", referenced from:
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in select_test.cc.o
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_storage.a(util.cc.o)
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_operators.a(select.cc.o)
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_operators.a(lazy_table.cc.o)
__ZN5arrow12ChunkedArrayC1ESt10shared_ptrINS_5ArrayEE in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN9__gnu_cxx13new_allocatorIN5arrow12ChunkedArrayEE9constructIS2_JRSt6vectorISt10shared_ptrINS1_5ArrayEESaIS8_EEEEEvPT_DpOT0_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow12StringScalarC1ENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN9__gnu_cxx13new_allocatorIN5arrow12StringScalarEE9constructIS2_JRA3_KcEEEvPT_DpOT0_ in select_test.cc.o
"__ZN5arrow13SchemaBuilder8AddFieldERKSt10shared_ptrINS_5FieldEE", referenced from:
__Z11make_schemaRKN6hustle7catalog11TableSchemaE in libhustle_src_storage.a(util.cc.o)
__ZN6hustle9operators14OperatorResult11materializeERKSt6vectorINS0_15ColumnReferenceESaIS3_EEb in libhustle_src_operators.a(operator_result.cc.o)
"__ZN5arrow15MakeArrayOfNullERKSt10shared_ptrINS_8DataTypeEExPNS_10MemoryPoolE", referenced from:
__Z21array_to_chunkedarraySt10shared_ptrIN5arrow5ArrayEEi in libhustle_src_storage.a(util.cc.o)
__ZN6hustle7Context26apply_indices_internal_strERKSt10shared_ptrIN5arrow12ChunkedArrayEERKS1_INS2_5ArrayEESA_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context23apply_indices_internal2IxEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_SF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context22apply_indices_internalIxEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context23apply_indices_internal2IjEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_SF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context22apply_indices_internalIjEEvRKSt10shared_ptrIN5arrow12ChunkedArrayEEPPKT_RKS2_INS3_5ArrayEESF_i in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow18RecordBatchBuilder4MakeERKSt10shared_ptrINS_6SchemaEEPNS_10MemoryPoolEPSt10unique_ptrIS0_St14default_deleteIS0_EE", referenced from:
__Z18read_from_csv_filePKcSt10shared_ptrIN5arrow6SchemaEEib in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow2io12ReadableFile4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPNS_10MemoryPoolE", referenced from:
__Z14read_from_filePKcbS0_ in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow2io16FileOutputStream4OpenERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb", referenced from:
__Z13write_to_filePKcRN6hustle7storage7DBTableE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow3ipc13NewFileWriterEPNS_2io12OutputStreamERKSt10shared_ptrINS_6SchemaEERKNS0_15IpcWriteOptionsERKS4_IKNS_16KeyValueMetadataEE", referenced from:
__Z13write_to_filePKcRN6hustle7storage7DBTableE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow3ipc21RecordBatchFileReader4OpenERKSt10shared_ptrINS_2io16RandomAccessFileEERKNS0_14IpcReadOptionsE", referenced from:
__Z14read_from_filePKcbS0_ in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow4util6detail19StringStreamWrapper3strB5cxx11Ev", referenced from:
__ZN5arrow4util13StringBuilderIJRA46_KcRxRA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in select_test.cc.o
__ZN5arrow4util13StringBuilderIJRA36_KcRxRA19_S2_S5_RA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in select_test.cc.o
__ZN5arrow4util13StringBuilderIJRA50_KcxRA22_S2_RxEEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in select_test.cc.o
__ZN5arrow4util13StringBuilderIJRA46_KcRxRA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow4util13StringBuilderIJRA36_KcRxRA19_S2_S5_RA2_S2_EEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow4util13StringBuilderIJRA24_KcEEENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEDpOT_ in libhustle_src_storage.a(sma.cc.o)
"__ZN5arrow5DatumC1ERKSt10shared_ptrINS_5ArrayEE", referenced from:
__ZN6hustle9operators6Select6FilterERKSt10shared_ptrINS_7storage5BlockEERKS2_INS0_9PredicateEE in libhustle_src_operators.a(select.cc.o)
__ZN5arrow5DatumC1INS_12BooleanArrayEvEERKSt10shared_ptrIT_E in libhustle_src_operators.a(select.cc.o)
__ZZZN6hustle7Context5matchEPNS_4TaskERKN5arrow5DatumES6_ENKUlS2_E_clES2_ENKUlvE_clEv in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7Context18apply_filter_blockERKSt10shared_ptrIN5arrow5ArrayEES6_RSt6vectorIS4_SaIS4_EE in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZZN6hustle7Context12apply_filterEPNS_4TaskERKN5arrow5DatumES6_RS4_ENKUlS2_E_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN6hustle7storage3SmaC2ERKSt10shared_ptrIN5arrow5ArrayEE in libhustle_src_storage.a(sma.cc.o)
__ZN6hustle7storage3SmaC1ERKSt10shared_ptrIN5arrow5ArrayEE in libhustle_src_storage.a(sma.cc.o)
...
"__ZN5arrow5DatumC1ESt10shared_ptrINS_12ChunkedArrayEE", referenced from:
__ZN9__gnu_cxx13new_allocatorIN6hustle9operators9LazyTableEE9constructIS3_JRSt10shared_ptrINS1_7storage7DBTableEERS6_IN5arrow12ChunkedArrayEENSB_5DatumESF_EEEvPT_DpOT0_ in libhustle_src_operators.a(select.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEi in libhustle_src_operators.a(lazy_table.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEPNS_4TaskEiRN5arrow5DatumE in libhustle_src_operators.a(lazy_table.cc.o)
__ZZN6hustle7Context13apply_indicesEPNS_4TaskEN5arrow5DatumES4_S4_RS4_ENKUlS2_E0_clES2_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
"__ZN5arrow5fieldENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS_8DataTypeEEbS6_IKNS_16KeyValueMetadataEE", referenced from:
__ZN17SelectTestFixture5SetUpEv in select_test.cc.o
__Z11make_schemaRKN6hustle7catalog11TableSchemaE in libhustle_src_storage.a(util.cc.o)
"__ZN5arrow6StatusC1ENS_10StatusCodeERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN5arrow6Status8FromArgsIJRA46_KcRxRA2_S2_EEES0_NS_10StatusCodeEDpOT_ in select_test.cc.o
__ZN5arrow6Status8FromArgsIJRA36_KcRxRA19_S2_S5_RA2_S2_EEES0_NS_10StatusCodeEDpOT_ in select_test.cc.o
__ZN5arrow6Status8FromArgsIJRA50_KcxRA22_S2_RxEEES0_NS_10StatusCodeEDpOT_ in select_test.cc.o
__ZN5arrow6Status8FromArgsIJRA46_KcRxRA2_S2_EEES0_NS_10StatusCodeEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow6Status8FromArgsIJRA36_KcRxRA19_S2_S5_RA2_S2_EEES0_NS_10StatusCodeEDpOT_ in libhustle_src_utils_ArrowComputeWrappers.a(arrow_compute_wrappers.cc.o)
__ZN5arrow6Status8FromArgsIJRA24_KcEEES0_NS_10StatusCodeEDpOT_ in libhustle_src_storage.a(sma.cc.o)
"__ZN5arrow6schemaESt6vectorISt10shared_ptrINS_5FieldEESaIS3_EES1_IKNS_16KeyValueMetadataEE", referenced from:
__ZN17SelectTestFixture5SetUpEv in select_test.cc.o
"__ZN5arrow9ArrayData4MakeERKSt10shared_ptrINS_8DataTypeEExSt6vectorIS1_INS_6BufferEESaIS8_EExx", referenced from:
__ZN5arrow14NumericBuilderINS_10DoubleTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in select_test.cc.o
__ZN5arrow14NumericBuilderINS_9Int64TypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in select_test.cc.o
__ZN5arrow17BaseBinaryBuilderINS_10BinaryTypeEE14FinishInternalEPSt10shared_ptrINS_9ArrayDataEE in select_test.cc.o
__Z17copy_record_batchRKSt10shared_ptrIN5arrow11RecordBatchEE in libhustle_src_storage.a(util.cc.o)
__ZN6hustle7storage5BlockC2EiRKSt10shared_ptrIN5arrow6SchemaEEib in libhustle_src_storage.a(block.cc.o)
__ZN6hustle7storage5BlockC1EiRKSt10shared_ptrIN5arrow6SchemaEEib in libhustle_src_storage.a(block.cc.o)
__ZN6hustle7storage5BlockC2EiSt10shared_ptrIN5arrow11RecordBatchEEib in libhustle_src_storage.a(block.cc.o)
...
"__ZN5arrow9MakeArrayERKSt10shared_ptrINS_9ArrayDataEE", referenced from:
__ZNK6hustle7storage5Block10get_columnEi in select_test.cc.o
__ZNK6hustle7storage5Block18get_column_by_nameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(select.cc.o)
__ZNK6hustle7storage5Block10get_columnEi in libhustle_src_storage.a(ma_block.cc.o)
__ZN6hustle7storage5Block9out_blockEPvPFiS2_iPPcS4_E in libhustle_src_storage.a(block.cc.o)
__ZN6hustle7storage5Block5printEv in libhustle_src_storage.a(block.cc.o)
__ZNK6hustle7storage5Block16get_valid_columnEv in libhustle_src_storage.a(table.cc.o)
__ZNK6hustle7storage5Block10get_columnEi in libhustle_src_operators.a(lazy_table.cc.o)
...
"__ZNK5arrow5Array6EqualsERKSt10shared_ptrIS0_ERKNS_12EqualOptionsE", referenced from:
__ZN39SelectTestFixture_SingleSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN36SelectTestFixture_AndSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN35SelectTestFixture_OrSelectTest_Test8TestBodyEv in select_test.cc.o
__ZN49SelectTestFixture_SingleSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
__ZN46SelectTestFixture_AndSelectManyBlocksTest_Test8TestBodyEv in select_test.cc.o
"__ZNK5arrow6Schema11field_namesB5cxx11Ev", referenced from:
__ZN6hustle7storage5Block9out_blockEPvPFiS2_iPPcS4_E in libhustle_src_storage.a(block.cc.o)
"__ZNK5arrow6Schema13GetFieldIndexERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZNK6hustle7storage5Block18get_column_by_nameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(select.cc.o)
__ZN6hustle7storage21MetadataAttachedBlock14SearchMetadataERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKN5arrow5DatumENSA_7compute15CompareOperatorE in libhustle_src_operators.a(select.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE in libhustle_src_operators.a(lazy_table.cc.o)
__ZN6hustle9operators9LazyTable17MaterializeColumnEPNS_4TaskENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN5arrow5DatumE in libhustle_src_operators.a(lazy_table.cc.o)
"__ZNK5arrow6Schema14GetFieldByNameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE", referenced from:
__ZN6hustle9operators14OperatorResult11materializeERKSt6vectorINS0_15ColumnReferenceESaIS3_EEb in libhustle_src_operators.a(operator_result.cc.o)
"__ZNK5arrow6Status8ToStringB5cxx11Ev", referenced from:
__Z15evaluate_statusRKN5arrow6StatusEPKci in libhustle_src_storage.a(util.cc.o)
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make[3]: *** [src/operators/hustle_select_test] Error 1
make[2]: *** [src/operators/CMakeFiles/hustle_select_test.dir/all] Error 2
make[1]: *** [src/operators/CMakeFiles/hustle_select_test.dir/rule] Error 2
make: *** [hustle_select_test] Error 2
cmake 3.15 has a few nice features that we would like to use.
But it is not the default cmake version on most linux distros.
We should find an easy way to install cmake 3.15 or upgrade the older versions
Integrate the code that converts SQL to execution plan using Sqlite to HustleDB
There are around 500 simple issues in some of the important modules, as we planned we can slowly bring this number down, so we will be able to integrate with CircleCI and provide a warning on linter issues on merge.
The parser, catalog, resolver, HustleDB tests are flaky and fail with a seg fault.
(The test are currently disabled)
To reproduce run all tests with ctest.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.