grammatech / gtirb Goto Github PK
View Code? Open in Web Editor NEWIntermediate Representation for Binary analysis and transformation
Home Page: https://grammatech.github.io/gtirb/
License: Other
Intermediate Representation for Binary analysis and transformation
Home Page: https://grammatech.github.io/gtirb/
License: Other
I compiled the latest code(1ded8bd3e62b216d237c1df02b0e9f60470dc062) of gtirb and then encountered the following error. I checked out the tag v1.10.4 and it works well.
error: ‘SymStackConst’ is not a member of ‘gtirb’; did you mean ‘gtirb::proto::SymStackConst’?
210 | std::is_same_v<T, gtirb::SymStackConst>) {
❯ sudo cmake --build ./gtirb-pprinter/build -j --target all install
Scanning dependencies of target gtirb_layout
Scanning dependencies of target gtest
Scanning dependencies of target gtirb_pprinter
[ 4%] Building CXX object src/gtirb_layout/CMakeFiles/gtirb_layout.dir/gtirb_layout.cpp.o
[ 8%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/AttPrettyPrinter.cpp.o
[ 16%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/PrettyPrinter.cpp.o
[ 16%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/ElfBinaryPrinter.cpp.o
[ 20%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/Arm64PrettyPrinter.cpp.o
[ 24%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/BinaryPrinter.cpp.o
[ 28%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/ElfPrettyPrinter.cpp.o
[ 32%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/file_utils.cpp.o
[ 40%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/Registration.cpp.o
[ 40%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/IntelPrettyPrinter.cpp.o
[ 44%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/string_utils.cpp.o
[ 48%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/Syntax.cpp.o
[ 52%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/PePrettyPrinter.cpp.o
[ 64%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/MasmPrettyPrinter.cpp.o
[ 64%] Building CXX object src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/PeBinaryPrinter.cpp.o
[ 64%] Building CXX object googletest-build/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 68%] Linking CXX shared library ../../lib/libgtest.so
[ 68%] Built target gtest
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp: In lambda function:
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:135:61: error: ‘SymStackConst’ is not a member of ‘gtirb’; did you mean ‘gtirb::proto::SymStackConst’?
135 | std::is_same_v<T, gtirb::SymStackConst>) {
| ^~~~~~~~~~~~~
In file included from /usr/local/include/gtirb/SymbolicExpression.hpp:19,
from /usr/local/include/gtirb/ByteInterval.hpp:22,
from /usr/local/include/gtirb/gtirb.hpp:27,
from /data/username/ddisasm/deps/gtirb-pprinter/include/gtirb_pprinter/PrettyPrinter.hpp:21,
from /data/username/ddisasm/deps/gtirb-pprinter/include/gtirb_pprinter/ElfPrettyPrinter.hpp:18,
from /data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:15:
/usr/local/include/gtirb/proto/SymbolicExpression.pb.h:110:7: note: ‘gtirb::proto::SymStackConst’ declared here
110 | class SymStackConst : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:gtirb.proto.SymStackConst) */ {
| ^~~~~~~~~~~~~
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp: In lambda function:
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:210:59: error: ‘SymStackConst’ is not a member of ‘gtirb’; did you mean ‘gtirb::proto::SymStackConst’?
210 | std::is_same_v<T, gtirb::SymStackConst>) {
| ^~~~~~~~~~~~~
In file included from /usr/local/include/gtirb/SymbolicExpression.hpp:19,
from /usr/local/include/gtirb/ByteInterval.hpp:22,
from /usr/local/include/gtirb/gtirb.hpp:27,
from /data/username/ddisasm/deps/gtirb-pprinter/include/gtirb_pprinter/PrettyPrinter.hpp:21,
from /data/username/ddisasm/deps/gtirb-pprinter/include/gtirb_pprinter/ElfPrettyPrinter.hpp:18,
from /data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:15:
/usr/local/include/gtirb/proto/SymbolicExpression.pb.h:110:7: note: ‘gtirb::proto::SymStackConst’ declared here
110 | class SymStackConst : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:gtirb.proto.SymStackConst) */ {
| ^~~~~~~~~~~~~
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp: In lambda function:
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:236:59: error: ‘SymStackConst’ is not a member of ‘gtirb’; did you mean ‘gtirb::proto::SymStackConst’?
236 | std::is_same_v<T, gtirb::SymStackConst>) {
| ^~~~~~~~~~~~~
In file included from /usr/local/include/gtirb/SymbolicExpression.hpp:19,
from /usr/local/include/gtirb/ByteInterval.hpp:22,
from /usr/local/include/gtirb/gtirb.hpp:27,
from /data/username/ddisasm/deps/gtirb-pprinter/include/gtirb_pprinter/PrettyPrinter.hpp:21,
from /data/username/ddisasm/deps/gtirb-pprinter/include/gtirb_pprinter/ElfPrettyPrinter.hpp:18,
from /data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:15:
/usr/local/include/gtirb/proto/SymbolicExpression.pb.h:110:7: note: ‘gtirb::proto::SymStackConst’ declared here
110 | class SymStackConst : public ::google::protobuf::Message /* @@protoc_insertion_point(class_definition:gtirb.proto.SymStackConst) */ {
| ^~~~~~~~~~~~~
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp: In lambda function:
/data/username/ddisasm/deps/gtirb-pprinter/src/gtirb_pprinter/ElfPrettyPrinter.cpp:138:13: error: control reaches end of non-void function [-Werror=return-type]
138 | },
| ^
cc1plus: all warnings being treated as errors
make[2]: *** [src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/build.make:115: src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/ElfPrettyPrinter.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 72%] Linking CXX shared library ../../lib/libgtirb_layout.so
[ 72%] Built target gtirb_layout
Scanning dependencies of target gtirb-layout
Scanning dependencies of target TestGtirbLayout
[ 76%] Building CXX object src/gtirb_layout/driver/CMakeFiles/gtirb-layout.dir/gtirb_layout.cpp.o
[ 80%] Building CXX object src/gtirb_layout/test/CMakeFiles/TestGtirbLayout.dir/layout_test.cpp.o
make[1]: *** [CMakeFiles/Makefile2:467: src/gtirb_pprinter/CMakeFiles/gtirb_pprinter.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 84%] Linking CXX executable ../../../bin/gtirb-layout
[ 84%] Built target gtirb-layout
[ 88%] Linking CXX executable ../../../bin/TestGtirbLayout
[ 88%] Built target TestGtirbLayout
make: *** [Makefile:163: all] Error 2
I saw the gtirb vscode extension at https://grammatech.github.io/prj/gtirb-vscode/. That's pretty cool! When will this work be available to us?
Node::uuidMap is not thread safe, but users have no way to protect against data races against it. I know we aren't intending to use multithreading this library, but we shouldn't provide unsafe APIs that prevent the user from doing their own concurrency.
I think we should design the APIs such that the user can use the APIs in a thread-safe manner, even if we don't use multiple threads ourselves.
I noticed you have a plugin for Ghidra, but it is not the only one FOSS tool available.
Radare2 is a highly-portable cross-platform reverse engineering framework and a toolkit without dependencies. It has support for analyzing binaries, disassembling code, debugging programs, attaching to remote GDB/LLDB, WinDbg servers, rich plugin system (see r2pm
), and integration with various decompilers. For example, ghidra decompiler plugin - r2ghidra-dec. It is actively developed and can be easily integrated in various open source and commercial products. I believe, it will be highly beneficial to support these and provide a package for install from r2pm
, see the package repository here: https://github.com/radareorg/radare2-pm
For documentation on writing plugins for radare2 see Scripting and Plugins Radare2 Book chapters.
Cutter is a crossplatform Qt/C++ GUI frontend to radare2:
For documentation on writing plugins for Cutter see the official tutorial and the curated list of various popular plugins.
The compilation speed is ridiculously slow due to the dependency on boost.
Precompiled header can improve the situation a lot.
I am trying to insert assembly instructions containing labels into a GTIRB file. Specifically, I want to insert a jmp label
, where the label is added using the GTIRB addSymbol()
function.
Currently, the method I found for inserting assembly instructions involves converting the instructions into binary sequences using Keystone and then using the provided insertBytes()
function. However, there is an issue because Keystone cannot handle labels within the assembly instructions.
Therefore, I would like to know if there are any other ways in GTIRB that I might have overlooked to achieve the insertion of assembly instructions containing labels into a GTIRB file?
[100%] Built target googletest -- Found PythonInterp: /usr/bin/python (found version "2.7.17") -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Found Protobuf: /usr/local/lib/libprotobuf.so;-lpthread (found suitable version "3.12.3", minimum required is "3.0.0") -- clang-tidy not found. -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) CMake Warning at doc/python/CMakeLists.txt:30 (message): sphinx-apidoc was not found. HTML documentation for the Python API will not be generated. -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) (Required is at least version "1.8.11") APIs to be built: C++ ON Python ON Lisp OFF Java OFF -- Configuring incomplete, errors occurred! See also "/build/CMakeFiles/CMakeOutput.log". See also "/build/CMakeFiles/CMakeError.log". CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: Boost_INCLUDE_DIR (ADVANCED) used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/gtirb used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googlemock used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/googletest-src/googletest used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/proto used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/gtirb/proto used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/src/test/testInputBinary used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/python used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/cpp used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/python used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/examples used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general used as include directory in directory /build/gtirb/doc/general
wasn't getting this error earlier. But now when I tried to make a fresh build its showing this error.
Hello Everyone
Is there any inbuilt Value Set Analysis implementation available for GTIRB? If yes can anyone point me to the repo? If not, is it possible for making use of Angr's analysis tools to work on GTIRB?
Thanking you
Sai
[ 70%] Built target gtest_main
/home/akochkov/tmp/gtirb/src/CFG.cpp:24:54: warning: statement should be inside braces [readability-braces-around-statements]
if (auto it = IdTable.find(N); it != IdTable.end())
^
{
/home/akochkov/tmp/gtirb/src/CFG.cpp:36:54: warning: statement should be inside braces [readability-braces-around-statements]
if (auto it = IdTable.find(N); it != IdTable.end())
^
{
/home/akochkov/tmp/gtirb/src/CFG.cpp:107:5: warning: use auto when initializing with a template cast to avoid duplicating the type name [modernize-use-auto]
CfgNode* N =
^~~~~~~
auto
/home/akochkov/tmp/gtirb/src/Module.cpp:29:28: warning: pass by value and use std::move [modernize-pass-by-value]
Module::Module(Context& C, const std::string& X)
^~~~~~~~~~~~~~~~~~
std::string
/home/akochkov/tmp/gtirb/src/Module.cpp:40:7: warning: use auto when initializing with a template cast to avoid duplicating the type name [modernize-use-auto]
if (Block* B = dyn_cast<Block>(N))
^~~~~
auto
/home/akochkov/tmp/gtirb/src/Module.cpp:40:37: warning: statement should be inside braces [readability-braces-around-statements]
if (Block* B = dyn_cast<Block>(N))
^
{
/home/akochkov/tmp/gtirb/src/Module.cpp:42:12: warning: use auto when initializing with a template cast to avoid duplicating the type name [modernize-use-auto]
else if (ProxyBlock* P = dyn_cast<ProxyBlock>(N))
^~~~~~~~~~
auto
/home/akochkov/tmp/gtirb/src/Module.cpp:42:52: warning: statement should be inside braces [readability-braces-around-statements]
else if (ProxyBlock* P = dyn_cast<ProxyBlock>(N))
^
{
/home/akochkov/tmp/gtirb/src/Module.cpp:44:7: warning: statement should be inside braces [readability-braces-around-statements]
else
^
{
/home/akochkov/tmp/gtirb/src/Module.cpp:93:43: warning: statement should be inside braces [readability-braces-around-statements]
for (const auto& Elt : Message.blocks())
^
{
/home/akochkov/tmp/gtirb/src/Module.cpp:95:41: warning: statement should be inside braces [readability-braces-around-statements]
for (const auto& Elt : Message.data())
^
{
/home/akochkov/tmp/gtirb/src/Module.cpp:97:44: warning: statement should be inside braces [readability-braces-around-statements]
for (const auto& Elt : Message.proxies())
^
{
/home/akochkov/tmp/gtirb/src/Module.cpp:99:45: warning: statement should be inside braces [readability-braces-around-statements]
for (const auto& Elt : Message.sections())
^
{
in cl/gtirb.lisp it computes the path pf version.txt using a form that misses parts of the pathname. In prticular, in LispWorks for Windows the host has the driver letter, so it doesn't work without it.
The right way to compute relative pathname is to use asdf:system-relative-pathname, i.e. to replace the call make-pathname inside defvar version.txt by:
(asdf/system:system-relative-pathname "gtirb" "../version.txt")
.
The same apply to *gtirb-dir* in cl/test.lisp, which should be:
(defvar *gtirb-dir* (asdf/system:system-relative-pathname "gtirb" "../" ))
I notice that some classes seem to be constructed in an incomplete state and I'm wondering why they have so many ways to mutate their data. For instance, Module
and ImageByteMap
let you get and set all of the data members but only have a default constructor.
Relatedly, why is Node::setUUID() public? It seems to me like the version accepting no arguments shouldn't exist at all (otherwise, you can trivially change the underlying identity of an object that's being tracked by that UUID which seems like a dangerous operation), and the version accepting a UUID should be a friend of serialization (perhaps only setNodUUIDFromBytes()).
This is a bad code smell, so I'm wondering what the design rationale was. Anything we can do to reduce the mutability of the interfaces makes the code easier to use because it's hard to get objects in an inconsistent state that way.
The test idempotent-aux-data-decode-encode compares the result of gtirb::aux-data-encode to the original data. But if the data argument to gtirb::aux-data-encode is a hash-table, it goes through the elements using maphash (inside encode). The order in which maphash goes through the elements is uspecified, and can be different between calls. Therefore this test sometimes fails in this situation.
If the order of the elements needs to be fixd, then encode needs to be fixed to sort the table first somehow, or never use a hash-table for the data. If the order does not need to be fixed, then the test needs to be fixed to not test for it.
Build fails in TestGTIRB.dir/AuxData.test.cpp.o
at BOOST_STATIC_ASSERT(detail::is_endian_reversible<EndianReversible>::value);
Using the depository at https://download.grammatech.com/gtirb/files/apt-repo, the packages updates don't trigger the removal of the old packages, causing conflicts where different versions of ddisasm, gtirb, and GrammaTech's libcapstone-dev-4.0.2-gt?
have the same file paths. To be able to install the newer packages, I had to first uninstall the old ones, then select the new ones.
OS: Ubuntu 20.04
I would like to trace data from the stack memory with static analysis. Does gtirb support that? If it does not, what else I could look at?
Thank you.
Hello Everyone
Is there any way to associate a function to a different section? For now, the function attributes do not have a section name, based on my understanding. Can anyone provide any pointers?
Thanking you
Sai
in cl/gtirb.lisp, there is a definition that starts (line 164):
(defmethod :around is-equal-p-internal ((left t) (right t))
That defines a method for tha :around generic function, with qualifier is-equal-p-internal. LispWorks errors on that. Presumably the tokens are swicthed, and it should be:
(defmethod is-equal-p-internal :around ((left t) (right t))
We currently download some third-party packages automatically for the user (googletest and gsl). This is a nice thing when getting started on the project as it makes it easier to get the source code, compile, and start playing with it. However, in places where this can touch the API, it introduces potential problems where the end user's project is using GSL and we download an ABI-incompatible version internally and use it. We have interfaces like ByteMap::setData()
that work with GSL interfaces and this would cause hard to understand compile errors in the best case, or silent ODR violations and subtly broken code at runtime in the worst case.
We should figure out whether these are prerequisites for the project (so the user is responsible for supplying them) or implementation details for the project (so the user is generally unaware of their presence and we can download them ourselves). For instance, we may decide to download these only if they're not already present on the system (with some version checks?), never download them and only enable the feature if they're present (such as disable building tests if googletest cannot be found), or some other approach.
Hi,
I completed the common build process but run into errors when attempting to import the gtirb libraries. The first error is <gtirb/gtirb.hpp> does not exist in my default include paths. I'm not sure if this is by design but using -I /path/to/GTIRB/include/
fixes that issue.
Next I receive what appears to be boost issues related to CFG.hpp:
$ clang++ -I ~/GTIRB/include/ populating_ir.cpp
In file included from populating_ir.cpp:12:
In file included from ~/GTIRB/include/gtirb/gtirb.hpp:25:
In file included from ~/GTIRB/include/gtirb/AuxData.hpp:19:
In file included from ~/GTIRB/include/gtirb/Block.hpp:19:
~/GTIRB/include/gtirb/CFG.hpp:71:5: error: no template named 'optional' in namespace 'std'; did you mean 'boost::optional'?
std::optional<std::tuple<ConditionalEdge, DirectEdge, EdgeType>>;
^~~~~~~~~~~~~
boost::optional
/usr/local/include/boost/optional/optional.hpp:845:7: note: 'boost::optional' declared here
class optional
^
This repeats four times. There appears to be a conflict between 'std' and 'boost' that is exclusive to Clang. I tried g++ and it returned "compilation terminated."
GSL requires you to pass -fno-strict-aliasing in Clang and GCC in order to avoid potential issues stemming from undefined behavior around strict aliasing, and we do this in the CMake files.
However, the protobuf-generated header files (Foo.pb.h) all also ignore strict aliasing warnings using #pragma GCC diagnostic ignored "-Wstrict-aliasing"
. I think we need to disable strict aliasing optimizations here as well.
However, it's really unfortunate to have to disable compiler features because we want to rely on undefined behavior. This makes me worry for the other compilers our end users use when compiling the library and what users should do if that UB causes them runtime problems.
Hello!
Just wondering if Cap'n'proto was considered, and if so why protobuf was chosen over it? Assuming there is no design limitation for it, I would like to add support for it to GTIRB. Is there anywhere in particular you all would recommend I start in the codebase to accomplish that?
Hi,
I tried to compile the latest commit (40c9b83) and it fails. The compiler points out that AuxData.hpp
used std::set
without including it.
Error:
In file included from /root/gtirb/src/AuxData.cpp:15:
/root/gtirb/include/gtirb/AuxData.hpp:389:53: error: 'set' is not a member of 'std'
389 | template <class... Args> struct auxdata_traits<std::set<Args...>> {
| ^~~
/root/gtirb/include/gtirb/AuxData.hpp:25:1: note: 'std::set' is defined in header '<set>'; did you forget to '#include <set>'?
24 | #include <list>
+++ |+#include <set>
25 | #include <map>
/root/gtirb/include/gtirb/AuxData.hpp:389:61: error: expected parameter pack before '...'
389 | template <class... Args> struct auxdata_traits<std::set<Args...>> {
| ^~~
/root/gtirb/include/gtirb/AuxData.hpp:389:61: error: template argument 1 is invalid
/root/gtirb/include/gtirb/AuxData.hpp:389:64: error: expected unqualified-id before '>' token
389 | template <class... Args> struct auxdata_traits<std::set<Args...>> {
| ^~
make[2]: *** [src/CMakeFiles/gtirb.dir/build.make:63: src/CMakeFiles/gtirb.dir/AuxData.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:449: src/CMakeFiles/gtirb.dir/all] Error 2
make: *** [Makefile:163: all] Error 2
Compilation command (used in dockerfile):
RUN git clone https://github.com/GrammaTech/gtirb.git && \
cd gtirb && \
git checkout 40c9b835c2982d57f6c33f506fe87f660c7f93db && \
cmake -B build -DGTIRB_JAVA_API=OFF -DGTIRB_PY_API=OFF -DGTIRB_CL_API=OFF . && \
cd build && \
make install && \
ldconfig && \
cd ~
Cloning into 'gtirb'...
Note: switching to '282161cd1e9f81da6a31eddb2d2e7ed746d41274'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 282161cd No need to explicitly install libgtirb
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Boost: /usr/local/lib/cmake/Boost-1.81.0/BoostConfig.cmake (found suitable version "1.81.0", minimum required is "1.67")
-- Configuring done
-- Generating done
-- Build files have been written to: /root/gtirb/build/googletest-download
Scanning dependencies of target googletest
[ 11%] Creating directories for 'googletest'
[ 22%] Performing download step (git clone) for 'googletest'
Cloning into 'googletest-src'...
Note: switching to 'release-1.10.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 703bd9ca Googletest export
[ 33%] No patch step for 'googletest'
[ 44%] Performing update step for 'googletest'
[ 55%] No configure step for 'googletest'
[ 66%] No build step for 'googletest'
[ 77%] No install step for 'googletest'
[ 88%] No test step for 'googletest'
[100%] Completed 'googletest'
[100%] Built target googletest
-- Found PythonInterp: /usr/bin/python (found version "2.7.18")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found Protobuf: /usr/local/lib/libprotobuf.so;-lpthread (found suitable version "3.20.3", minimum required is "3.0.0")
-- clang-tidy not found.
-- Looking for include file sys/resource.h
-- Looking for include file sys/resource.h - found
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.17") found components: doxygen missing components: dot
-- Found Python3: /usr/bin/python3.8 (found version "3.8.10") found components: Interpreter
-- Found Doxygen: /usr/bin/doxygen (found suitable version "1.8.17", minimum required is "1.8.11") found components: doxygen missing components: dot
APIs to be built:
C++ ON
Python OFF
Lisp OFF
Java OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /root/gtirb/build
[ 1%] Running cpp protocol buffer compiler on /root/gtirb/proto/SymbolicExpression.proto
[ 2%] Generating ../../../include/gtirb/proto/SymbolicExpression.pb.h
[ 4%] Running cpp protocol buffer compiler on /root/gtirb/proto/AuxData.proto
[ 4%] Running cpp protocol buffer compiler on /root/gtirb/proto/ByteInterval.proto
[ 5%] Running cpp protocol buffer compiler on /root/gtirb/proto/CFG.proto
[ 7%] Running cpp protocol buffer compiler on /root/gtirb/proto/CodeBlock.proto
[ 7%] Running cpp protocol buffer compiler on /root/gtirb/proto/DataBlock.proto
[ 8%] Running cpp protocol buffer compiler on /root/gtirb/proto/IR.proto
[ 10%] Running cpp protocol buffer compiler on /root/gtirb/proto/Module.proto
[ 10%] Running cpp protocol buffer compiler on /root/gtirb/proto/Offset.proto
[ 11%] Running cpp protocol buffer compiler on /root/gtirb/proto/ProxyBlock.proto
[ 12%] Running cpp protocol buffer compiler on /root/gtirb/proto/Section.proto
[ 12%] Running cpp protocol buffer compiler on /root/gtirb/proto/Symbol.proto
[ 14%] Generating ../../../include/gtirb/proto/AuxData.pb.h
[ 14%] Generating ../../../include/gtirb/proto/ByteInterval.pb.h
[ 15%] Generating ../../../include/gtirb/proto/CFG.pb.h
[ 17%] Generating ../../../include/gtirb/proto/CodeBlock.pb.h
[ 17%] Generating ../../../include/gtirb/proto/DataBlock.pb.h
[ 18%] Generating ../../../include/gtirb/proto/IR.pb.h
[ 20%] Generating ../../../include/gtirb/proto/Module.pb.h
[ 20%] Generating ../../../include/gtirb/proto/Offset.pb.h
[ 21%] Generating ../../../include/gtirb/proto/ProxyBlock.pb.h
[ 21%] Generating ../../../include/gtirb/proto/Section.pb.h
[ 22%] Generating ../../../include/gtirb/proto/Symbol.pb.h
Scanning dependencies of target gtirb_proto
[ 22%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/AuxData.pb.cc.o
[ 24%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/ByteInterval.pb.cc.o
[ 25%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/CFG.pb.cc.o
[ 25%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/CodeBlock.pb.cc.o
[ 27%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/DataBlock.pb.cc.o
[ 28%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/IR.pb.cc.o
[ 28%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/Module.pb.cc.o
[ 30%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/Offset.pb.cc.o
[ 31%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/ProxyBlock.pb.cc.o
[ 31%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/Section.pb.cc.o
[ 32%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/Symbol.pb.cc.o
[ 34%] Building CXX object src/gtirb/proto/CMakeFiles/gtirb_proto.dir/SymbolicExpression.pb.cc.o
[ 34%] Linking CXX static library ../../../lib/libgtirb_proto.a
[ 34%] Built target gtirb_proto
Scanning dependencies of target gtirb
[ 35%] Building CXX object src/CMakeFiles/gtirb.dir/AuxData.cpp.o
In file included from /root/gtirb/src/AuxData.cpp:15:
/root/gtirb/include/gtirb/AuxData.hpp:389:53: error: 'set' is not a member of 'std'
389 | template <class... Args> struct auxdata_traits<std::set<Args...>> {
| ^~~
/root/gtirb/include/gtirb/AuxData.hpp:25:1: note: 'std::set' is defined in header '<set>'; did you forget to '#include <set>'?
24 | #include <list>
+++ |+#include <set>
25 | #include <map>
/root/gtirb/include/gtirb/AuxData.hpp:389:61: error: expected parameter pack before '...'
389 | template <class... Args> struct auxdata_traits<std::set<Args...>> {
| ^~~
/root/gtirb/include/gtirb/AuxData.hpp:389:61: error: template argument 1 is invalid
/root/gtirb/include/gtirb/AuxData.hpp:389:64: error: expected unqualified-id before '>' token
389 | template <class... Args> struct auxdata_traits<std::set<Args...>> {
| ^~
make[2]: *** [src/CMakeFiles/gtirb.dir/build.make:63: src/CMakeFiles/gtirb.dir/AuxData.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:449: src/CMakeFiles/gtirb.dir/all] Error 2
make: *** [Makefile:163: all] Error 2
I checked the issue tracker and it seems that no one else reported this previously. Is it due to my dependency versions being too new or something? Thanks.
@AaronBallman hello,
I have one small issue , how to open generated IR?
If any one has gtirb commands please share
a@a-VirtualBox:~$ gtirb-pprinter --ir helloworld
[INFO] Reading GTIRB file: "helloworld"
[ERROR] Failed to load the GTIRB data from the file.
i am getting this error
We expose implementation details when we encode containers as part of signatures, and I think we want to avoid that when the container is not integral to the data being contained.
Functions returning containers like:
std::vector<Module>& IR::getModules();
const std::vector<Module>& IR::getModules() const;
std::vector<const Module*> IR::getModulesWithPreferredEA(EA x) const;
Functions accepting containers like:
template <typename T, size_t Size>
void ImageByteMap::setData(EA ea, const std::array<T, Size>& data);
This allows us to more easily refactor APIs to use different containers and it makes it easier to integrate the library into other code bases. Instead of working directly with the container, we should be working with iterators and ranges to abstract away the container.
I would like to disassemble windows resource file generated by gtirb to human-readable .rc file. Is that possible?
make[2]: *** [googletest-build/googletest/CMakeFiles/gtest.dir/build.make:82: googletest-build/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:504: googletest-build/googletest/CMakeFiles/gtest.dir/all] Error 2
gmake: *** [Makefile:182: all] Error 2
Cant figure out how to solve this. NO specific information.
Currently when we want to use gtirb from Makefile, we have to firstt set the variable export PKG_CONFIG_PATH= to point to the folder pkgconfig and to edit our own file such as:
libgtirb.pc:
prefix=/usr/lib
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${exec_prefix}/lib
Name: libgtirb
Description: The foo library
Version: 1.0.0
Cflags: -I${includedir}/libgtirb
Libs: -L${libdir} -llibgtirb
It is not easy to use. I suggest to edit Makefile to autogenerate these.
During the review for #13, it was observed that serializing or deserializing constituent components requires the user to have intimate knowledge of internal details or else the operations may not behave as the user expected. For instance, the following code has a subtle bug:
Symbol* Sym = Symbol::Create(Ctx, Addr(0), "test", Block::Create(Ctx));
Symbol::MessageType Msg;
Sym->toProtobuf(&Msg);
The problem here is that the block referent is not serialized out, but the symbol is. When deserializing, the symbol will claim to not have a referent because one could not be located, unless the same Ctx
instance is passed when deserializing, in which case it will work.
I think we should privatize the serialization code and only expose the APIs we want users to call (which is at the IR level, not at the individual component level). We can always expose the individual methods if we come up with use cases for them.
Would it be possible to add an install repository for Ubuntu 22?
Hello! I am writing a C# library for manipulating GTIRB (https://github.com/SapientGuardian/GtirbSharp). Initially it began as a port of the Java implementation, but I found that to be rather incomplete and have had to piece together some details from the other languages.
One thing that is not clear from the documentation or the existing implementations is the intended scope for Node identifiers (UUIDs).
gtirb/java/com/grammatech/gtirb/Node.java
Line 23 in 57c7e9d
Line 424 in 68b763e
gtirb/python/tests/test_node_from_uuid.py
Line 17 in 6b4bc8f
I just read code from gtirb-stack-stamp, but I feel it is too complex and not generic. It does not work on windows executable for example.
https://github.com/GrammaTech/gtirb-stack-stamp/blob/5586ffb6e8ba2e929f974bb225335777e99721a5/src/gtirb_stack_stamp.cpp#L114
For example, I want to modify a string literal in the binary, it looks it is very hard with this.
I get the following messages when attempt those installation instructions in a docker container. grammatech.github.io does not appear to be there.
root@135289f9d42a:/# apt-get update
Hit:1 http://ppa.launchpad.net/maarten-fonville/protobuf/ubuntu bionic InRelease
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease
Ign:4 https://grammatech.github.io/gtirb/pkgs/xenial ./ InRelease
Hit:5 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Err:6 https://grammatech.github.io/gtirb/pkgs/xenial ./ Release
404 Not Found [IP: 185.199.108.153 443]
Hit:7 http://ppa.launchpad.net/mhier/libboost-latest/ubuntu bionic InRelease
Hit:8 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Reading package lists... Done
E: The repository 'https://grammatech.github.io/gtirb/pkgs/xenial ./ Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
The operations on the Node class have me concerned. Node is a base class type for a type hierarchy and so it has virtual functions. The first thing I looked for was to see that the copy constructor was deleted so we don't have to worry as much about slicing problems -- yay! But then I noticed that the copy constructor is not deleted. So you can copy construct, but not copy assign, which is odd. Then I saw that you can move assign and move construct, which is even more odd. Users should not think of Node as a value type, so I think that this unorthodox mixture of operations should either be removed or made into protected members so that subclasses can use them but end users cannot.
Do we expect end users to subclass any of the Node subclasses? If not, we should make use of the "final" keyword on some the terminal classes. Additionally, if any virtual functions in the hierarchy are expected to not be overridden (but others within the class may be), we should mark those functions as final.
Based on the above two points and the fact that the only virtual functions we expose are destructors, I think we may want to go with a slightly different design. For performance reasons, I would expect that we'd want to vend pointers to these objects using a custom allocator and go with something akin to LLVM's casting infrastructure (http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/Casting.h?view=markup). This gives us improved data locality, reduces object size (no vtables, no need for RTTI), and allows us to use data structures that are optimized for working with pointers. As a very rough example of what I'm talking about, see https://is.gd/SAjitt
We've been discussing whether to do a binary release or not, but one thing we may want to change if we want a binary release is our use of angle bracket includes.
Popular compilers (Clang, GCC, and MSVC) all perform different lookup heuristics when encountering angle brackets vs quotes for file inclusion. Namely, they'll include the directory containing the source
file doing the inclusion on the search path only when using the quoted include paths.
If we're thinking of doing binary distributions of a shared library, one common build configuration is for users to download the header files and shared libraries (bypassing our CMake entirely), so they may
put the header files next to their source file and not change the include paths passed to the compiler. However, this will fail because we use angle brackets for including things that are local to our source. Their #include "gtirb/Module.hpp" may find the correct file, but our #include <gtirb/Addr.h> may fail due to header search paths.
The approach I'm used to is to prefer quoted includes for any header files supplied as part of the root include/ directory and angle bracket includes for things the user is expected to make available themselves (STL, system headers, etc). However, our header files are laid out... more complicated than that. We have include/gtirb that we supply, but we also have boost/include/boost that we supply, and then there's gsl and googletest which we don't supply as part of the source directory but the user cannot supply either with their own version (due to ABI issues), but we rely on as part of our header files (at least for gsl, we do).
BTW, what should i do if i want to trace a constant that can reach for the CFG?
For example
movq $3,%rax
movq %rax,(%rsp)
movq %rax,%rbx
How can I trace all the code that $3 could reach for example?
Log here: http://report.quicklisp.org/2020-09-15/failure-report/gtirb.html#proto
I get this:
; caught COMMON-LISP:ERROR:
; COMMON-LISP:READ error during COMMON-LISP:COMPILE-FILE: Symbol "LENGTH32" not found in the VARINT package.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.