andreafioraldi/qasan

QASan is a custom QEMU 3.1.1 that detects memory errors in the guest using AddressSanitizer.

QASan (QEMU-AddressSanitizer)

Written and maintaned by Andrea Fioraldi [email protected]

For just usermode fuzzing, a more updated version of QASan is now embedded into AFL++, use it instead

QASan is a custom QEMU 3.1.1 that detects memory errors in the guest using AddressSanitizer.

As QEMU is a binary code emulator, QASan is a binary-only sanitizer and does not require compiler-based instrumentation.

Dowload it with:

git clone --recursive https://github.com/andreafioraldi/qasan.git

Build

QASan comes in two possible twists, one based on my own ASan implementation and the other on the clang's implementation of ASan.

build.py is the script used to build all.

The flag --system allows you to build full-system QASan, an experimental feature ATM.

asan-giovese

asan-giovese is my implementation of AddressSanitizer. It is in pure C11 and allows you to get useful informations from the target process like stacktraces on allocations and on errors.

It will be the only supported option in future, but at the moment is not already completely thread safe.

This is the default mode, built when you don't specify the --asan-dso flag.

compiler-rt ASan

You need the lief python3 package.

Build using the build.py script specifying the path to the ASan DSO.

./build.py --asan-dso /path/to/libclang_rt.asan-ARCH.so

On Ubuntu 18.04, the path is /usr/lib/llvm-8/lib/clang/8.0.0/lib/linux/libclang_rt.asan-x86_64.so

Note that QASan will not output meaningful stacktraces or error reports when using this mode.

The reported errors show informaton about the QEMU host and so they are not useful for debugging.

other options

Other available build options are:

• --arch to specify the target architecture (default is x86_64)
• --cc and --cxx to specify C and C++ compilers (default clang-8)
• --cross to specify the cross C compiler for libqasan
• --clean to clean builded files

Tested only on Ubuntu 18.04 with x86[_64] and arm[64] targets.

Usage

To simply run a binary under QASan:

./qasan ./program args...

To set the LD path (useful when running cross arch, in this case armhf):

./qasan --prefix /usr/arm-linux-gnueabihf/ ./program args...

Other options are --preload to preload a shared object, --verbose to get a verbose output with memory mappings and --debug to log hooked actions if libqasan is compiled in debug mode.

By default, only the main executable memory accesses are instrumented. To enable the instrumentation of all the libraries, use AFL_INST_LIBS=1.

Beware that glibc have a lot of assumptions on buffer size and a lot of handwritten magic (see this). If you have an error caused by these optimizations you can disable the instrumentation for single functions adding them to libqasan/uninstrument.c.

Fuzzing

To fuzz a binary with QASan and AFL++ use a command similar to the following:

~/AFLplusplus/afl-fuzz -U -i in -o out -m none -- python3 ~/qasan/qasan ./program

It supports all the AFL++ QEMU configurations, AFL_COMPCOV_LEVEL=2 is higly suggested.

To improve speed, set the env variables QASAN_MAX_CALL_STACK=0 and QASAN_SYMBOLIZE=0.

FAQ

When I should use QASan?

If your target binary is PIC x86_64, you should before give a try to retrowrite for static rewriting.

If it fails, or if your binary is for another architecture, QASan is the tool that you want/have to use.

Note that the overhead of AFL++ libdislocator is much lower but it can catch less bugs. This is a short blanket, take your choice.

Another discriminat for the choice is CompareCoverage. If your target has fuzzing roadblocks, you can use QASan+CompCov to fuzz it with Sanitization and Roadblocks bypassing.

QEMU segfaults with big endian archs

See https://bugs.launchpad.net/qemu/+bug/1701798, use the workaround described here.

Performance

Native (slowdown: 1x):

$ time /usr/bin/objdump -g -x /usr/bin/objdump
...
real	0m0,058s
user	0m0,010s
sys	0m0,029s

QEMU (slowdown: 2.4x):

$ time qemu-x86_64 /usr/bin/objdump -g -x /usr/bin/objdump
...
real	0m0,141s
user	0m0,096s
sys	0m0,020s

QASan (slowdown: 3.6x):

$ time ./qasan /usr/bin/objdump -g -x /usr/bin/objdump
...
real	0m0,209s
user	0m0,120s
sys	0m0,032s

Valgrind (slowdown: 17.4x):

$ time valgrind /usr/bin/objdump -g -x /usr/bin/objdump
...
real	0m1,009s
user	0m0,921s
sys	0m0,076s

Cite

If you use QASan for your academic work, you should cite the following pubblication:

• Andrea Fioraldi, Daniele Cono D’Elia, and Leonardo Querzoni. “Fuzzing binaries for memory safety errors with QASan”. In 2020 IEEE Secure Development Conference (SecDev), 2020.

Preprint: https://andreafioraldi.github.io/assets/qasan-secdev20.pdf

Bibtex:

@INPROCEEDINGS{QASan-SecDev20,
  author={Fioraldi, Andrea and D’Elia, Daniele Cono and Querzoni, Leonardo},
  title={Fuzzing Binaries for Memory Safety Errors with {QASan}},
  booktitle={2020 IEEE Secure Development Conference (SecDev)},
  doi={10.1109/SecDev45635.2020.00019},
  pages={23--30},
  year={2020}
}

Video: https://www.youtube.com/watch?v=UtFXU7Nkd8g

Slides: https://andreafioraldi.github.io/assets/qasan-secdev20-slides.pdf

qasan's Issues

TODO: Implement Stack Use-After-Return

Map memory accesses inside the stackframe boundaries to an allocated separate memory.
This shit cost a bit (not too cause the accesses are already instrumented), but I don't see any other way to do that.
Suggestions are welcome.

Build error on Ubuntu 16.04

Andrea,

I'm trying to get QASAN installed on an Ubuntu 16.04 VM with clang, but while running ./build.py --cc clang --cxx clang++, I'm encountering the following error:

clang-8  -fPIC -shared -I ../include libqasan.c hooks.c malloc.c string.c uninstrument.c patch.c -o libqasan.so -ldl -pthread
patch.c:210:12: error: use of undeclared identifier 'explicit_bzero'; did you mean 'p_explicit_bzero'?
  HOTPATCH(explicit_bzero)
           ^~~~~~~~~~~~~~
           p_explicit_bzero
patch.c:194:59: note: expanded from macro 'HOTPATCH'
  if (p_## fn) __libqasan_patch_jump(p_## fn, (uint8_t*)&(fn)); \
                                                          ^
patch.c:210:3: note: 'p_explicit_bzero' declared here
  HOTPATCH(explicit_bzero)
  ^
patch.c:193:12: note: expanded from macro 'HOTPATCH'
  uint8_t* p_## fn = (uint8_t*)dlsym(libc, # fn); \
           ^
<scratch space>:62:1: note: expanded from here
p_explicit_bzero
^
1 error generated.
Makefile:9: recipe for target 'all' failed
make: *** [all] Error 1
Traceback (most recent call last):
  File "./build.py", line 226, in <module>
    % (os.path.join(dir_path, "libqasan"), cross_cc, libqasan_cflags)) == 0 )
AssertionError

Any ideas?

HELP : can't trigger qasan crash

I use this code snippet to test QASan.

#include <stdio.h>
#include <malloc.h>

void func0(unsigned char byte) {
    unsigned char *p = (unsigned char *)malloc(0x10);
    p[0x11] = byte;
}

int main(int argc, char *argv[]) {
    if (argc != 2) {
        printf("Usage: %s filename\n", argv[0]);
        return 1;
    }
    FILE *fp = fopen(argv[1], "rb");
    if (!fp) {
        printf("Failed to open file %s\n", argv[1]);
        return 1;
    }
    unsigned char buf[1];
    size_t n = fread(buf, sizeof(unsigned char), 1, fp);
    if (n != 1) {
        printf("Failed to read file\n");
        fclose(fp);
        return 1;
    }

    func0(buf[0]);

    fclose(fp);
    return 0;
}

run afl++ with

$AFLPP/afl-fuzz -Q -i ./in -o ./out/ -M fuzzer1 -- ./test @@

not crash detected...

I think I might have made a mistake in some part...

SEGV when running a 32-bit binary

Hello.
I tried to run a 32-bit binary on an Ubuntu 16.04, which crashed with a SEGV instead.

System Details:
Test on Ubuntu 16.04 LTS(x86_64)

Code:

//test.c
#include<stdio.h>
int main(){
	int a;
	a=1;
	return 0;
}

Command:

gcc -m32 ./test.c -o ./test
./build.py --arch i386
./qasan ./test

QASAN Output:

QEMU-AddressSanitizer:DEADLYSIGNAL
=================================================================
==3414==ERROR: QEMU-AddressSanitizer: SEGV on unknown address 0x00008d413dfc (pc 0x00008d413dfc bp 0x0000ffffdb49 sp 0x0000ffffdadc T3414)
    #0 0x00008d413dfc
    #1 0x0000ff7cf871 in strstr (/home/spiderman/qasan_32/libqasan.so+0x3871)

QEMU-AddressSanitizer can not provide additional info.
SUMMARY: QEMU-AddressSanitizer: 
==3414==ABORTING
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)

Probably QASAN_LOAD leads to it.

By the way, I got no problem running this binary on qasan-qemu. Do you have any idea？

Does QASAN now support for ARM64 binary?If it support, can it support all binarys that qemu supports(For example, ARM Trusted Firmware (ATF))?

come across "Fork server handshake failed" when work with AFL++

Hello:
I try to use qasan with AFL++ and come across a "Fork server handshake failed" Error:

zero@u:~/Documents/fuzzcat$ AFL_COMPCOV_LEVEL=2 ~/Documents/AFLplusplus-stable/afl-fuzz -U -i in -o out -m none -- python3 ~/Documents/qasan/qasan /bin/cat
afl-fuzz++2.66c based on afl by Michal Zalewski and a big online community
[+] afl++ is maintained by Marc "van Hauser" Heuse, Heiko "hexcoder" Eißfeldt, Andrea Fioraldi and Dominik Maier
[+] afl++ is open source, get it at https://github.com/AFLplusplus/AFLplusplus
[+] Power schedules from github.com/mboehme/aflfast
[+] Python Mutator and llvm_mode instrument file list from github.com/choller/afl
[+] MOpt Mutator from github.com/puppet-meteor/MOpt-AFL
[*] Getting to work...
[+] Using exploration-based constant power schedule (EXPLORE, default)
[+] You have 2 CPU cores and 5 runnable tasks (utilization: 250%).
[!] WARNING: System under apparent load, performance may be spotty.
[*] Checking CPU core loadout...
[+] Found a free CPU core, try binding to #0.
[*] Checking core_pattern...
[!] WARNING: Could not check CPU scaling governor
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning 'in'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[*] Attempting dry run with 'id:000000,time:0,orig:test'...
[*] Spinning up the fork server...

[-] Hmm, looks like the target binary terminated before we could complete a handshake with the injected code.
If the target was compiled with afl-clang-lto then recompiling with AFL_LLVM_MAP_DYNAMIC might solve your problem.
Otherwise there is a horrible bug in the fuzzer.
Poke <[email protected]> for troubleshooting tips.

[-] PROGRAM ABORT : Fork server handshake failed
         Location : afl_fsrv_start(), src/afl-forkserver.c:849

(In ubuntu 20.04 x86_64)
Any idea how to fix this?
thanks.

Unexpeced exception when running aarch64 binary

Hello:

The following exception was encountered when testing an aarch64 binary. The excepted behavior is to finish normally. Not sure why this happened, I’m still trying to minimize the binary.

QEMU-AddressSanitizer:DEADLYSIGNAL
=================================================================
==12666==ERROR: QEMU-AddressSanitizer: SEGV on unknown address 0x7fde991faad8 (pc 0x7fde991faad8 bp 0x7fde9dd15140 sp 0x7fde9dd15010 T12666)
    #0 0x7fde991faad8 in __dl___set_errno_internal __dl_sfp-exceptions.c:?

QEMU-AddressSanitizer can not provide additional info.
SUMMARY: QEMU-AddressSanitizer:  in __dl___set_errno_internal __dl_sfp-exceptions.c:?
==12666==ABORTING

Support for running qasan on aarch64 host

Right now qasan does not support running aarch64 binaries on a aarch64 host. Having native support would be nice now that arm CPUs are gaining popularity.

Running ./qasan /bin/ls gives me the following crash:

qemu:handle_cpu_signal received signal outside vCPU context @ pc=0xaaaad6ce68c6

Tested on Ubuntu 18.04.1 LTS, aarch64, built with ./build.py --arch=aarch64 --cc gcc --cxx g++

fail to compiling libqasan with NDK

hello，I am trying to fuzz Android cmdline elf with qasan and afl++

./build.py --cc /usr/bin/clang --cxx /usr/bin/clang++ --arch arm64  --cross ~/Documents/android-ndkr21b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android29-clang

come across some error when compiling libqasan with NDK :

/home/zero/Documents/android-ndk-r21b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android29-clang  -fPIC -shared -I ../include libqasan.c hooks.c malloc.c string.c uninstrument.c patch.c -o libqasan.so -ldl -pthread
hooks.c:45:8: error: conflicting types for 'malloc_usable_size'
size_t malloc_usable_size (void * ptr) {
       ^
/home/zero/Documents/android-ndk-r21b/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/malloc.h:104:8: note: previous declaration is here
size_t malloc_usable_size(const void* __ptr) __INTRODUCED_IN(17);
       ^
hooks.c:170:22: error: invalid application of 'sizeof' to an incomplete type 'FILE' (aka 'struct __sFILE')
  QASAN_LOAD(stream, sizeof(FILE));
                     ^     ~~~~~~
../include/qasan.h:243:45: note: expanded from macro 'QASAN_LOAD'
  QASAN_CALL2(QASAN_ACTION_CHECK_LOAD, ptr, len)
                                            ^~~
../include/qasan.h:236:43: note: expanded from macro 'QASAN_CALL2'
  syscall(QASAN_FAKESYS_NR, action, arg1, arg2, NULL)
                                          ^~~~
/home/zero/Documents/android-ndk-r21b/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/stdio.h:58:8: note: forward declaration of 'struct __sFILE'
struct __sFILE;
       ^
hooks.c:288:12: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
void bzero(void *s, size_t n) {
           ^
hooks.c:288:21: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
void bzero(void *s, size_t n) {
                    ^
hooks.c:288:6: error: conflicting types for '__bionic_bzero'
void bzero(void *s, size_t n) {
     ^
/home/zero/Documents/android-ndk-r21b/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/strings.h:61:23: note: expanded from macro 'bzero'
#define bzero(b, len) __bionic_bzero((b), (len))
                      ^
/home/zero/Documents/android-ndk-r21b/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/strings.h:62:40: note: previous definition is here
static __inline__ __always_inline void __bionic_bzero(void* b, size_t len) {
                                       ^
hooks.c:288:12: error: parameter name omitted
void bzero(void *s, size_t n) {
           ^
hooks.c:288:21: error: parameter name omitted
void bzero(void *s, size_t n) {
                    ^
hooks.c:293:15: error: use of undeclared identifier 's'
  QASAN_STORE(s, n);
              ^
hooks.c:293:18: error: use of undeclared identifier 'n'
  QASAN_STORE(s, n);
                 ^
hooks.c:294:21: error: use of undeclared identifier 's'
  __libqasan_memset(s, 0, n);
                    ^
hooks.c:294:27: error: use of undeclared identifier 'n'
  __libqasan_memset(s, 0, n);

Any idea? Thanks

Port recent AFL++ patches like CmpLog

TODO

If someone wants to contribute, this is a good first issue

Usage of Full-System QASan

I compiled qasan-system for ARM and ran an ARM system like this:

qasan-system -M vexpress-a9 -m 256M -dtb vexpress-v2p-ca9.dtb -nographic -kernel zImage -append "root=/dev/mmcblk0 console=ttyAMA0 rw" -sd a9rootfs.ext3

Then I ran the following program inside qasan-system (it had been cross-compiled with arm-linux-gnueabi-gcc statically and put into a9rootfs.ext3 beforehand):

#include <stdio.h>
#include <stdlib.h>

int main() {
    int *a = (int *)malloc(3 * sizeof(int));
    int b = a[10];  // out of bound
    printf("b = %d\n", b);
    free(a);
    return 0;
}

I expected QASan to detect the out-of-bound access and output some information, but it didn't. Is this the right usage of qasan-system? I know I can run this program with user-mode QASan but want to experiment with the system mode. Thanks in advance!

Installation and usage problems for the 32-bit architectures

Hi, I found it an interesting job and tried to use it. But I met with some confusion. So, I have two questions for help.

1. I built qasan for the 32-bit x86 architecture. As the readme mentioned, i386 libraries need to be installed and PKG_CONFIG_PATH needs to be set. Then I installed the i386 libraries in some manners similar to apt-get install lib32ncurses5 ​lib32z1 or apt-get install g++-multilib libncurses5:i386 libc6:i386 libgcc1:i386 gcc-4.6-base:i386 libstdc++5:i386 libstdc++6:i38.
   Unfortunately, I can not find the /usr/lib/i386-linux-gnu/pkgconfig in my filesystem with Ubuntu 18.04.3. So, the error is as follows:

sizeof(size_t) doesn't match GLIB_SIZEOF_SIZE_T.        
You probably need to set PKG_CONFIG_LIBDIR       
to point to the right pkg-config files for your build target

I also tried online solutions, but it didn't work. I'd like to know how you solved the problem.

2. After reading the build.py code, I understood that the asan-giovese mode implemented by yourself only supported x86 (_64), not arm, etc. Is my understanding correct？
   Thank you very much. Looking forward to your reply.

How can I build qasan for arm?

Hi, I encountered some difficulties related to qasan for arm.
I sincerely ask for a help.

If I need to build qasan for arm，what <path_for_dso> to specify?
Command: ./build.py --asan-dso <path_for_dso> --arch arm --cross arm-linux-gnueabi-gcc

And I didn't find anything concerned with arm under usr/lib/llvm-8/lib/clang/8.0.0/lib/linux/.
If I ignore something important?

Additionally, could you please add some examples on how to build qasan for other architectures supported?

python not found and error building

I got error when building on ubuntu 20.04 with llvm 12, python3.8
At first it doesn't found my python so I add to build.py but it still got error

cd '/home/aldo/qasan/qemu' ; ./configure --target-list="x86_64-linux-user" --disable-system --enable-pie       --cc="/home/aldo/llvm-project/build/bin/clang-12" --cxx="/home/aldo/llvm-project/build/bin/clang++"  --extra-cflags="-O3 -ggdb " --extra-ldflags="-L /home/aldo/qasan -lclang_rt.asan-x86_64 -Wl,-rpath,.,-rpath,/home/aldo/qasan"       --enable-linux-user --disable-gtk --disable-sdl --disable-vnc --disable-strip --python=/usr/bin/python3

ERROR: "/home/aldo/llvm-project/build/bin/clang-12" cannot build an executable (is your linker broken?)

Traceback (most recent call last):
  File "build.py", line 210, in <module>
    assert (os.system(cmd) == 0)
AssertionError

Any ideas why?