decaf-project/decaf

DECAF (short for Dynamic Executable Code Analysis Framework) is a binary analysis platform based on QEMU. This is also the home of the DroidScope dynamic Android malware analysis platform. DroidScope is now an extension to DECAF.

DECAF

DECAF (short for Dynamic Executable Code Analysis Framework) is a binary analysis platform based on QEMU.

DECAF++

DECAF++, the new version of DECAF, taint analysis is around 2X faster making it the fastest, to the best of our knowledge, whole-system dynamic taint analysis framework. This results in a much better usability imposing only 4% overhead (SPEC CPU2006) when no suspicious (tainted) input exists. Even under heavy taint analysis workloads, DECAF++ has a much better performance, around 25% faster on nbench, because of its elasticity. DECAF++ elasticity makes it a very suitable case for security analysis tasks that would selectively analyze the input e.g. Intrusion Detection Systems (IDS) that can filter out benign traffic. For further technical details, see our RAID 2019 paper. To activate the optimizations, see our DECAF++ wiki page.

PUBLICATIONS

1. Ali Davanian, Zhenxiao Qi, Yu Qu, and Heng Yin, DECAF++: Elastic Whole-System Dynamic Taint Analysis, In the 22nd International Symposium on Research in Attacks, Intrusions and Defenses (RAID), September 2019. (If you wish to cite the new optimized version of DECAF, please cite this paper)

2. "Make it work, make it right, make it fast: building a platform-neutral whole-system dynamic binary analysis platform", Andrew Henderson, Aravind Prakash, Lok Kwong Yan, Xunchao Hu, Xujiewen Wang, Rundong Zhou, and Heng Yin, to appear in the International Symposium on Software Testing and Analysis (ISSTA'14), San Jose, CA, July 2014.(If you wish to cite DECAF, please cite this paper)

3. Lok Kwong Yan, Andrew Henderson, Xunchao Hu, Heng Yin, and Stephen McCamant?.On soundness and precision of dynamic taint analysis. Technical Report SYR-EECS-2014-04, Syracuse University, January 2014.

4. DroidScope: Seamlessly Reconstructing OS and Dalvik Semantic Views for Dynamic Android Malware Analysis", Lok-Kwong Yan and Heng Yin, in the 21st USENIX Security Symposium, Bellevue, WA, August 8-10, 2012.

Select Academic Systems Using DECAF

1. Luo, L., Fu, Y., Wu, D., Zhu, S., & Liu, P. (2016, June). Repackage-proofing android apps. In 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) (pp. 550-561). IEEE.

2. Wei, J., Yan, L. K., & Hakim, M. A. (2015, December). Mose: Live migration based on-the-fly software emulation. In Proceedings of the 31st Annual Computer Security Applications Conference (ACSAC) (pp. 221-230).

3. Carmony, C., Hu, X., Yin, H., Bhaskar, A.V., & Zhang, M. (2016). Extract Me If You Can: Abusing PDF Parsers in Malware Detectors. In Proceedings of 2016 Network and Distributed System Security Symposium (NDSS).

4. Feng, Q., Prakash, A., Yin, H., & Lin, Z. (2014, December). Mace: High-coverage and robust memory analysis for commodity operating systems. In Proceedings of the 30th annual computer security applications conference (pp. 196-205)

5. Alwabel, A., Shi, H., Bartlett, G., & Mirkovic, J. (2014). Safe and automated live malware experimentation on public testbeds. In 7th Workshop on Cyber Security Experimentation and Test (CSET'14).

Introduction

DECAF (Dynamic Executable Code Analysis Framework) is the successor to the binary analysis techniques developed for TEMU (dynamic analysis component of BitBlaze ) as part of Heng Yin's work on BitBlaze project headed up by Dawn Song. DECAF builds upon TEMU. We appreciate all that worked with us on that project.

Fig 1 the overall architecture of DECAF

Fig 1 illustrates the overall architecture of DECAF. DECAF is a platform-agnostic whole-system dynamic binary analysis framework. It provides the following key features.

Right-on-Time Virtual Machine Introspection

Different with TEMU, DECAF doesn’t use guest driver to retrieve os-level semantics. The VMI component of DECAF is able to reconstruct a fresh OS-level view of the virtual machine, including processes, threads, code modules, and symbols to support binary analysis. Further, in order to support multiple architectures and operating systems, it follows as a platform-agnostic design. The workflow for extracting OS-level semantic information is common across multiple architectures and operating systems. The only platform- specific handling lies in what kernel data structures and what fields to extract information from.

Support for Multiple Platforms

Ideally, we would like to have the same analysis code (with minimum platform-specific code) to work for different CPU architectures (e.g, x86 and ARM) and different operating systems (e.g., Windows and Linux). It requires that the analysis framework hide the architecture and operating system specific details from the analysis plugins. Further, to make the analysis framework itself maintainable and extensible to new architectures and operat-ing systems, the platform-specific code within the framework should also be minimized. DECAF can provide support for both multiple architectures and multiple operating systems. Currently, DECAF supports 32 bit Windows xp/Windows 7/linux and X86/arm.

Precise and Lossless Tainting

DECAF ensures precise tainting by maintaining bit-level precision for CPU registers and memory, and inlining precise tainting rules in the translated code blocks. Thus, the taint status for each CPU register and memory location is processed and updated synchronously during the code execution of the virtual machine. The propagation of taint labels is done in an asynchronous manner . By implementing such a tainting logic mainly in the intermediate representation level (more concretely, TCG IR level), it becomes easy to extend tainting support to a new CPU architecture.

Event-driven programming interfaces

DECAF provides an event-driven programming interface. It means that the paradigm of ”instrument” in the translation phase and then analyze in the execution phase” is invisible to the analysis plugins. The analysis plugins only need to register for interested events and implement corresponding event handling functions. The details of code instrumentation are taken care of by the framework.

Dynamic instrumentation management

To reduce runtime overhead, the instrumentation code is inserted into the translated code only when necessary. For example, when a plugin registers a function hook at a function’s entry point, the instrumentation code for this hook is only placed at the function entry point. When the plugin unregisters this function hook, the instrumentation code will also be removed from the translated code accordingly. To ease the development of plugins, the management of dynamic code instrumentation is completely taken care of in the framework, and thus invisible to the plugins.

Help Documents

Please refer to our wiki page for help documents.

decaf's Issues

DroidScope: Supported Android Version

So is DroidScope currently capable of analyzing samples on Android 4.3? Not sure what to make of the comments in Google Code since they seem a bit dated

About address of instruction

Hello:
May I ask how to get address of instructions like Pin does with INS_Address(ins) ?
Thanks!

Procinfo compilation error on Ubuntu 14.04.5

Hello,

I am currently trying to get the information needed in the procinfo.ini file from a new guest that I am setting up. The guest is Ubuntu 14.04.5 and I have moved the Makefile and procinfo.c file over to the guest. Upon compilation it gives the attached error.

I know that these files work since I have used the same ones on an Ubuntu 12.04 guest and it worked fine. Do you know of any reason that this would be breaking?

Keylogger

Hi,
I'm using keylogger-plugin to get malware‘s behavior.
Why the keylogger-plugin gets tainted data read by many processes other than the target malicious process？

Error while compiling

I'm also having an error while running this command: "sudo apt-get build-dep qemu"
The output that I'm getting is:
"Reading package lists... Done
E: You mut put some 'deb-src' URIs in your sources.list"

Makefile:214 qmp-commands.h : Python error on make?

GEN   qmp-commands.h
  File "/home/wren/code/DECAF/decaf/scripts/qapi-commands.py", line 376
    except getopt.GetoptError, err:
                             ^
SyntaxError: invalid syntax
make: *** [Makefile:214: qmp-commands.h] Error 1

When attempting to make for the first time.

On archlinux python --version == 3.7.4

how to build decaf on OSX

on my mac 10.11 , i'm try to compile decaf,but i failed. i remove some error about sleuthkit unit_tests,and i failed again with cocoa.m error。it seems a little guys interest to it, any build guide here?thx,

problem in running emulator commond in DroidScope

daya@Vostro-ubunu:~/tdroid/tdroid-4.3_r1/external/DroidScope/objs$ ./emulator -sysdir /home/daya/tdroid/tdroid-4.3_r1/out/target/product/generic/ -kernel /home/daya/ -memory 512 -qemu -monitor stdio
emulator: WARNING: system partition size adjusted to match image file (238 MB > 200 MB)

Failed to load libGL.so
error libGL.so: cannot open shared object file: No such file or directory
Failed to load libGL.so
error libGL.so: cannot open shared object file: No such file or directory
qemu: could not load kernel '/home/daya/'

VMI Configuration for new kernel version

Hi!

I am trying to configure VMI on ubuntu 16.04 32bits. The kernel version is 4.15.0-45-generic.

I followed the steps detailed here: https://github.com/decaf-project/DECAF/wiki/Startup.

On the third step, when compiling procinfo.c, i encounter the following error:
'struct thread_info' has no member named 'task'

I have read that this is because the procinfo.c file is designed for older kernel versions. Is there any new procinfo.c file that would support newer kernel versions (such as 4.15)?

Thanks a lot!

Query on Starting the Virtual Machine

Can anyone please tell me where can I find the DECAF/trunk/i386-­softmmu directory?
I looked in the DECAF source folder but it doesnt have the trunk folder inside.

libtsk.so error doing make in decaf directory

make: *** [Makefile:53: /home/aleff/Documents/NetworkSecurity/PDF_Parser_Differential_Attack/demo/DECAF/decaf/shared/sleuthkit/lib/libtsk.so] Error 2

What can i do to solve it?

Tracecap is not logging taint instructions, trace reader not working

Hi !

I'm currently trying to log tainted instructions using tracecap plugin. I followed this document to proceed: https://github.com/decaf-project/DECAF/blob/master/decaf/plugins/tracecap/USAGE

Here is the command i use to launch the VM:
./DECAF/decaf/i386-softmmu/qemu-system-i386 -monitor stdio -m 4096 -netdev user,id=mynet,hostfwd=tcp::10080-:80 -device rtl8139,netdev=mynet /opt/images/windows.img

Whether I use the "taint_nic" (with a program that reads data from the net, here apache server) command or the "taint_sendkey" command (with a program that reads data input by user on command line), there is 0 tainted instruction written to the trace file.

Also, before performing any taint operation, I make sure to enable tainting using the command "enable_tainting"

Finally, I noticed that one command specified in the USAGE doc enable_emulation, is not present in the list of commands associated to this plugin: https://github.com/decaf-project/DECAF/blob/master/decaf/plugins/tracecap/plugin_cmds.h.

How can I proceed to get the tainted instructions reported on the trace file ?

Also, when trying to compile the trace_reader, I encounter 2 problems, while I'm following instructions provided here https://github.com/decaf-project/DECAF/blob/master/decaf/plugins/tracecap/trace_reader/INSTALL:

• First is that there is no configure file
• Second, when compiling with the provided makefile I encounter the following errors:

libdasm.o: In function 'get_operand':
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:320: undefined reference to 'MODE_CHECK_ADDR'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:464: undefined reference to 'FETCH8'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:470: undefined reference to 'FETCH16'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:473: undefined reference to 'FETCH32'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:504: undefined reference to 'MODE_CHECK_OPERAND'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:509: undefined reference to 'FETCH8'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:518: undefined reference to 'FETCH32'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:519: undefined reference to 'FETCH16'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:523: undefined reference to 'FETCH16'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:533: undefined reference to 'MODE_CHECK_OPERAND'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:537: undefined reference to 'FETCH32'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:537: undefined reference to 'FETCH16'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:538: undefined reference to 'FETCH16'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:550: undefined reference to 'FETCH8'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:555: undefined reference to 'FETCH32'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:555: undefined reference to 'FETCH16'
libdasm.o: In function 'get_operand_string':
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:602: undefined reference to 'MODE_CHECK_OPERAND'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:676: undefined reference to 'MODE_CHECK_ADDR'
libdasm.o: In function 'get_mnemonic_string':
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:1068: undefined reference to 'MODE_CHECK_OPERAND'
/opt/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:1088: undefined reference to 'MODE_CHECK_OPERAND'
collect2: error: ld returned 1 exit status
Makefile:16: recipe for target 'trace_reader_cpp' failed
make: *** [trace_reader_cpp] Error 1

Unloading modules and emptying cache without having to unload plugin

Hello,

I am trying to write a plugin that will involve running a process in the guest multiple times. The issue I am facing is that between guest runs the modules in decaf are not cleared. I have found the issue to be within the lm variable that is a _LoadModule_Params struct. Do you know of a way to clear out past lm modules so that decaf sees the program as new each time it runs?

Currently to do this we need to unload the plugin and then reload it.

Tracecap sometimes logs the same instruction for twice

During the analysis for all CBs, some analysis failed because tracecap logged the same instruction for twice. The detailed info needs further investigation.

decaf ftp

I use the ftp command in the qemuVM to transfer things to the server, only the connection can be established, but the data is transmitted.

VMI won't work

I use DECAF in ubuntu 18.04 64bits, and guest OS is ubuntu 14.04 32bit. I used procinfo_generic in ubuntu18.04 64bits to get kernel info. Isn't right? Because I got a large init_task addr that is:

[info]
total = 1

[1]
strName = 4.15.0-55-generic
init_task_addr  = 18446744071811703936
init_task_size  = 9088
ts_tasks        = 1960
ts_pid          = 2216
ts_tgid         = 2220
ts_group_leader = 2280
ts_thread_group = 2392
ts_real_parent  = 2232
ts_mm           = 2040
ts_stack        = 24
module_name   = 24
module_size   = 392
module_init   = 384
module_list   = 8
ts_real_cred    = 2624
ts_cred         = 2632
ts_comm         = 2640
cred_uid        = 4
cred_gid        = 8
cred_euid       = 20
cred_egid       = 24
mm_mmap         = 0
mm_pgd          = 80
mm_arg_start    = 296
mm_start_brk    = 272
mm_brk          = 280
mm_start_stack  = 288
vma_vm_start    = 0
vma_vm_end      = 8
vma_vm_next     = 16
vma_vm_file     = 160
vma_vm_flags    = 80
vma_vm_pgoff    = 152
file_dentry     = 32
file_inode      = 48
dentry_d_name   = 32
dentry_d_iname  = 56
dentry_d_parent = 24
ti_task         = 8
inode_ino   = 64

I found decaf found task struct in a small address.tulInitTaskAddr: 3249244800

DECAF/decaf/shared/linux_procinfo.cpp

Line 1788 in 8c12a8d

tulInitTaskAddr = findTaskStructFromThreadInfo(env, threadinfo, &pi, 0);

➜  i386-softmmu git:(master) ✗ ./qemu-system-i386 -monitor stdio -m 2048 ./ubuntu1404_32.img -cdrom ~/ubuntu-14.04.6-desktop-i386.iso

(process:20336): GLib-WARNING **: 21:40:36.398: ../../../../glib/gmem.c:489: custom memory allocation vtable not supported
register_savevm slirp
register_savevm cpu
QEMU 1.0 monitor - type 'help' for more information
(qemu) inside bdrv open, drv addr= 0x0x560fb28b6e60, size= 32212254720
fs_open = Fragment 
inside bdrv open, drv addr= 0x0x560fb28c7950, size= 1170964480
fs_open error! drv addr=0x0x560fb28c7950
register_savevm DECAF
register_savevm funmap
register_savevm hookapi
inside vmi init 
tulInitTaskAddr: 3249244800

Procinfo path: /home/shaoshuai/DECAF/decaf/i386-softmmu/../shared/kernelinfo/procinfo_generic/procinfo.ini
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::property_tree::ptree_bad_data> >'
  what():  conversion of data to type "j" failed
[1]    20336 abort (core dumped)  ./qemu-system-i386 -monitor stdio -m 2048 ./ubuntu1404_32.img -cdrom 

how can I start vmi?

when I set init_task_addr = 3249244800, decaf will work.But how can I verify this VMI works righ?

Dokcer couldn't find 'libsdl1.2-dev'

Hi,

I'm trying to use DECAF using docker. The installation failed after I issued the following commands:
'sudo docker volum create decafvolume'
'sudo docker build -t decaf ./decaf/'

The error is:

Sending build context to Docker daemon 4.096kB
Step 1/57 : FROM ubuntu:16.04
---> 0458a4468cbc
Step 2/57 : RUN apt-get update
---> Using cache
---> 6564bd249788
Step 3/57 : RUN apt-get install libsdl1.2-dev -y
---> Running in 0346fc09fd5b
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package libsdl1.2-dev
E: Couldn't find any package by glob 'libsdl1.2-dev'
E: Couldn't find any package by regex 'libsdl1.2-dev'
The command '/bin/sh -c apt-get install libsdl1.2-dev -y' returned a non-zero code: 100

It seems that the package 'libsdl1.2-dev' is not found.
My system is Ubuntu 14.04, docker version is 17.05.0-ce.
Any help will be fully appreciated!

Query for starting up the project

Peace!
I'm trying to run DECAF on my Ubuntu virtual machine. The software that I'm interested in doing a binary analysis of runs on Windows. I wanted to know if I run a Windows machine inside my Linux virtual machine which runs on Vmware workstation how would the step of creating a Qemu image change for me?

ARM support

Hi,

I'm running into an issue trying to get DECAF working with an ARM guest and hoping that someone can point me in the right direction. I have created and inserted the procinfo kernel driver to get the correct values for procinfo.ini, and DECAF is able to detect the correct kernel during boot. After the VM is fully booted and I run guest_ps it only shows the kernel though:

(qemu) guest_ps
0	cr3=0x00000000	<kernel>

The lsmod command also doesn't show any results. Is there anything special that I need to do in order to get DECAF to work with an ARM guest?

EDIT --
Here is the output from the qemu monitor when the VM is first booted:

inside bdrv open, drv addr= 0x0x55cdc4d2cd40, size= 26843545600
QEMU 1.0 monitor - type 'help' for more information
(qemu) fs_open = Fragment 
inside bdrv open, drv addr= 0x0x55cdc4d2e7c0, size= 0
fs_open error! drv addr=0x0x55cdc4d2e7c0
inside vmi init 

Procinfo path: /decafroot/DECAF/decaf/arm-softmmu/../shared/kernelinfo/procinfo_generic/procinfo.ini
Total Sections: 1
Match 3.2.0-4-versatile
swapper task @ [c0393310] 

Thanks,
Cory

decaf live migration

I recently live migration on decaf, but always complain when starting, as follows:

zk@instance:~$ ~/DECAF/decaf/i386-softmmu/qemu-system-i386 -monitor stdio -m 4096 -usbdevice tablet -vnc :2 -netdev user,id=mynet -device ne2k_pci,netdev=mynet ~/WinXP_clean_snapshot.qcow2 -incoming tcp:0:6666
QEMU 1.0 monitor - type 'help' for more information
(qemu) inside bdrv open, drv addr= 0x4532dc90, size= 4294967296
fs_open error!
inside bdrv open, drv addr= 0x45492630, size= 0
fs_open error!
inside vmi init
ERROR: invalid runstate transition: 'running' -> 'inmigrate'
Aborted (core dumped)

Is it because decaf does not support live migration?

API tracer

It seems that only the Windows API function can be hooked. I am trying to hook the underlying c language implementation of PHP, I just cannot get any reult. Does decaf only hook Windows API?

What's the difference between PANDA and DECAF

Does DECAF have any advantages over PANDA https://github.com/panda-re/panda

Loading plugin causes crash

I can load my plugin fine a few times, but when I unload the plugin and then reload it a couple of times, decaf seg faults.

The full process goes:

1. load_plugin A
2. .... use plugin ....
3. unload_plugin
4. Go back to step 1 and repeat

Any ideas why it is seg_faulting?

unknown command:'load_plugin'

I installed Qemu1.0 and DECAF1.9 on my ubuntu 12.04.
I converted a vmdk image to qcow2 format by: qemu-img convert win7.vmdk -O qcow win7.img.
And started it by: ./qemu-­system­-i386 -­monitor stdio -­m 512 ­-netdev user,id=mynet -­device rtl8139,netdev=mynet win7.img.
Windows 7 started but when i tried to load a plugin by:load_plugin .It said unknown command, as follow.
I followed https://code.google.com/archive/p/decaf-platform/wikis/build_conf.wiki to install qemu and decaf.Both Windows 7 and Ubuntu are 32bit.
What's the problem?

Running performance benchmarks on DECAF

Firstly, thank you for making Decaf open-source. I managed to get it running on my machine!

I am wondering whether you can share your experiences with running performance benchmarks on the system. Essentially, I want to measure the overhead of Decaf on my machine. Are there any tips and tricks that I should keep in mind please?

In particular, when I launch DECAF, I am struggling to actually start any of my performance experiments due to the significant overhead incurred (with tainting). In particular, I can't even start a terminal due to the overhead. Is this expected?

Moreover, I am running Ubuntu as a guest. Is support as stable for Linux as for Windows? I have VMI enabled.

Thanks!

virtual machine stops in the callbacktests plugin

In two tests ("Block Begin Page" and "Block End From Page"), VM will automatically stop. User has to enter "continue" or "c" to resume the virtual machine, so the tests can continue. This is likely due to the cache flush mechanism in DECAF that calls vm_stop and vm_start.

How to use DECAF optioin？

If I want to use the load_plugin option when starting a virtual machine, how do I add parameters?

        case QEMU_OPTION_load_plugin:       // DECAF option
            load_plugin = optarg;
            break;

DroidScope TaintTracker

Currently, it looks like DroidScope taint tracker is not working (as for the wiki) and only the DalvikInstructionTracer does. however, in decaf there is a good support in terms taint tracking API. Is it possible to use those within DroidScope? How?

The version of QEMU in DECAF matters?

I want to use qemu in DECAF while using the firmadyne to run mips firmware, but the network doesn't work like this:
Please press Enter to activate this console. [ 54.832000] ------------[ cut here ]------------
[ 54.832000] WARNING: at net/sched/sch_generic.c:255 dev_watchdog+0x160/0x2c4()
[ 54.832000] NETDEV WATCHDOG: eth0 (pcnet32): transmit queue 0 timed out
[ 54.832000] Modules linked in:
[ 54.832000] Call Trace:
[ 54.832000] [<8056b7fc>] dump_stack+0x8/0x34
[ 54.832000] [<80124180>] warn_slowpath_common+0x78/0xa4
[ 54.832000] [<80124234>] warn_slowpath_fmt+0x2c/0x38
[ 54.832000] [<8044a5e4>] dev_watchdog+0x160/0x2c4
[ 54.832000] [<801302a4>] run_timer_softirq+0x188/0x254
[ 54.832000] [<80129f68>] __do_softirq+0xcc/0x190
[ 54.832000] [<8012a1c8>] do_softirq+0x5c/0x94
[ 54.832000] [<8012a414>] irq_exit+0x4c/0xbc
[ 54.832000] [<80101a44>] ret_from_irq+0x0/0x4
[ 54.832000] [<80101c80>] r4k_wait+0x20/0x40
[ 54.832000] [<801037a4>] cpu_idle+0x30/0x60
[ 54.832000] [<80706908>] start_kernel+0x3bc/0x3dc
[ 54.832000]
[ 54.832000] ---[ end trace 1e517aac3df85c47 ]---
[ 54.832000] pcnet32: eth0: transmit timed out, status 07fb, resetting
^C
Please press Enter to activate this console.

And I checked the version of QEMU used in DECAF is 1.0 while the QEMU downloading by apt-get is 2.11.1.
So I want to ask how can we update the version of QEMU used in DECAF

I have a question

Hello, I am Han-wool Lim who is a graduate student of Sejong University in Korea.

I am sending you an e-mail because I have a questions about DECAF.

I am now reading a paper from DECAF and studying about it.

I read a paper about DECAF and learned a lot and admired it.

Previously, there was a program called PANDA as a similar program.

But I read the paper and I don't know exactly the difference between DECAF and PANDA.

I would appreciate it if you could tell me the difference.

thanks

Demonstration of the research study Extract Me If You Can

Does anyone know if there is a technical demonstration of the paper Extract Me If You Can by Carmony, C., Hu, X., Yin, H., Bhaskar, A.V., & Zhang, M. (2016)?

I am trying to set DECAF correctly using the wiki presents on the github repository but without success.
Could anyone tell me how to properly start a virtual machine created with VirtualBox with iso Windows SP3 XP?

I tried the following commands in sequents:

1. sudo apt-get install qemu sudo apt-get build-dep qemu
2. sudo apt-get install binutils-dev
3. sudo apt-get install libboost-all-dev
4. ./configure make
5. VBoxManage clonehd --format RAW Windows-SP3-XP-NS-Project.vdi img.raw qemu-img convert -f raw win.img -O qcow2 win.qcow
6. not having DECAF/trunk/i386-­softmmu i used the qemu-­system­-i386 command of sudo apt-get install -y qemu-system-i386

I'm using Ubuntu 22

when make receive some problome

GEN i386-softmmu/config-devices.mak
GEN arm-softmmu/config-devices.mak
GEN mips-softmmu/config-devices.mak
GEN config-all-devices.mak
Building Sleuthkit, please wait...
GEN Sleuthkit
PREP Sleuthkit headers/libs
GEN config-host.h
GEN trace.h
GEN qemu-options.def
GEN qmp-commands.h
File "/home/tower/DECAF/decaf/scripts/qapi-commands.py", line 376
except getopt.GetoptError, err:
^
SyntaxError: invalid syntax
Makefile:214: recipe for target 'qmp-commands.h' failed
make: *** [qmp-commands.h] Error 1

when i use python3 can receive these problome.
I resolve it when use python2 then run make (run make in conda virtual env)

how to get the value of mips_pgd_current in procinfo.ini

I have read the code in procinfo.c, but no information relevant to the value mips_pgd_current is present.
I want to know how to get this value, thanks a lot!

Does it support Android versions >= 6?

Hi guys,

Just curious does DroidScope support Android 6 or later versions?

Thanks!

find_shadow_arg in tcg_taint.c

Hi, I use the taint analysis of DECAF these days. During test, I find no new instructions are inserted. After I check the I see that the find_shadow_arg function in tcg_taint.c always return 0 because of the check on temp in TCGContext:
TCGv find_shadow_arg(TCGv arg)
{
if (arg < tcg_ctx.nb_globals)
return shadow_arg[arg];

/* Check if this temp is allocated in the context */
if (!tcg_ctx.temps[arg].temp_allocated)
return 0;

However, in QEMU, most instructions are handled in tcg-op.h and do not allocate new TCGTemp such as qemu_ld and qemu_st (In fact, is allocated but freed). Therefore, no new instructions can be added because no shadow args are found:
case INDEX_op_qemu_ld32:
arg0 = find_shadow_arg(gen_opparam_ptr[-3]);
if (arg0) {
/* Patch qemu_ld* opcode into taint_qemu_ld* */
gen_opc_ptr[-1] += (INDEX_op_taint_qemu_ld8u - INDEX_op_qemu_ld8u);

Could you provide some advices to help me to use DECAF?
Thanks~

trace_ reader build error

Hi!

now i am trying to build trace_reader(tracecap is done)
but i have some trouble with building DECAF plugin.

I can build tracecap plugin and extract binary instruction from decaf.
And next, i try to build trace_reader but there is an error

libdasm.o: In function `get_operand_string':
/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:602: undefined reference to `MODE_CHECK_OPERAND'
/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:676: undefined reference to `MODE_CHECK_ADDR'
libdasm.o: In function `get_mnemonic_string':
/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:1068: undefined reference to `MODE_CHECK_OPERAND'
/DECAF/decaf/plugins/tracecap/trace_reader/libdasm.c:1088: undefined reference to `MODE_CHECK_OPERAND'

Have i missed something?

uninitialized class member in decaf/shared/vmi.h make hook unstable

in head file, decaf/shared/vmi.h
31 class module{
32 public:
33 char name[VMI_MAX_MODULE_PROCESS_NAME_LEN];
34 char fullname[VMI_MAX_MODULE_FULL_NAME_LEN];
35 uint32_t size;
36 uint32_t codesize; // use these to identify dll
37 uint32_t checksum;
38 uint16_t major;
39 uint16_t minor;
40 bool symbols_extracted;
41 unordered_map < uint32_t, string> function_map_offset;
42 unordered_map < string, uint32_t> function_map_name;
43 unsigned int inode_number;
44
45 module()
46 {
47 this->inode_number = 0;
48 }
49 };

symbols_extracted is uninitialized, which will make class member symbols_extracted' default value be true or false. plugin hookapitests may works well in some times , but always failed in my testing.
path is:

 module()
{
      this->inode_number = 0;
      this->symbols_extracted=false;
 }

pls submit the code, thanks.

Is there any way to implement VMI without using procinfo_generic?

In order to implement Linux VMI, DECAF needs to compile procinfo.ko and run it in the target kernel. But in my case, I have a restricted virtual machine and cannot get the kernel's source code and it is not easy to insert modules on it.

So I'm curious, Is there a theoretically feasible way to get these procinfo(address and structure offset, etc.) directly from the QEMU level without extra work?

NOTE: This is not a request for DECAF enhancement, just a free technical discussion.

shadow memory

There is a place on shadow memory that doesn't quite understand, as described below:

Take memory size 4G as an example.

In taint_memory.h：
#define BITPAGE_LEAF_BITS TARGET_PAGE_BITS
#define BITPAGE_MIDDLE_BITS (32-TARGET_PAGE_BITS)/2

BITPAGE_LEAF_BITS=12，BITPAGE_MIDDLE_BITS=(32-12)/2=10

//definition of leaf node
typedef struct _tbitpage_leaf {
uint8_t bitmap[2 << BITPAGE_LEAF_BITS]; //bitmap[2^13]
} tbitpage_leaf_t;
The bitmap size is 2^13bytes（8KB）

/* Middle node for holding memory taint information */
typedef struct _tbitpage_middle {
tbitpage_leaf_t *leaf[2 << BITPAGE_MIDDLE_BITS]; //leaf[2^11]
} tbitpage_middle_t;
Each middle node contains 2^11 leaf nods。

/* Root node for holding memory taint information */
tbitpage_middle_t **taint_memory_page_table = NULL;

static void allocate_taint_memory_page_table(void) {
if (taint_memory_page_table) return; // AWH - Don't allocate if one exists
taint_memory_page_table_root_size = ram_size >> (BITPAGE_LEAF_BITS + BITPAGE_MIDDLE_BITS); //ram_size=2^32,taint_memory_page_table_root_size=2^10
taint_memory_page_table = (tbitpage_middle_t *)
g_malloc0(taint_memory_page_table_root_size * sizeof(void));
allocate_leaf_pool();
allocate_middle_pool();
middle_nodes_in_use = 0;
leaf_nodes_in_use = 0;
}

In the function allocate_taint_memory_page_table(), we assign the size of the root node,ram_size = 2^32，taint_memory_page_table_root_size = ram_size >> (BITPAGE_LEAF_BITS + BITPAGE_MIDDLE_BITS)=(2^32)> > (12 + 10)= 2^10 = 1024

Qeustion1:Generally, the page size of 4G RAM is 4KB, and the size of a leaf node defined here is not equal to the size of a page. Why?
Qeustion2:From the above allocation, if the ram_size is 4G, then the size of the entire shadow memory should be 2^13 * 2^11 * 2^10 bytes = 2^34bytes = 16G > ram_size, which is wrong or deliberate So designed?

Error When creating qemu-system-arm vm

Below is the command:
sudo ./qemu-system-arm -m 256 -M virt -kernel /home/test/tools/firmware-analysis-toolkit/firmadyne/binaries//zImage.armel -drive if=none,file=/home/test/tools/firmware-analysis-toolkit/firmadyne/scratch//1//image.raw,format=raw,id=rootfs -device virtio-blk-device,drive=rootfs -append "root=/dev/vda1 console=ttyS0 nandsim.parts=64,64,64,64,64,64,64,64,64,64 rdinit=/firmadyne/preInit.sh rw debug ignore_loglevel print-fatal-signals=1 user_debug=31 firmadyne.syscall=0" -device virtio-net-device,netdev=net0 -netdev tap,id=net0,ifname=tap1_0,script=no -device virtio-net-device,netdev=net1 -netdev socket,id=net1,listen=:2001 -device virtio-net-device,netdev=net2 -netdev socket,id=net2,listen=:2002 -device virtio-net-device,netdev=net3 -netdev socket,id=net3,listen=:2003。

And the error is as below:

droidscope: is the taint analysis plugin available now?

Hi, is the taint analysis plugin available now?

Thanks

DECAF memory read/write callback not correct

The memory read/write callback is not correct. It looks like the memory read/write trace is complete. But the value we extracted is not correct for memory read.

Compilation error on Ubuntu

I'm trying to compile DECAF on my i686 Ubuntu 14.04 machine, and am facing some compiler issues.

In particular, I am getting the following error:

In file included from DECAF/decaf/softmmu_template.h:398:0,
                 from DECAF/decaf/target-i386/op_helper.c:5002:
DECAF/decaf/softmmu_taint_template.h: In function ‘taint_slow_stl_mmu’:
DECAF/decaf/softmmu_taint_template.h:469:13: warning: passing argument 1 of ‘__taint_stl_raw’ makes pointer from integer without a cast [enabled by default]
             glue(glue(__taint_st, SUFFIX), _raw)((unsigned long)(addr+addend),addr);
             ^
In file included from DECAF/decaf/target-i386/../shared/tainting/taintcheck_opt.h:23:0,
                 from DECAF/decaf/target-i386/op_helper.c:31:
DECAF/decaf/target-i386/../shared/tainting/taint_memory.h:114:14: note: expected ‘void *’ but argument is of type ‘long unsigned int’
 void REGPARM __taint_stl_raw(void * p, gva_t vaddr);
              ^
In file included from DECAF/decaf/softmmu_template.h:398:0,
                 from DECAF/decaf/target-i386/op_helper.c:5005:
DECAF/decaf/softmmu_taint_template.h: In function ‘__taint_ldq_mmu’:
DECAF/decaf/softmmu_taint_template.h:143:13: warning: passing argument 1 of ‘__taint_ldq_raw’ makes pointer from integer without a cast [enabled by default]
             glue(glue(__taint_ld, SUFFIX), _raw)((unsigned long)(addr+addend),addr);
             ^
In file included from DECAF/decaf/target-i386/../shared/tainting/taintcheck_opt.h:23:0,
                 from DECAF/decaf/target-i386/op_helper.c:31:
DECAF/decaf/target-i386/../shared/tainting/taint_memory.h:106:14: note: expected ‘void *’ but argument is of type ‘long unsigned int’
 void REGPARM __taint_ldq_raw(void * p, gva_t vaddr);
              ^
In file included from DECAF/decaf/softmmu_template.h:398:0,
                 from DECAF/decaf/target-i386/op_helper.c:5005:
DECAF/decaf/softmmu_taint_template.h: In function ‘taint_slow_ldq_mmu’:
DECAF/decaf/softmmu_taint_template.h:235:13: warning: passing argument 1 of ‘__taint_ldq_raw’ makes pointer from integer without a cast [enabled by default]
             glue(glue(__taint_ld, SUFFIX), _raw)((unsigned long)(addr+addend),addr);
             ^
In file included from DECAF/decaf/target-i386/../shared/tainting/taintcheck_opt.h:23:0,
                 from DECAF/decaf/target-i386/op_helper.c:31:
DECAF/decaf/target-i386/../shared/tainting/taint_memory.h:106:14: note: expected ‘void *’ but argument is of type ‘long unsigned int’
 void REGPARM __taint_ldq_raw(void * p, gva_t vaddr);
              ^
In file included from DECAF/decaf/softmmu_template.h:398:0,
                 from DECAF/decaf/target-i386/op_helper.c:5005:
DECAF/decaf/softmmu_taint_template.h: In function ‘__taint_stq_mmu’:
DECAF/decaf/softmmu_taint_template.h:382:13: warning: passing argument 1 of ‘__taint_stq_raw’ makes pointer from integer without a cast [enabled by default]
             glue(glue(__taint_st, SUFFIX), _raw)((unsigned long)(addr+addend),addr);
             ^
In file included from DECAF/decaf/target-i386/../shared/tainting/taintcheck_opt.h:23:0,
                 from DECAF/decaf/target-i386/op_helper.c:31:
DECAF/decaf/target-i386/../shared/tainting/taint_memory.h:115:14: note: expected ‘void *’ but argument is of type ‘long unsigned int’
 void REGPARM __taint_stq_raw(void * p, gva_t vaddr);
              ^
In file included from DECAF/decaf/softmmu_template.h:398:0,
                 from DECAF/decaf/target-i386/op_helper.c:5005:
DECAF/decaf/softmmu_taint_template.h: In function ‘taint_slow_stq_mmu’:
DECAF/decaf/softmmu_taint_template.h:469:13: warning: passing argument 1 of ‘__taint_stq_raw’ makes pointer from integer without a cast [enabled by default]
             glue(glue(__taint_st, SUFFIX), _raw)((unsigned long)(addr+addend),addr);
             ^
In file included from DECAF/decaf/target-i386/../shared/tainting/taintcheck_opt.h:23:0,
                 from DECAF/decaf/target-i386/op_helper.c:31:
DECAF/decaf/target-i386/../shared/tainting/taint_memory.h:115:14: note: expected ‘void *’ but argument is of type ‘long unsigned int’
 void REGPARM __taint_stq_raw(void * p, gva_t vaddr);
              ^
DECAF/decaf/target-i386/op_helper.c: In function ‘helper_flds_FT0’:
DECAF/decaf/target-i386/op_helper.c:3648:6: error: frame pointer required, but reserved
 void helper_flds_FT0(uint32_t val)
      ^
In file included from DECAF/decaf/target-i386/op_helper.c:22:0:
DECAF/decaf/dyngen-exec.h:64:20: note: for ‘env’
 register CPUState *env asm(AREG0);

It seems like this issue has been encountered before: https://groups.google.com/forum/#!msg/decaf-platform-discuss/eFYLlRxaOGg/hv2_2Y6tJaUJ

Finally, my configure command is as follows:

./configure --enable-tcg-taint --target-list=i386-softmmu

Can anybody help me resolve the issue please?

decaf crashes after hookapitests is unloaded

Guest OS: Windows XP
Host environment: Ubuntu 15.04, 64-bit

Symptom:

In DECAF monitor window:
load_plugin ../plugins/hookapitests/hookapitests.so
do_hookapitests cmd.exe

run cmd.exe in guest

after cmd.exe appears, in DECAF monitor window run
unload_plugin

DECAF will crash in a few seconds.

symbol extraction bug and fix

Hi, during my project with DECAF, I found sometimes DECAF cannot extract symbols for some modules, e.g. user32.dll, ws2_32.dll, etc. After simple investigation, seems there is a little bug existed in the module class in shared/vmi.h. It doesn't intialize most of its member variables, so the symbols_extracted variable can sometimes be intialized as true randomly. And it will cause DECAF not extracting symbols for that module. The fix will be intialization in the constructor of module class. Please confirm the bug. Thanks.

Optional Flags in cmds.h

Hello,

I am working on a plugin that uses DECAF and was wondering whether there are flag options for the plugin_cmd.h. From what I have seen in DECAFs cmd file it does not appear so but I was hoping that there might be. Please let me know if this is possible.

Also if you can link any information on how to build a plugin for DECAF that would help immensely.

Respectfully,
Dakota Fulp

Where does the parameter proc_exec_connector in procinfo.ini file come from?

I compiled the procinfo.c into the linux kernel, and the dmesg log is below, no proc_exec_connector parameter is found.
So I want to ask how can I figure out this value.

[ 0.220574] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.221914] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[ 0.222650] romfs: ROMFS MTD (C) 2007 Red Hat, Inc.
[ 0.227781] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[ 0.228425] io scheduler noop registered
[ 0.228680] io scheduler cfq registered (default)
[ 0.229279] strName = 4.1.17
[ 0.229279] init_task_addr = 3227491000
[ 0.229279] init_task_size = 872
[ 0.229279] ts_tasks = 292
[ 0.229279] ts_pid = 400
[ 0.229279] ts_tgid = 404
[ 0.229279] ts_group_leader = 432
[ 0.229279] ts_thread_group = 488
[ 0.229279] ts_real_parent = 408
[ 0.229279] ts_mm = 300
[ 0.229279] ts_stack = 4
[ 0.230525] module_name = 12
[ 0.230525] module_size = 208
[ 0.230525] module_init = 200
[ 0.230525] module_list = 4
[ 0.231035] ts_real_cred = 616
[ 0.231035] ts_cred = 620
[ 0.231035] ts_comm = 624
[ 0.231035] cred_uid = 4
[ 0.231035] cred_gid = 8
[ 0.231035] cred_euid = 20
[ 0.231035] cred_egid = 24
[ 0.231725] mm_mmap = 0
[ 0.231725] mm_pgd = 32
[ 0.231725] mm_arg_start = 136
[ 0.231725] mm_start_brk = 124
[ 0.231725] mm_brk = 128
[ 0.231725] mm_start_stack = 132
[ 0.232340] vma_vm_start = 0
[ 0.232340] vma_vm_end = 4
[ 0.232340] vma_vm_next = 8
[ 0.232340] vma_vm_file = 80
[ 0.232340] vma_vm_flags = 40
[ 0.232340] vma_vm_pgoff = 76
[ 0.233120] file_dentry = 12
[ 0.233120] file_inode = 40
[ 0.233120] dentry_d_name = 24
[ 0.233120] dentry_d_iname = 44
[ 0.233120] dentry_d_parent = 16
[ 0.233120] ti_task = 12
[ 0.233120] inode_ino = 28
[ 0.304148] vma_link = 3221761964
[ 0.304378] vma_adjust = 3221762088
[ 0.304517] remove_vma = 3221760012
[ 0.304670] modules = 3227509296
[ 0.304763] trim_init_extable = 3222849576

Live snapshots not working

Hello! I built your code with --enable-vmi on Ubuntu 18.04 64-bit and I see a few DLLs successfully scanned for exports when booting Windows 7 SP1 32-bit, so VMI seems to be working. It appears however that snapshots of the running guest do not work properly.

I run my VM with the following options:

i386-softmmu/qemu-system-i386 -m 2048 -monitor stdio -usb -device usb-tablet -netdev user,id=mynet -device rtl8139,netdev=mynet -vnc 127.0.0.1:0 images/win7sp1_x86.qcow2

I tried also changing the disk caching policy to unsafe and disabling the USB tablet pointer with no effect. So when I try to capture a snapshot with savevm mysnap and then load it later with command-line option -loadvm mysnap the display is eventually restored correctly (and some VMI info on libraries is displayed) but the machine is left unresponsive. Checking with info cpus I can see the instruction pointer changing, so perhaps it's not totally frozen but I have no clue of what is going on.

To rule out QEMU known problems, I downloaded the tarball for version 1.0.0 that apparently you use in DECAF and compiled it with --target-list=i386-softmmu --disable-docs --disable-kvm --disable-libiscsi --disable-xen (basically I disabled all the components that led to compilation failures on my machine; I also added LIBS+=-lrt to Makefile.target to get around a linking error). Snapshots can be saved and restored correctly for the same image within vanilla QEMU. I tried restoring a QEMU snapshot into DECAF with no luck (guest still unresponsive), while for the other way around QEMU legitimately complains about the DECAF section:

(qemu) Unknown savevm section or instance 'DECAF' 0
qemu-system-i386: Error -22 while loading VM state

In the step "3. VMI configuration for Linux", after copying procinfo.c and Makefile into guest OS, errors occur while running the command "make".

My system information:
Host OS: Ubuntu20.04
Guest OS: Ubuntu18.04/16.04/14.04/12.04 ( try four versions, but all failed while "make" )

In Ubuntu 18.04, the error information goes like this:
/home/username/desktop/DECAF/procinfo.c:21:22:error: array type has incomplete element type 'struct jprobe'
static struct jprobe jprobes[JPROBE_TOTAL]; (jprobe error)

In Ubuntu 16.04/14.04/12.04, the same error is:
error: implicit declaration of function 'jprobe_return'
error:'struct module' has no member named 'core_size'/'module core'
with other similar errors.

what version of kernel should I use?(like linux-headers-5.4.0-150-generic)
And how to deal with the "make" error?

The meaning of the output of "keylogger"？

I am very confused that why it produces much output while i didn't send any keystrokes into the guest system.

This is the output: