tencent/habomalhunter

HaboMalHunter is a sub-project of Habo Malware Analysis System (https://habo.qq.com), which can be used for automated malware analysis and security assessment on the Linux system.

HaboMalHunter: Habo Linux Malware Analysis System

(中文版本请参看这里)

参与贡献

腾讯开源激励计划 鼓励开发者的参与和贡献，期待你的加入。

Introduction

HaboMalHunter is a sub-project of Habo Malware Analysis System (https://habo.qq.com), which can be used for automated malware analysis and security assessment on the Linux system. The tool help security analyst extracting the static and dynamic features from malware effectively and efficiently. The generated report provides significant information about process, file I/O, network and system calls.

Features

The tool can be used for the static and dynamic analysis of ELF files on the Linux x86/x64 platform.

Static analysis

1. Basic Information: md5, name, file type, size and SSDEEP.
2. SO Files Dependency: SO files information (only applied for dynamic linked files).
3. Strings Information.
4. ELF Header and Entry Point.
5. IP and PORTS
6. ELF Segment, Section and Hash.
7. Source File Names.

Dynamic analysis

1. Starting and Termination: Time Stamps and Elapsed Time.
2. Processes Information: clone, execve and exit etc.
3. File I/O: open, read, write and delete etc.
4. Network: TCP, UDP, HTTP and HTTPS etc.
5. Typical Malicous Actions: self deletion, midification and lock.
6. API Information: getpid, system, dup and other libc functions.
7. syscall sequences.

Screenshot

1. The HTML report.

2. The JSON report.

Demo

1.Setup Enviroment

The tool will run on the VirtualBox 5.1 with Ubuntu 14.04 LTS.

in order to install thrid party software, please execute the following command after obtaining the code:

root# cd ./util/update_image
root# bash update_image.sh

2.Get Source Code

git clone https://github.com/Tencent/HaboMalHunter.git

3.Compile

Firstly, please upload the source code into the VM. Execute the following command with root permision under the /root directory.

cp -ra /media/sf_Source/* .

The command will compile and package the source code, and then will generate two zip files.

bash package.sh

4.Analysis

using ./test/bin/read.32.elf to make a test. The second command will copy report and log outside the VM.

python AnalyzeControl.py -v -l ./test/bin/read.32.elf
cp ./log/output.zip /media/sf_Source/

Among the result, output.static is static analysis result, output.dynamic is dynamic analysis result, and system.log is runtime log. Users can also upload samples to the Habo Malware Analysis System (https://habo.qq.com) to get a brief report.

Future Work

1. [done] Memory Analysis.
2. More YARA rules (./utils/yara/malware/)
3. [done] HTML output format

Errors and Issues

1. Malware Analysis should be done inside a Virtual Machine enviroment and Intel-VT should be enabled on the host's BIOS. We shall not be liable to the damage of the analysed malware.
2. VirtualBox 5.1 is recommended.
3. The tool will also generate dynamic log, which contains one error message, for ELF files which can not be executed, such as so files.

哈勃分析系统(HaboMalHunter)

功能描述

HaboMalHunter是哈勃分析系统 (https://habo.qq.com) 的开源子项目，用于Linux平台下进行自动化分析、文件安全性检测的开源工具。使用该工具能够帮助安全分析人员简洁高效的获取恶意样本的静态和动态行为特征。分析报告中提供了进程、文件、网络和系统调用等关键信息。

功能清单

开源代码支持Linux x86/x64 平台上的ELF文件的自动化静态动态分析功能。

静态分析

1. 基础信息：包括文件md5，名称，类型，大小和SSDEEP等信息。
2. 依赖so信息：对于动态链接的文件，输出依赖的so信息。
3. 字符串信息
4. ELF头信息，入口点
5. IP和端口信息
6. ELF段信息，节信息和hash值
7. 源文件名称

动态分析

1. 动态运行启动结束信息：耗时等
2. 进程信息：clone系统调用，execve调用，进程创建结束等
3. 文件操作信息：打开，读取，修改，删除等文件IO操作
4. 网络信息：TCP, UDP, HTTP, HTTPS, SSL等信息
5. 典型恶意行为：自删除，自修改和自锁定等
6. API信息：getpid, system, dup 等libc函数调用
7. syscall 序列信息

Demo

1.环境配置

使用哈勃Linux开源版进行病毒分析，需要首先制作用于运行病毒的虚拟机环境。切勿直接在真实环境下运行和分析病毒。项目默认使用VirtualBox 5.1 运行Ubuntu 14.04 LTS 作为分析环境。

安装相关的软件，获取源代码之后，请在虚拟机内以root身份运行如下命令：

root# cd ./util/update_image
root# bash update_image.sh

2.获取源代码

使用git工具获取源代码。

git clone https://github.com/Tencent/HaboMalHunter.git

3.编译源代码

大部分源代码是python, 有一部分c代码需要进行编译和打包。 首先将代码上传到虚拟机中。 使用root身份，在/root/ 目录下使用命令，如图：

cp -ra /media/sf_Source/* .

运行命令,进行编译和打包，会输出AnalyzeControl_1129.zip 和test_1129.zip 两个文件, 如图:

bash package.sh

4.进行分析

本次使用测试文件 ./test/bin/read.32.elf 进行测试。使用如下命令: 其中第二条命令会将分析结果拷贝到虚拟机外，用于分析人员阅读。

python AnalyzeControl.py -v -l ./test/bin/read.32.elf
cp ./log/output.zip /media/sf_Source/

分析结果中，output.static 是静态分析结果，output.dynamic 是动态分析结果，system.log是运行时的日志。同时也可以结合哈勃分析系统 (https://habo.qq.com) 中的结果展示进行样本分析。

未来规划

1. [已完成] 希望使用volatility和LiME进行内存分析
2. 希望增加更多的病毒规则(./util/yara/malware)
3. [已完成] 希望将输出的json数据格式转化成为HTML页面进行展示

已知故障和错误列表

1. 分析病毒请在虚拟机环境下进行，并在BIOS设置中开启Intel-VT功能，对因运行病毒引起的任何软件安全问题，本项目不承担责任。
2. 推荐使用VirtualBox 5.1以上版本运行虚拟机。
3. 对于无法运行的ELF文件，例如so文件，哈勃分析系统默认会生成动态日志，但是里面只有无法运行的报错信息。

habomalhunter's Issues

7z: command not found

运行 bash package.sh 报错如下
package.sh: line 60: 7z: command not found

[Feature] memory analysis

Memory Analysis

Volatility is an tool for memory analysis. And LiME is a tool for memory dump.
Both of them will help HaboMalHunter to do memory analysis after the sample execution.

how can i use in vmware

i look it like lemon sandbox

ConfigParser.NoSectionError: No section: 'main'

habo@habo-VirtualBox:~/HaboMalHunter$ sudo python AnalyzeControl.py -v -l ./test/bin/read.64.elf
Traceback (most recent call last):
File "AnalyzeControl.py", line 688, in
ret = main(len(sys.argv), sys.argv)
File "AnalyzeControl.py", line 615, in main
cfg = init_arguments(argv)
File "AnalyzeControl.py", line 156, in init_arguments
cfg = init_cfg(args.config_path,args)
File "AnalyzeControl.py", line 96, in init_cfg
for k,v in conf_parser.items(SECTION_DEF):
File "/usr/lib/python2.7/ConfigParser.py", line 642, in items
raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'main'

只开源了Linux沙箱？

在腾讯的HABO网站上，看到了支持常规的文件格式和Windows应用的格式？
只开源了Linux，Windows是否有计划开源？

Support for arm binary

Will like to request to support arm, mips binary for the analysis.

Hardcoded file separator

HaboMalHunter/iot_hunter/dynamic_analysis/PluginManager.py

Line 21 in 74fd9fb

self.plugin_folder = '\\DynamicPlugins\\'

The path split symbol problem is different for different operating systems. Hard-coded file separators should not be used. Instead, use a platform-independent API provided by the language library,such as os.path.join or os.sep

Encounter issue with execution of HaboMalHunter

Following the steps of the readme, when i execute the AnalyzeControl.py, i encounter the following error. Kindly assist.

python AnalyzeControl.py -e ./config.ini -v -l ./test/bin/read.64.elf
Traceback (most recent call last):
File "AnalyzeControl.py", line 57, in
import static
ImportError: No module named static

test

OSX sandbox

Hello Guys,

Can you please tell me how to setup the osx sandbox for mac malware samples.
.
It would be very grateful for us, waiting for positive reply.
.
Thanks & Regards
Seantree

Errors on execution

Followed the instructions on Readme but receivedthe following error messages upon execution.

~/HaboMalHunter$ sudo python AnalyzeControl.py -v -l ./test/bin/read.32.elf
Traceback (most recent call last):
File "AnalyzeControl.py", line 688, in
ret = main(len(sys.argv), sys.argv)
File "AnalyzeControl.py", line 615, in main
cfg = init_arguments(argv)
File "AnalyzeControl.py", line 156, in init_arguments
cfg = init_cfg(args.config_path,args)
File "AnalyzeControl.py", line 96, in init_cfg
for k,v in conf_parser.items(SECTION_DEF):
File "/usr/lib/python2.7/ConfigParser.py", line 642, in items
raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'main'

failed to locate target loader target_loader.32.elf

Hi, may I ask how to fix this problem?
Does it matter when I analyze other elf files?

maybe wrong output from habo.qq.com

The result is from https://habo.qq.com/
and the ELF source code is in the picture.
Why the habo has "文件行为"？

Any Malware Compiled with PIE would Evade Dynamic Analysis

Current, HaboMalHunter uses file command output to check whether target is executable here. However, file command would regard any executable complied with PIE as shared object.

That means HoboMalHunter would not run dynamic analysis for any malware compiled with PIE.

Example

Following are two executables from the same source code and compiled w/ and w/o "-no-pie" flag.

$ cat test.c
#include<stdio.h>
int main() {
    puts("test pie");
}

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.3.0-27ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04)

Compiled with PIE

pie.zip

$ gcc test.c -o pie
$ file pie
pie: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 3.2.0, BuildID[sha1]=1d677eb3a12c5b5fe6d135fe815efde29474e5ad, not stripped

Analysis result of Habo is available here

Compiled without PIE

nopie.zip

$ gcc test.c -no-pie -o nopie
$ file nopie
nopie: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 3.2.0, BuildID[sha1]=701f6befacfc89af3debea2ce2bfa4c3a74bb371, not stripped

Analysis result of Habo is available here

Missing target_loader.32.elf file

when I run the demo using python AnalyzeControl.py -v -l ./test/bin/read.32.elf,
I get the false:
cannot load the /root/bin/target_loader.32.elf
and I found in /root/bin ,no exist the target_loader.32.elf file only exist the target_loader.64.elf file.

Undefined name 'self' in ./util/stat/is_exec/is_exec.py

flake8 testing of https://github.com/Tencent/HaboMalHunter on Python 2.7.14

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./AnalyzeControl.py:556:15: F821 undefined name 'main_dynamic'
		dest_file = main_dynamic
              ^
./AnalyzeControl.py:560:43: F821 undefined name 'main_dynamic'
		log.info("main dynamic log updated %s", main_dynamic)
                                          ^
./util/stat/is_exec/is_exec.py:40:4: F821 undefined name 'self'
			self.log.error("CalledProcessError: %s",str(e))
   ^
3     F821 undefined name 'main_dynamic'
3

transfer log to html

a module that can transfer habo log to html

habo cant be compiled:yara import error

habo@habo-VirtualBox:$ yara -v
yara 3.4.0
habo@habo-VirtualBox:$ cd /home/habo/HaboMalHunter
habo@habo-VirtualBox:~/HaboMalHunter$ python AnalyzeControl.py -v -l ./test/bin/read.32.elf
Traceback (most recent call last):
File "AnalyzeControl.py", line 57, in
import static
File "/home/habo/HaboMalHunter/static/init.py", line 14, in
from static_analyzer import *
File "/home/habo/HaboMalHunter/static/static_analyzer.py", line 27, in
import yara
ImportError: No module named yara