airbus-seclab/bincat

Binary code static analyser, with IDA integration. Performs value and taint analysis, type reconstruction, use-after-free and double-free detection

bincat's Issues

Test docker install and remote BinCAT

Test installation procedures from scratch

Targets are

• Windows
• Ubuntu
• Debian
• Arch linux
• Docker
• MacOS ?

Missing python requirements?

I am attempting to try the tutorial. I'm using the docker container because that seems like the quickest way to get started. However, my IDAPython is missing modules that the bincat plugin needs.

When I tried it with IDA's bundled python I was missing a module called requests which is needed to connect to the remote bincat analyzer.

I tried the instructions here:
https://github.com/airbus-seclab/bincat/blob/master/doc/install_plugin.md#manual-install

but I'm still missing more packages. When I open IDA it says:

__init__.py: No module named urllib3
Traceback (most recent call last):
  File "/home/scott/ida-6.95/python/ida_idaapi.py", line 509, in IDAPython_ExecScript
    execfile(script, g)
  File "/home/scott/.idapro/plugins/__init__.py", line 43, in <module>
    import urllib3
ImportError: No module named urllib3

I'm pretty sure that __init__.py is belongs to requests. How do I install the missing package for my IDA's bundled python?

sprintf("%x") doesn't propagate taint

in get_key:

.text:000008E3                 movzx   eax, byte ptr [esi]
.text:000008E6                 push    eax
.text:000008E7                 push    ebp             ; format
.text:000008E8                 push    edi             ; s
.text:000008E9                 call    _sprintf

if eax is tainted, the resulting string should be but is not in practice.

Add possible missing x86 relocations in ELF loader

Fix override section generation from plugin

Wait for #38 to know which syntax to use.

Fix taint computation with rainbow tainting

In x86 get_key:

.text:0000072A                 test    edx, edx

edx's LSB is tainted but the resulting zf is not.

Also, calls which manipulate tainted data are not tainted.

Check backward analysis

Pointer.combine problem

• X1 is a register with a S0x0 value
• an instruction sets x1[0,31] to G0

=> Bottom value :(

Create installation scripts/procedure for IDA 7

Installing with IDA on Wine

Just letting you know I figured a way to make the plugin load with the Windows version of IDA on Linux (with Wine). The issue I had at Recon turned out to be rather simple to fix. The only dependency I had to manually install was "pip". You may want to had to those steps in the installation for user using Wine.

• Download "get-pip.py" : https://bootstrap.pypa.io/get-pip.py
• Verify the signature of the script
• ~/.wine/drive_c/Python27/python.exe get-pip.py

After that the Windows install script works fine.

v0.7rc fails with IDA 7 32 bits

Traceback (most recent call last):
  File "/home/raph/.idapro/plugins/bcplugin.py", line 277, in procanalyzer_on_finish
    self.process_output()
  File "/home/raph/.idapro/plugins/bcplugin.py", line 299, in process_output
    self.finish_cb(self.outfname, self.logfname, self.cfaoutfname)
  File "/home/raph/.idapro/plugins/bcplugin.py", line 574, in analysis_finish_cb
    self.netnode["analyzer.log"] = f.read()
  File "/home/raph/.idapro/python/idabincat/netnode.py", line 147, in __setitem__
    d = self._compress(self._encode(value))
  File "/home/raph/.idapro/python/idabincat/netnode.py", line 94, in _encode
    return json.dumps(data)
  File "/home/raph/bin/ida7-32/python/lib/python27.zip/json/__init__.py", line 243, in dumps
  File "/home/raph/bin/ida7-32/python/lib/python27.zip/json/encoder.py", line 201, in encode
SystemError: Objects/stringobject.c:3904: bad argument to internal function

Encoding issues between IDA/bincat plugin/ini file

environment: win10 + IDA7.0
x86 can work,but armv7 is error.

error info

Using FLIRT signature: ARM library little endian
Propagating type information...
Function argument information has been propagated
The initial autoanalysis has been finished.
WARNING:bincat.gui.pluginoptions:IDAUSR not defined, using C:\Users\CKCat\AppData\Roaming\Hex-Rays\IDA Pro
Note: FormToPyQtWidget: importing 'sip' module into <module 'main' from ''>
INFO:bincat.plugin:IDABinCAT ready.
DEBUG:bincat-cfg:Reading config from C:\Users\CKCat\AppData\Roaming\Hex-Rays\IDA Pro\idabincat\conf\default.ini
DEBUG:bincat-cfg:Reading OS config from C:\Users\CKCat\AppData\Roaming\Hex-Rays\IDA Pro\idabincat\conf\linux-armv7.ini
INFO:bincat.gui:Launching the analyzer
Traceback (most recent call last):
File "C:/Users/CKCat/AppData/Roaming/Hex-Rays/IDA Pro/plugins\idabincat\gui.py", line 265, in launch_analysis
self.s.configurations[config_name] = self.s.edit_config
File "C:/Users/CKCat/AppData/Roaming/Hex-Rays/IDA Pro/plugins\idabincat\analyzer_conf.py", line 553, in wrap
f(self, *args, **kwargs)
File "C:/Users/CKCat/AppData/Roaming/Hex-Rays/IDA Pro/plugins\idabincat\analyzer_conf.py", line 599, in setitem
self._configs[name] = str(config)
File "C:/Users/CKCat/AppData/Roaming/Hex-Rays/IDA Pro/plugins\idabincat\analyzer_conf.py", line 407, in str
self._config.write(sio)
File "C:\python27-x64\Lib\ConfigParser.py", line 412, in write
key = " = ".join((key, str(value).replace('\n', '\n\t')))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-13: ordinal not in range(128)

Optimize flags computations

Don't update flags when they are updated (set/undef) without being tested

Handle UTF-8 paths in config

Currently the config parser is quite strict and limits the allowed characters for paths.

It should be permitted to use UTF-8 paths.

Add GUI to edit memory definitions

Defining memory for the initial configuration can only be done manually ATM.

It should be possible to define it in the IDA plugin.

Bonus points:

• allow allocation and value configuration by right clicking on function args (the plugin would need to know the ABI)

IDA plugin does not work well after IDB change

When closing and loading a new IDB, BinCAT gets confused.

Test and fix regressions from v0.6

Make BinCAT OCaml code compile under Windows

This should be doable. This issue will be used to track progress.

Handle imports in a more generic way in `interpreter.ml`

Currently, interpreter.ml checks wether is it reaching an import by matching on call/jmp targets:

      and process_vertices (vertices: Cfa.State.t list) (s: Asm.stmt): (Cfa.State.t list * bool) =
[...]
             | Jmp (A a) ->
		begin
		  try
		    let res = import_call vertices a (fun v -> Cfa.pred g (Cfa.pred g v)) fun_stack in
		    fun_stack := List.tl !fun_stack;
		    res
		  with Not_found ->
		    List.map (fun v -> v.Cfa.State.ip <- a; v) vertices, false		      
		end
		  
             | Jmp (R target) ->
		  fold_to_target (fun _a -> ()) vertices target (fun v -> Cfa.pred g (Cfa.pred g v))
			 
             | Call (A a) ->
		add_to_fun_stack a;
		begin
		  try		   
		    import_call vertices a (fun v -> Cfa.pred g v) fun_stack 
		  with Not_found ->
		    List.iter (fun v -> v.Cfa.State.ip <- a) vertices;
		    vertices, false
		end
	     | Call (R target) -> fold_to_target add_to_fun_stack vertices target (fun v -> Cfa.pred g v)
		
             | Return -> List.fold_left (fun (l, b) v ->
			     let v', b' = process_ret fun_stack v in
			     match v' with
			     | None -> l, b||b'
			     | Some v -> v::l, b||b') ([], false) vertices
				  
             | _       -> vertices, false

This code has several problems:

• import_call is using its second argument to determine the "return address". Here it means: right after the call instruction, or right after the instruction preceding the jmp to an import (typical case: call plt ; jmp import`. Unfortunately this only works on well-formed calls in x86 binaries and breaks on ARM plt which as several instructions.
• push import_addr ; ret does not detect the import

So, ideally, the code should:

• detect whenever the instruction pointer reaches an import (call/jmp/ret)
• use ABI info (from the stub ?) to determine how to return from the caller:
  • bx lr on ARM
  • ret on Intel
  • whatever the ABI dictates.

test_memcpy_push_ret in test_x86_stubs.py tests the push/ret call (armv7 branch)

PowerPC decoder

x86_64 decoder

Add missing ARMv8 instructions to pass get_key (tutorial program)

Add MMX and SSE instructions support

Add AARCH64 relocations in ELF loader

Stub memset()

Ability to load core dumps with ELF loader

This already works from IDA.

Symbolic expression domain

Polish documentation

• Polishing
• Install
• IDA Python BinCAT API ?
• Use cases, new tutorials

IDA Plugin should not use PE format in .ini until supported

Env: WIN7 X64  IDA7 
        x86_64-7.2.0-release-win32-seh-rt_v5-rev1_mingv64_GCC

Please Tell Me, How To Fix It?
Thank you.

==============================================


analyzer.log
------------------------------------------------------------------
[INFO]  main: BinCAT version v0.7
[EXCEPTION] parser: failed to load header 
Sys_error(": No such file or directory")
Raised by primitive operation at file "pervasives.ml", line 366, characters 28-54
Called from file "pervasives.ml" (inlined), line 374, characters 2-47
Called from file "c2newspeak/typedC.ml", line 345, characters 14-30
Called from file "frontend/parser.mly", line 135, characters 15-33
[ABORT] main: PE file format not implemented yet
Raised by primitive operation at file "utils/log.ml", line 157, characters 41-69
Called from file "main.ml", line 70, characters 19-76
Called from file "bincat.ml", line 25, characters 4-55
[EXCEPTION] main: Exception caught in main loop
Exceptions.Error("PE file format not implemented yet")
Raised at file "utils/log.ml", line 160, characters 4-32
Called from file "main.ml", line 70, characters 19-76
[STOP] nothing analyzed
------------------------------------------------------------------

IDAPythonLog
------------------------------------------------------------------
INFO:bincat.gui:Launching the analyzer
DEBUG:netnode:set: data length: 0xf86
DEBUG:netnode:set: chunk run: 0x1 to 0x3 (0x3 chunks)
DEBUG:netnode:set: data length: 0x22
DEBUG:netnode:set: data length: 0xf86
DEBUG:netnode:set: chunk run: 0x1 to 0x3 (0x3 chunks)
DEBUG:netnode:set: data length: 0x22
DEBUG:bincat.plugin:Using C:\Users\Administrator\Desktop\Dbgview.exe as source binary path
DEBUG:bincat.plugin:Current analyzer path: c:\users\admini~1\appdata\local\temp\tmpcvjx2ibincat
DEBUG:bincat.plugin:Generating .no files...
DEBUG:bincat.plugin:Initial header files: ['']
DEBUG:bincat.plugin.npkgen:Generating TNPK file in c:\users\admini~1\appdata\local\temp\tmpr2zzrcbincat-generate-header
ERROR:bincat.plugin.npkgen:Error encountered while running c2newspeak.
--- start of c2newspeak output ---
Fatal error: c:\\users\\admini~1\\appdata\\local\\temp\\tmpr2zzrcbincat-generate-header\\ida-generated.h:233#0: directive #pragma pack(push, 1) not supported yet, rewrite your code or try option --ignore-pragma

--- end of c2newspeak output ---
Traceback (most recent call last):
  File "C:/Users/Administrator/AppData/Roaming/Hex-Rays/IDA Pro/plugins\idabincat\npkgen.py", line 130, in generate_tnpk
    "pre-processed.c"], stderr=subprocess.STDOUT)
  File "C:\python27-x64\Lib\subprocess.py", line 219, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['c2newspeak', '--typed-npk', '-o', 'c:\\users\\admini~1\\appdata\\local\\temp\\tmpr2zzrcbincat-generate-header\\pre-processed.no', 'pre-processed.c']' returned non-zero exit status 1
WARNING:bincat.plugin:.no file containing type data for the file being analyzed could not be generated, continuing. The ida-generated header could be invalid.
DEBUG:bincat.plugin:Final npk files: []
DEBUG:bincat.plugin:Analyzer cmdline: [bincat c:\users\admini~1\appdata\local\temp\tmpcvjx2ibincat\init.ini c:\users\admini~1\appdata\local\temp\tmpcvjx2ibincat\out.ini c:\users\admini~1\appdata\local\temp\tmpcvjx2ibincat\analyzer.log]
DEBUG:bincat.plugin:Analyzer new state: Starting
DEBUG:bincat.plugin:Analyzer new state: Running
INFO:bincat.plugin:Analyzer: starting process
INFO:bincat.plugin:Analyzer started.
DEBUG:bincat.plugin:Analyzer new state: Not running
INFO:bincat.plugin:Analyzer process finished
ERROR:bincat.plugin:analyzer returned exit code=2
INFO:bincat.plugin:---- stdout ----------------
INFO:bincat.plugin:BinCAT v0.7

INFO:bincat.plugin:---- stderr ----------------
INFO:bincat.plugin:EXCEPTION: Exceptions.Error("PE file format not implemented yet")
Check log file for details [c:\users\admini~1\appdata\local\temp\tmpcvjx2ibincat\analyzer.log]

DEBUG:bincat.plugin:---- logfile ---------------
DEBUG:bincat.plugin:[INFO]  main: BinCAT version v0.7
DEBUG:bincat.plugin:[EXCEPTION] parser: failed to load header
DEBUG:bincat.plugin:Sys_error(": No such file or directory")
DEBUG:bincat.plugin:Raised by primitive operation at file "pervasives.ml", line 366, characters 28-54
DEBUG:bincat.plugin:Called from file "pervasives.ml" (inlined), line 374, characters 2-47
DEBUG:bincat.plugin:Called from file "c2newspeak/typedC.ml", line 345, characters 14-30
DEBUG:bincat.plugin:Called from file "frontend/parser.mly", line 135, characters 15-33
DEBUG:bincat.plugin:[ABORT] main: PE file format not implemented yet
DEBUG:bincat.plugin:Raised by primitive operation at file "utils/log.ml", line 157, characters 41-69
DEBUG:bincat.plugin:Called from file "main.ml", line 70, characters 19-76
DEBUG:bincat.plugin:Called from file "bincat.ml", line 25, characters 4-55
DEBUG:bincat.plugin:[EXCEPTION] main: Exception caught in main loop
DEBUG:bincat.plugin:Exceptions.Error("PE file format not implemented yet")
DEBUG:bincat.plugin:Raised at file "utils/log.ml", line 160, characters 4-32
DEBUG:bincat.plugin:Called from file "main.ml", line 70, characters 19-76
DEBUG:bincat.plugin:[STOP] nothing analyzed
DEBUG:bincat.plugin:====== end of logfile ======
DEBUG:bincat.plugin:Parsing analyzer result file
ERROR:bincat.plugin:Could not parse result file
------------------------------------------------------------------

Test 0x66 prefix on some x86 instructions

Test 0x66 prefix on

• jmp
• call
• push byte
• push word
• push reg

Imports information table displays wrong data about typing/tainting rules

Configuration handling in IDA plugin is confusing

• Don't load "last used" when running the analysis from an address which is an entrypoint for another config
• selecting a config X, modifying it and then saving it doesn't save it back to "X"

Bug get_proc_name() SDK 7.0?

Hello, I install bincat on IDA 7.0 but i got an issue :
I'm on Windows
procname = idaapi.get_inf_structure().get_proc_name()
AttributeError: 'idainfo' object has no attribute 'get_proc_name'

Have you got an idea ?

Output results into a sqlite db instead of a .ini file

Fix tainting statement creation for imports from config file

This is parsed but not used. Check ocaml/src/fixpoint/rules.ml

[libc]
#tainting rules for libc
#default calling convention for this lib
# call_conv = fastcall
* = open(@, _)
#read uses a different calling convention
* = read<stdcall>(@, *, @)

GUI Polishing

• memory edition
• better hex dump
• renaming some windows, widget reorganization
• alternative to line background coloring ?
• rainbow tainting (ability to display all sources of taints)
• easily run backward analysis
• clear values/taints/types overrides
• better desktop state/windows handling
• hide IL window by default
• analysis can be long, allow the user to cancel
• Goto next node dropdown is confusing

Can bincat work with IDA Pro 6.8?

I only have IDA 6.8, can bincat work with it? or how can I do to make them work together,tks

Handle value overrides in plugin

Once #38 is fixed, we should allow users to override values in the IDA GUI. Goes with #27

Add support for synchronous operations in the engine

We need a way to (synchronously) ask the user what to do when the analyzer is "lost".

It should be usable not only from the IDA plugin to allow for scripted interactions.

py slove pyqt5 for python 2.7

import error : no module name pyqt5,

Allow overriding taint without overriding value. Ability to taint only 1 byte

Possible syntax:

[override] 0x1234 = stack[0x2000], !0xabcd ; stack[0x2004]; !|ff|

Fix eggloaders for armv7 and armv8 to allow tests to mess with the stack

Fix GUI taint display in IDA from rainbow tainting data

Extend memory value declaration syntax in config file

Make it possible to use quoted strings, if possible with escapes, and that can be concatenated with ||

mem[0x10000] = "my string"
mem[0x20000] = "my string terminated with zero\x00"
mem[0x30000] = "my string terminated with zero too"|00|
mem[0x40000] = "abcd"|646566|"efgh\x00"

Also make it possible to add !all to taint the whole memory zone.

Merge c2newspeak

Merge c2newspeak repository to ease installation.

Allow creating an initial state from a debugger

It would be convenient for complex code to be able to create the initial state by dumping data from a stopped debugger: gdb, IDA, Windbg ?

We could also use the debugger dynamically (for example through GDB remote) as a source of concrete data.

C++ domain

Improve performance

• .ini generation is very slow
• Identify bottlenecks and fix them :)
• Python's ConfigParser is slow as hell

#45 should help.

Honor initialization order in configuration file

Currently initial memory content is defined in config.ml as a Hashtbl:

let memory_content: ctbl = Hashtbl.create 10

which makes the following initialization not deterministic:

mem[0x1000*8192]=|00|
mem[0x2000]=0x12345678