cea-sec/sibyl

A Miasm2 based function divination.

This file is part of Sibyl.

Copyright 2014 - 2019 Camille MOUGEY

Sibyl

A Miasm2 based function divination.

Idea

In reverse engineer work, stripped binaries are common (malwares, firmwares, ...). Often, they carry usual libraries, such as libc or openssl. Identifying such libraries and their functions can be an interesting starting point. But it is a time consuming task. Moreover, this task is made more difficult due to optimizations, architectures and compilers diversity, custom implementations, obfuscation, ...

Tools have been developed to automate this task. Some are based on CFG (Control Flow Graph) signature (Bindiff), others on magic constants (FindCrypt) or enhanced pattern matching (FLIRT).

Sibyl is one of these tools, dynamic analysis oriented and based on Miasm2 (https://github.com/cea-sec/miasm). The idea is to identify functions from their side effects. That way, identification is independent of the used implementation.

Identifications are done thanks to these steps:

1. Initialize a minimalist VM for the targeted architecture, with only needed elements
2. Prepare the function call using the correct ABI and API
3. Run the target function code inside the VM
4. If the function crashes (null derefencement, not enough stack arguments, ...), switch to the next test case 4b. If the function ends correctly, compare the final VM state with the expected one. If they match, consider the test case as a candidate

For instance, if one want to identify a strlen, the test will be as follow:

1. Allocate a string containing "Hello %sworld!" in a read-only memory page
2. Call the function with a pointer on the string as first argument
3. Compare the result with 14
4. Execute the same test with a different string to avoid false positives (detecting a function which always returns 14)

Sibyl test cases are written architecture and ABI independant.

Basically, Sibyl suffers from false positives (identifying a non strlen as a strlen one) and false negatives (misidentifying or skipping a real strlen). Given the hypothesis that the ABI is exactly the one used by the function, Sibyl becomes complete (no more false negatives).

As a sideline, Sibyl can be used to bruteforce a program ABI.

Long story short, this is an enhanced API bruteforcing tool.

Basic usage

Sibyl comes with a CLI, named sibyl, an IDA (https://www.hex-rays.com) stub and a GHIDRA (https://ghidra-sre.org/) stub.

CLI

The sibyl tool is a wrapper on several sub-actions.

$ sibyl
Usage: /usr/local/bin/sibyl [action]

Actions:
	config   Configuration management
	find     Function guesser
	func     Function discovering
	learn    Learn a new function

The main usage of Sibyl, function recognition, is done through the find action. This action comes with several options, to specify ABI, architecture, test cases, ...

To launch function recognition on the ARMv6 binary busybox-amv6l(busybox 1.21.1 http://www.busybox.net/downloads/binaries/1.21.1/), targetting address 0x8230 and 0x8550 and using included test cases:

$ sibyl find binaries/busybox-armv6l 0x00008550 0x00008230
0x00008230 : strlen
0x00008550 : memmove

IDA stub

The IDA stub is located in ext/ida/find.py. If sibyl is installed on the system, no other action is needed to have it running (see section Installation for more details)

Once the script has been loaded by IDA, the user is asked to launch Sibyl either on the current function, or on all function detected by IDA.

The architecture and ABI are provided by IDA. Optionnaly, the set of test to use can be modified.

On busybox-i486:

And the associated result:

Python>
Launch identification on 3085 function(s)
Found memcpy at 0x8057120
Found memmove at 0x805714c
Found memset at 0x8057174
Found strcat at 0x80571a8
Found strchr at 0x80571cc
Found strcmp at 0x8057208
Found strcpy at 0x8057228
Found strlen at 0x8057244
Found strncmp at 0x8057258
Found strncpy at 0x8057280
Found strnlen at 0x80572a8
Found strrchr at 0x80572c0
Found memcmp at 0x80572ff
Found strsep at 0x80576ac
Found strspn at 0x8057704
Found stricmp at 0x805799c
Found strpbrk at 0x8057ab8
Found strtok at 0x8057b30
Found strcmp at 0x8057b48
Found atoi at 0x805df1c
Current: 64.83% (sub_0x80b4ab3)| Estimated time remaining: 14.45s
Found atoi at 0x80f1cf3
Current: 100.00% (sub_0x80f7a93)| Estimated time remaining: 0.00s
Finished ! Found 21 candidates in 42.70s
Results are also available in 'sibyl_res'

The corresponding function get an additionnal comment like [Sibyl] memmove?

Additionnaly, a method launch_on_funcs is provided for scripting purposes, and the result of the last run, in addition to the human output on console, is available in sibyl_res variable.

Binary Ninja stub

An external stub for Binary Ninja is available here, maintained by @kenoph.

GHIDRA stub

The GHIDRA stub is located in ext/ghidra/find.py. If sibyl is installed on the system, no other action is needed to have it running (see section Installation for more details). One just need to copy or link the script to path known by GHIDRA (as ~/ghidra_scripts).

Then, it can be called from GHIDRA inteface, through the "Script Manager", category "FunctionID".

Documentation

A more detailed documentation is available in doc:

• Advanced usage
• Configuration
• Testing
• Learning
• Adding a new signature
• Adding a new ABI

Current version is v0.2. See changelog for more details.

Installation

Standard

Sibyl requires at least Miasm2 version v0.1.1 and the corresponding version of Elfesteem. For the qemu engine, the unicorn python package must be installed (refer to the documentation of Unicorn for more detail).

Sibyl comes as a Python module, and the installation follow the standard procedure:

$ python setup.py build
# Add the resulting build directory in your PYTHONPATH, or:
$ python setup.py install

In addition of the sibyl Python module, a CLI tool is provided, named sibyl. See the usage documentation for more information.

If needed, consult testing documentation to check your Sibyl installation.

IDA & GHIDRA

IDA & GHIDRA stub are respectively located in ext/ida and ext/ghidra. To benefit from multiprocessing, Sibyl is invoke through the CLI as a subprocess. Then, there is no need to have the sibyl module in IDA Python nor GHIDRA namespace.

Long story short, it should work out of the box once sibyl CLI is available.

Docker

Sibyl is also available through Docker automated build. Use:

$ docker run -i -t commial/sibyl
Usage: /usr/local/bin/sibyl [action]

Actions:
	config   Configuration management
	find     Function guesser
	func     Function discovering
	learn    Learn a new function

Support

Test cases

Sibyl comes with several test cases, located in sibyl/test. These tests are based on function from string.h, stdlib.h and ctype.h.

One can add its custom test cases, and reference it through the configuration file. Have a look at Configuration and Adding a new signature for more information.

Architectures by engine

Sibyl comes with the support of multiple architecture, and multiple engine.

arch/jit	python	tcc	gcc	llvm	qemu
arml	✔️	✔️	✔️	✔️	✔️
armb	✔️	✔️	✔️	✔️	✔️
armtl	❌	❌	❌	❌	⚠️ use arml with +1 offset
armtb	❌	❌	❌	❌	⚠️ use armb with +1 offset
sh4	❌	❌	❌	❌	❌
x86_16	✔️	✔️	✔️	✔️	✔️
x86_32	✔️	✔️	✔️	✔️	✔️
x86_64	✔️	⚠️ bad SSE support	⚠️ bad SSE support	✔️	✔️
msp430	✔️	✔️	✔️	✔️	✔️
mips32b	✔️	✔️	✔️	✔️	✔️
mips32l	✔️	✔️	✔️	✔️	✔️
aarch64l	✔️	✔️	✔️	✔️	✔️
aarch64b	✔️	✔️	✔️	✔️	✔️

FAQ

Do not hesitate to consult and open an issue if precisions are still needed.

How infinite loops are managed?

Behaviors close to infinite loop happen quite often, especially when the arguments are not formatted as expected by the function (trying another test case). To avoid these behaviors, there is a timeout on each sub-test. The -i/--timeout argument adjusts this parameter (2 by default, 0 to disable timeout).

How to run the tool on a custom architecture?

Once the architecture and corresponding semantic is implemented in Miasm2, one just needs to implement the wanted ABI in sibyl/abi/. If writing the jitter engine part is an issue, one can directly use the python jitter option with -j/--jitter argument. If the semantic is not complete enough, one can add the corresponding bridge with qemu in sibyl/engine/qemu.py, if available.

Is my sibyl func freezed?

Sibyl may take time due to the number of function to consider and the test set size (Sibyl time complexity is approximately in O(number function * test set size)).

In addition, library are often present in the same binary zone, giving the impression that Sibyl got result by burst.

A convenient way to observe its progress is the use of the -v option.

How many coffees could I take while Sibyl is running?

binary	architecture	test set size	addresses to check	number of function found	elapsed time
busybox-i486	x86_32	26	3085	21	36.0s
busybox-armv6l	arml	26	3063	48	1m16.5s
busybox-mipsel	mips32l	26	3065	16	44.0s

These tests have been done on a standard, 4 i7 CPU laptop, using the default configuration (ie. qemu jitter) and addresses provided by IDA.

Please note that, by design, Sibyl is embarrassingly parallel.

sibyl's Issues

Unable to recognize ABI for void functions in IDA

Hello,

It seems Sibyl (the IDA stub) is unable to recognize the ABI when it finds void functions in IDA.

I believe this issue could be resolved in modifying the gtype_matcher regex as the latter tries to match the keyword before the function arguments.

# int __cdecl(int, int) -> __cdecl
gtype_matcher = re.compile(".+ ([^\(]+)\([^\)]*\)")

Maybe this issue could occur with another keywords. As a reminder, the IDA functions types are defined here: https://www.hex-rays.com/products/ida/support/idadoc/1361.shtml

Code needs to be updated for Python3

I was trying to update this to python 3 , but there are some files that were missing when working with the latest version of Miasm.

Yes, I know to work with Miasm stable 1.1 but even that is python2 and it as everyone knows has been dead for a year now.

I was going through slowly updating it until I ran into this

inside

    from miasm2.expression.modint import mod_size2int
ModuleNotFoundError: No module named 'miasm.expression.modint'

and indeed there is no file called modint in Miasm. Now this despite being an error of not finding something in MIASM is a problem with Sibyl using/referencing old code.

API stubs: do they work with the qemu engine, the same way as in miasm?

Dear Sibyl developers,

I am writing regarding an issue I have with a function I am trying to identify.

This function is from a windows driver and imports ntoskrnl.memset. I have written a stub function to handle it but I could not find a way to use it with the (default) qemu engine.

The problematic block is :

.text:0000000000028ADC                 mov     [rsp+arg_10], rbx
.text:0000000000028AE1                 mov     [rsp+arg_0], rcx
.text:0000000000028AE6                 push    rbp
.text:0000000000028AE7                 push    rsi
.text:0000000000028AE8                 push    rdi
.text:0000000000028AE9                 push    r12
.text:0000000000028AEB                 push    r13
.text:0000000000028AED                 push    r14
.text:0000000000028AEF                 push    r15
.text:0000000000028AF1                 sub     rsp, 100h
.text:0000000000028AF8                 mov     ebx, edx
.text:0000000000028AFA                 xor     edx, edx        ; Val
.text:0000000000028AFC                 mov     rdi, r8
.text:0000000000028AFF                 mov     rsi, rcx
.text:0000000000028B02                 lea     rcx, [rsp+138h+Dst] ; Dst
.text:0000000000028B0A                 lea     r8d, [rdx+40h]  ; Size
.text:0000000000028B0E                 call    memset
.text:0000000000028B13                 mov     eax, 10h
.text:0000000000028B18                 cmp     ebx, eax
.text:0000000000028B1A                 cmovg   ebx, eax
.text:0000000000028B1D                 xor     r9d, r9d
.text:0000000000028B20                 movsxd  rdx, ebx
.text:0000000000028B23                 mov     r8, r9
.text:0000000000028B26                 cmp     rdx, r9
.text:0000000000028B29                 jle     short loc_28B3E

In my sibyl.conf configuration I added a custom_ntoskrnl.py stubs file. It contains a function for memset : a copy/paste from the miasm code for the same function in user-land.

[find]
stubs = /data/miasm/stubs/custom_ntoskrnl.py,/data/miasm/stubs/linux_stdlib.py,/data/ida-scripts/miasm/stubs/win_api_x86_32.py

import logging

from miasm2.jitter.csts import PAGE_READ, PAGE_WRITE, PAGE_EXEC
from miasm2.os_dep.common import heap, windows_to_sbpath
from miasm2.os_dep.common import set_str_unic, set_str_ansi
from miasm2.os_dep.common import get_fmt_args as _get_fmt_args
from miasm2.os_dep.win_api_x86_32_seh import tib_address

def ntoskrnl_memset(jitter):
    ret_ad, args = jitter.func_args_cdecl(['addr', 'c', 'size'])
    jitter.vm.set_mem(args.addr, chr(args.c) * args.size)
    jitter.func_ret_cdecl(ret_ad, args.addr)

With llvm, it works, but not with qemu. From the following extract (generated by using self.jitter.verbose_mode():

>>> Tracing instruction at 0x28adc, instruction size = 5
>>> Tracing instruction at 0x28ae1, instruction size = 5
>>> Tracing instruction at 0x28ae6, instruction size = 1
>>> Tracing instruction at 0x28ae7, instruction size = 1
>>> Tracing instruction at 0x28ae8, instruction size = 1
>>> Tracing instruction at 0x28ae9, instruction size = 2
>>> Tracing instruction at 0x28aeb, instruction size = 2
>>> Tracing instruction at 0x28aed, instruction size = 2
>>> Tracing instruction at 0x28aef, instruction size = 2
>>> Tracing instruction at 0x28af1, instruction size = 7
>>> Tracing instruction at 0x28af8, instruction size = 2
>>> Tracing instruction at 0x28afa, instruction size = 2
>>> Tracing instruction at 0x28afc, instruction size = 3
>>> Tracing instruction at 0x28aff, instruction size = 3
>>> Tracing instruction at 0x28b02, instruction size = 8
>>> Tracing instruction at 0x28b0a, instruction size = 4
>>> Tracing instruction at 0x28b0e, instruction size = 5
>>> Tracing instruction at 0x51262, instruction size = 6     // Entered memset after call here
Invalid memory access at 0x0L                                           // Invalid memory access just after

After trying to solve this, I couldn't succeed.

• Is there a way to have stubs in Sibyl with the qemu engine ?
• Do I have to switch to another engine (llvm, gcc, etc) ?

Thank you very much !

miasm2 link in readme gives 404

Release 0.2

Here is the TODO list for the v0.2:

• Land the new learning module
• Add regression tests for the learning module
• Add configuration associated with PIN (PIN_ROOT + tracer path)
• Add documentation on how-to-learn a function, associated principle and limits
• Support expanduser in config
• Restrict bzero implementation to avoid false positive
• Adds support for function returning a non-allocated pointer
• Support multiple strategy for trace pruning
• Add support for API stubbing
• Mine function address from IDA in sibyl func
• Reflect API changes for Miasm v0.1.1

Don't support UC_MOD_THUMB

Hi, am i right that sibyl can't work with emulation at UC_MOD_THUMB (unicorn thumb mode didn't use) ?

Because in "qemu.py" is using only:
uc_mode = unicorn.UC_MODE_ARM + unicorn.UC_MODE_LITTLE_ENDIAN

ImportError: No module named miasm2.analysis.machine

After installing the latest version of miasm, it seems to break sibyl based off how it tries to import miasm2 when that no longer appears to be a thing (cea-sec/miasm@944806c).

I'm not sure if I'm doing something wrong, perhaps changing all "miasm2" to "miasm" will help?

Release 0.3

Here is the TODO list for the v0.3 (non-definitive):

• Be pip compliant
• Integrate others function mining sources (maybe angr)
• Add ARM/Thumb information from IDA
• Add a debug/explain action to obtain more information on a given recognition attempt
• Add regression tests for configuration (dump, configuration file, info gathering)
• Fix TestIsCharset, which returns to many false positive
• Support Windows (via Unicorn?)
• Add Python3 support (now that Miasm supports it)

z3 translation crashes

Can not dissamble and find function that is here:(

Hi, I faced the question that i can't solve. Yeah, it is a firmware.
First, that i want to start is why sibyl can not even launch test at some addresses? For example, I launch sibyl for one address and after debugging I guessed that launch_tests was not started:

**sibyl find -p -v -t string task 0x103D0                                                                      
Guessed architecture: arml
Found 1 addresses
Found 3 test cases
1 / 1
Address  | Candidates**  

After debugging I figured out that, for addresses, which sibyl can work with, in emulation process(qemu), unicorn returned END_ADDR of emulation ( 0x1337babe),

**> /home/vitalym60/.local/lib/python2.7/site-packages/unicorn/unicorn.py(340)reg_read()->322419390L
-> return reg.value
(Pdb) s
--Return--
> /usr/local/lib/python2.7/dist-packages/sibyl/engine/qemu.py(226)__getattr__()->322419390L
-> return self.mu.reg_read(self.pc_reg_value)
(Pdb) getattr(self.cpu, self.ira.pc.name)
322419390L**

and after it, tests are launched.
But for some functions( example above), unicorn return only started address and emulation stopped and tests for guessing wouldn't be started:

if getattr(self.cpu, self.ira.pc.name) != END_ADDR:
   raise UnexpectedStopException()

Because of that sibyl can't find function that is there, for example:
IDA shows that at ROM:00182412 memcpy AND after decompiling this function in pseudo C code and building in binary - sibyl detect memcpy.
But now:

 **~/Downloads/test  sibyl find -p -v -j python -t string -a arml ware.bin 0x00182412                                    
Found 1 addresses
Found 3 test cases
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
1 / 1
Address  | Candidates  

 ~/Downloads/test  sibyl find -p -v -j qemu -t string -a arml ware.bin 0x00182412                                      
Found 1 addresses
Found 3 test cases
1 / 1
Address  | Candidates**  


 ~/Downloads/test  sibyl find -p -v -j gcc -t string -a arml ware.bin 0x00182412                                   
Found 1 addresses
Found 3 test cases
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
WARNING: cannot disasm at 182412
1 / 1
Address  | Candidates  

Exactly for this binary, Sibyl can't find any functions.

What the hell is going on?)

ERROR

i tried to work with sibyle , i run this command but it generates this error .

python2 find.py /home/younes/Bureau/a.out ABIStdCall_x86_32 0x400410

Guessed architecture: x86_64
Process Process-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "find.py", line 66, in do_test
    possible_funcs = tl.run(address, timeout_seconds=timeout)
  File "/home/younes/Bureau/Sibyl/sibyl/testlauncher.py", line 181, in run
    self.launch_tests(test, address, *args, **kwargs)
  File "/home/younes/Bureau/Sibyl/sibyl/testlauncher.py", line 169, in launch_tests
    status = test.tests.execute(launch_test)
  File "/home/younes/Bureau/Sibyl/sibyl/test/test.py", line 236, in execute
    if not callback(init, check):
  File "/home/younes/Bureau/Sibyl/sibyl/testlauncher.py", line 144, in launch_test
    self.abi.prepare_call(ret_addr=0x1337beef)
  File "/home/younes/Bureau/Sibyl/sibyl/abi/abi.py", line 83, in prepare_call
    self.vm_push(element)
  File "/home/younes/Bureau/Sibyl/sibyl/abi/x86.py", line 33, in vm_push
    self.jitter.push_uint32_t(element)
AttributeError: jitter_x86_64 instance has no attribute 'push_uint32_t'

Update API to Miasm release v0.1.1

As Miasm now uses versioning, it would be fine to have Sibyl following Miasm stable releases, avoiding APIs breaks.

• Update to LocationDB
• Update learn action

ImportError: No module named core.utils

Hi,
Thank you for your work, this looks amazing :-)
Upon installing Sibyl, I encountered some install errors. The first one it that the python cli tool imports miasm2, which appear to have be renamed to "miasm".

I replaced every occurrence to miasm2 accordingly in Sibyl. And then I have a weird bug:

python2 run_all_tests.py   
Traceback (most recent call last):
  File "run_all_tests.py", line 6, in <module>
    from learn import test_learn
  File "/home/vladimir/tools/Sibyl/test/learn/__init__.py", line 1, in <module>
    from .run_tests import test_learn
  File "/home/vladimir/tools/Sibyl/test/learn/run_tests.py", line 11, in <module>
    from sibyl.testlauncher import TestLauncher
  File "/usr/lib/python2.7/site-packages/sibyl/testlauncher.py", line 26, in <module>
    from sibyl.engine import QEMUEngine, MiasmEngine
  File "/usr/lib/python2.7/site-packages/sibyl/engine/__init__.py", line 3, in <module>
    from sibyl.engine.qemu import QEMUEngine
  File "/usr/lib/python2.7/site-packages/sibyl/engine/qemu.py", line 1, in <module>
    from miasm.core.utils import pck32, pck64
ImportError: No module named core.utils

However:

python2                                           :(
Python 2.7.16 (default, Mar 11 2019, 18:59:25) 
[GCC 8.2.1 20181127] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from miasm.core.utils import pck32, pck64
>>> 

So, what am I doing wrong? Because I have no idea what the heck is going on.

Binary Ninja plugin

Hi,

I wrote a simple Binja plugin for using Sibyl.
As of now I'm running it in a single thread but maybe I'll go the way of your plugin and use subprocess.

Anyway, if you can add a link to it in your repo it would be very nice :)

https://github.com/kenoph/binja_sibyl

(The reason why I did not add it as an ext is that binja requires a plugin.json file in the root of the repo, along with __init__.py... I don't want to pollute your repo)

EDIT: Oh yeah, forgot to thank you for the tool. When I read what it does I had an "aha" moment.

only support IDA7.0 above?

Hi,

It seems that the IDA script find.py only support IDA7.0 above ! Only IDA7.0 above has the module ida_kernwin!

PYCParser requirement issue

Calling sibyl find produces an error if pycparser is not installed. Do you think that it could be useful to add a requirements,txt that will install it?

No update?

Name conflicts in stubs

Hi :)

Sibyl by default includes two stubs: win_api_x86_32.py and linux_stdlib.py. Both stubs define different functions with the same name: get_fmt_args. But this breaks some of the stub functions in win_api_x86_32.py that call get_fmt_args, e.g. msvcrt_sprintf with the following error:

TypeError: get_fmt_args() takes exactly 3 arguments (4 given)

To avoid this issue, we have to specify a stub file explicitly.

error while using sibyl

root@ubuntu:/usr/local/bin# sibyl
Usage: /usr/local/bin/sibyl [action]

Actions:
Traceback (most recent call last):
File "/usr/local/bin/sibyl", line 33, in
key=lambda action: action.name)]
File "/usr/local/bin/sibyl", line 33, in
key=lambda action: action.name)]
AttributeError: 'ActionDesc' object has no attribute 'name'

On latest ubuntu 17.04 x86 and latest miasm

Unable to import find.py in IDA

How do you use IDA7.0 under Linux and successfully import find.py, I will get the error of not finding the file.

How sibyl can identify a function ?

I am interested to work with siby , the most interesting thing is that sibyl can detect the name of the function (strsep , strchr , strncmp ....) , how sibyl can do this ? it has a database of templates and it is just comparing ?

Python>identify_all("x86_32", "ABIStdCall_x86_32")
Launch identification on 3056 function(s)
Current: 65.45% (sub_0x80b6d8c)| Estimated time remaining: 49.80s
0x08057258 : strncmp
0x080576ac : strsep
0x08057b48 : strcmp,strncmp
0x080571cc : strchr
0x08057244 : strlen
0x08057704 : strspn
0x08057b30 : strtok

func action failed when using sibyl cli

I use function discovering of sibyl cli but it is error:
./sibyl func ~/Downloads/quarkslab.elf
Traceback (most recent call last):
File "./sibyl", line 46, in
load_action(guessed[0], sys.argv[2:])
File "/usr/local/lib/python2.7/dist-packages/sibyl/actions/init.py", line 33, in load_action
return getattr(mod, actiondesc.classname)(args)
File "/usr/local/lib/python2.7/dist-packages/sibyl/actions/action.py", line 36, in init
self.run()
File "/usr/local/lib/python2.7/dist-packages/sibyl/actions/func.py", line 90, in run
for addr in fh.guess():
File "/usr/local/lib/python2.7/dist-packages/sibyl/heuristics/func.py", line 191, in guess
for address, value in self.votes.iteritems():
File "/usr/local/lib/python2.7/dist-packages/sibyl/heuristics/heuristic.py", line 29, in votes
self.do_votes()
File "/usr/local/lib/python2.7/dist-packages/sibyl/heuristics/func.py", line 178, in do_votes
super(FuncHeuristic, self).do_votes()
File "/usr/local/lib/python2.7/dist-packages/sibyl/heuristics/heuristic.py", line 21, in do_votes
for name, vote in heuristic(self).iteritems():
File "/usr/local/lib/python2.7/dist-packages/sibyl/heuristics/func.py", line 102, in pattern_matching
for find_iter, vaddr_base in _virt_find(data, pattern):
File "/usr/local/lib/python2.7/dist-packages/sibyl/heuristics/func.py", line 72, in _virt_find
for s in virt.parent.ph:
AttributeError: 'ELF' object has no attribute 'parent'

It is elfesteem problem? If it true, can you give me a version of elfesteem to fix it. Thanks you

Using Sibyl under windows: multiprocessing forking errors

from the sibyl installation folder on windows I run:
C:\Python27\python.exe find.py d:\path\to\binary -j python -a arml ABI_ARM 0x1768

This gets me:

File "find.py", line 148, in <module>
    p.start()
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 562, in save_tuple
    save(element)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 600, in save_list
    self._batch_appends(iter(obj))
  File "C:\Python27\lib\pickle.py", line 636, in _batch_appends
    save(tmp[0])
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 396, in save_reduce
    save(cls)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 748, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <type 'thread.lock'>: it's not found as thre
ad.lock

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 380, in main
    prepare(preparation_data)
  File "C:\Python27\lib\multiprocessing\forking.py", line 495, in prepare
    '__parents_main__', file, path_name, etc
  File "D:\Sibyl\find.py", line 148, in <module>
    p.start()
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 258, in __init__
    cmd = get_command_line() + [rhandle]
  File "C:\Python27\lib\multiprocessing\forking.py", line 358, in get_command_l
ine
    is not going to be frozen to produce a Windows executable.''')
RuntimeError: 
            Attempt to start a new process before the current process
            has finished its bootstrapping phase.

            This probably means that you are on Windows and you have
            forgotten to use the proper idiom in the main module:

                if __name__ == '__main__':
                    freeze_support()
                    ...

            The "freeze_support()" line can be omitted if the program
            is not going to be frozen to produce a Windows executable.

Note that there are two errors (?).

Release 0.1

Here is the TODO list for the first release (0.1):

• Move to a Python module
• Use an unique entry point, dispatching for further actions
• Fix find output: JSON, human
• Let IDA use the JSON output
• Move heuristics to separate actions
• Link ABI to corresponding architectures, being optional (if only one is implemented)
• Use a configuration file for retrieving custom tests, default politics, ...
• Merge verbose and quiet option in action find
• Fix regression tests from SIbyl-tests
• Add QEMU support for MIPS
• Summarize result of find at end (in verbose mode)
• Add a doc/ directory with specific documentation