Giter Site home page Giter Site logo

archr's Introduction

archr

Code style: black

Traditionally, binary analysis has been implicitly program-centric, meaning that the atomic unit of concern is the binary being analyzed. This assumption is usually implicit: angr.Project is instantiated with the binary in question, afl launches the binary itself, generally hyper-modified to make it easier to fuzz, and so on.

However, outside of the CGC, programs do not exist in a vacuum. Specific library versions, values in configuration files, environment variables, and a myriad other factors combine with the program binary itself to make a unique holistic target, and in many cases, it is that target that needs to be analyzed, not just the program itself. This is specifically true for analysis that need extreme accuracy, such as automatic exploit generation.

archr is an implementation of such a target-centric analysis paradigm. It consists of two main concepts: Targets, which describe the specification of the target itself, how it is configured, how it will be launched, and how it would be interacted with, and Analyzers, which specialize targets for specific analysis actions, such as tracing, symbolic execution, and so on. To accomplish their tasks, Analyzers might inject Implants (i.e., qemu-user, gdbserver, and so on) into the target.

We have the following Targets:

  • DockerImageTarget, which takes a description of the target in the form of a docker image
  • LocalTarget, which just describes running the target in the local system

The following Analyzers exist:

  • DataScoutAnalyzer (will grabs the memory map, environment, and auxv of the process, exactly as it is at launch)
  • AngrProjectAnalyzer (can create an angr project with the right libs at the right offsets)
  • AngrStateAnalyzer (can create a angr states with the right env, args, and fs)
  • QEMUTraceAnalyzer (does qemu tracing of the target)
  • GDBServerAnalyzer (launches the target in a gdbserver)
  • STraceAnalyzer (straces a target)
  • CoreAnalyzer (launches the target and retrieves a core)
  • InputFDAnalyzer (determines the FD number for user input (in some cases))

Using archr

To use archr, one must first create a Target. First, build a docker image that launches your target. Here is an example dockerfile for a docker-cat image:

from ubuntu:latest
entrypoint ["/bin/cat"]

Then, load it as a target:

import archr
t = archr.targets.DockerImageTarget('docker-cat').build()

And viola!, your target is ready to use. archr will automatically figure out how your binary runs inside your target, and then you can launch and interact with it:

t.start()
assert t.run_command(stdin=subprocess.DEVNULL).wait() == 0
t.stop()

archr makes heavy use of with contexts, which will help clean up resources. Embrace them. For example, you can:

with t.start():
	with t.run_context() as p:
		print(p,"is a subprocess.Popen object!")
		p.stdin.write("hello")
		assert p.stdout.read(5) == "hello"

There is even a context that will allow you to temporarily replace files on the target!

with t.start():
	with t.replacemenet_context("/etc/passwd", "hahaha"), t.run_context(args_suffix=["/etc/passwd"]) as p:
		assert p.stdout.read() == "hahaha"
	assert t.run_command(args_suffix=["/etc/passwd"]).stdout.read() != "hahaha"

And even one that will temporarily replace the target binary's code with shellcode:

with t.start():
	with t.shellcode_context(asm_code="mov rax, 60; mov rdi, 42; syscall") as p:
		assert p.wait() == 42

You can retrieve files from the target with retrieve_contents, retrieve_paths, and retrieve_glob, inject files with inject_path, inject_contents, and so on, get network endpoints using ipv4_address, udp_ports, and tcp_ports, and some other interesting stuff! You can also make a LocalTarget to just run stuff on your host, and it is almost perfectly interchange with a DockerTarget:

import archr
t = archr.targets.LocalTarget(["/bin/cat"]).build()

To figure out how to run the binary, LocalTarget takes at least an argv list. You can also pass in an env.

Keep in mind that since some of the above examples need write access to files, you will need to use writable files instead of /etc/passwd and /bin/cat.

Caveats

Some caveats at the moment:

  • archr does not handle string-specified (as opposed to array-specified) entrypoint directives in the docker file. This isn't hard; we just haven't gotten around to it (see issue #1).

archr's People

Contributors

adamdoupe avatar angr-bot avatar astewart-bah avatar balbassam avatar bannsec avatar cl4sm avatar conand avatar connornelson avatar degrigis avatar dnivra avatar etrickel avatar github-actions[bot] avatar haehyun avatar jkrshnmenon avatar kyle-kyle avatar ltfish avatar lukas-dresel avatar m1ghtym0 avatar mborgerson avatar phat3 avatar pre-commit-ci[bot] avatar rhelmot avatar robwaz avatar sei-eschwartz avatar subwire avatar trentn avatar twizmwazin avatar zardus avatar zwimer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

archr's Issues

Several tests are disabled

Description

test_angr_ultimate_tracer.py:    @unittest.skip("for now")  # Unless(archr._angr_available, "angr required")
test_dockertarget_fs.py:@unittest.skip("broken")
test_sync.py:    @unittest.skip("broken")
test_sync.py:    @unittest.skip("broken")
test_sync.py:    @unittest.skip("broken")
test_analyzer_rr.py:    @unittest.skip("broken docker image")
test_analyzer_ltrace.py:    @unittest.skip("broken")
test_analyzer_ltrace.py:    @unittest.skip("broken")
test_analyzer_ltrace.py:    @unittest.skip("broken")
test_analyzer_ltrace.py:    @unittest.skip("broken")
test_analyzer_gdbserver.py:    @unittest.skip("broken")
test_analyzer_gdbserver.py:    @unittest.skip("broken")
test_analyzer_gdbserver.py:    @unittest.skip("broken")

Steps to reproduce the bug

N/A

Environment

As of f360585

Additional context

No response

QEMUTracerAnalyzer fails to set taint_fd when emulating CGC binaries

QEMUTracerAnalyzer complains that it was unable to find the last read marker and so cannot set taint_fd when emulating CGC binaries(https://github.com/angr/archr/blob/master/archr/analyzers/qemu_tracer.py#L176). Kyle said this check is part of a bigger feature that was not implemented in and not required for CGC qemu. Perhaps the check should be modified to be triggered only for targets where it is relevant. It is fairly straightforward to reproduce this: emulate any CGC binary with bytes from a PoV as input. I can provide a script to reproduce if needed.

DataScoutAnalzyer: Recent Linux Kernels do not Support Sendfile from /proc/<pid> Files

from  archr.targets import LocalTarget
from archr.analyzers import DataScoutAnalyzer

tgt = LocalTarget(["./test_bin"])

with tgt.build().start() as t:
    print(DataScoutAnalyzer(t).fire())

On recent linux kernels (5.11.0-27-generic) this code will print '([], [], b'', {})', failing to recover the env, argv, auxv, and map. This failure is due to sendfile not supporting an in_fd that is from /proc/. This change is also discussed here Gallopsled/pwntools#1871.

A smaller bug also exists in the datascout analyzer on line 55. "mov rdi, 1; mov rsi, rax; mov rdx, 0; mov r10, 0x1000000; mov rax, 40; syscall;" Given the bug already mentioned in keystone-engine that defaults to a radix of 16, this syscall is actually to syscall 64 which is semget rather than the intended sendfile call.

My branch: https://github.com/2over12/archr/tree/fix_sendfile_bug fixes the amd64 shellcode by using the stack as a buffer to do a read/write loop of the file to stdout. I am not sure if this is the best approach to fixing this problem. The other solution I could think of was just inserting breakpoint shellcode then finding the pid of the target and pulling the file with say the retrieve_contents call to the target.

Stop hard-depending on shellphish-qemu

There are many cases where shellphish-qemu isn't necessary for basic use, so this dependency should be made optional. Some refactoring will be required to make this happen.

Target analysis lifetime should not be attached to python scope

Target analysis (bows) lifetime (flight) should not be attached to python scope (context manager).

Using a bow looks something like this:

target.build().start()
strace_bow = archr.arsenal.STraceBow(target)
with strace_bow.fire_context() as flight:
    # do stuff with flight
# the flight will die

If I want to keep the flight around, I can be hacky with the context manager like this:

target.build().start()
strace_bow = archr.arsenal.STraceBow(target)
context = strace_bow.fire_context()
flight = context.__enter__()
# do stuff with flight

However, even still, when context goes out of scope, the context manager's backing generator will enter the finally scope and kill the flight. This means that in order to keep the flight alive, I need to maintain a reference to context.

It would be really nice if I could just do something like:

target.build().start()
strace_bow = archr.arsenal.STraceBow(target)
flight = strace_bow.flight()

QEMUTracerAnalyzer crashes with BrokenPipeError before entire input can be written to stdin of CGC binary

When attempting to generate a trace of a CGC binary using a PoV as input, the QEMUTracer crashes with BrokenPipeError. I debugged the issue a bit and found that the error occurs when the SendAction is being run(second action run by loop at https://github.com/angr/archr/blob/master/archr/targets/flight.py#L97). The size of the input PoV is about 1.7MB and I think the entire input is not transmitted in one send but is broken up into multiple chunks. However, the channel seems to have been closed by the time second chunk is written. This ZIP file has the binary, input and script needed to reproduce this crash.

Support non-array entrypoint specifications in Docker

In docker, there are two ways to redefine an entrypoint:

ENTRYPOINT ["/bin/blah"]

The above will cause /bin/blah to be executed on container start.

ENTRYPOINT /bin/blah

The above will cause /bin/sh -c /bin/blah to be executed on container start.

Currently, archr only supports the former, but the latter is frequently used as well. This needs to be added to the logic in DockerTarget.build.

Missing requirements

Just tried to install archr for the repo, and it's missing the following requirements:

docker
nclib

Temporary directories cleanup seems to not be working in some cases

I find that the temporary directories created by some analyzers are still around even after the entire script has exited. One case I found this is for angrProjectAnalyzer. Is it because angrProjectAnalyzer does not implement context management interface functions(eg: __enter__, __exit__ etc)?

`test_analyzer_angr.test_angr_fauxware_custom_plt_hooks` is failing in CI

Description

Traceback (most recent call last):
  File "/__w/archr/archr/build/src/archr/tests/test_analyzer_angr.py", line 140, in test_angr_fauxware_custom_plt_hooks
    self.assertEqual(num_authed, 2)
AssertionError: 0 != 2

Steps to reproduce the bug

Run nightly CI

Environment

CI

Additional context

No response

How to run archr on Ubuntu 16.04?

/usr/local/lib/python3.5/dist-packages/archr-8.18.10.5-py3.5.egg/archr/arrows/shellphish_qemu/bundle /tmp/tmp35qmlkh2 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/shellphish_qemu/__init__.py", line 19, in qemu_base
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1143, in resource_filename
    self, resource_name
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1650, in get_resource_filename
    return self._extract_resource(manager, zip_path)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1671, in _extract_resource
    timestamp, size = self._get_date_and_size(self.zipinfo[zip_path])
KeyError: 'shellphish_qemu/bin'
ldd: /bin: not regular file
ldd: /boot: not regular file
ldd: /cdrom: not regular file
ldd: /dev: not regular file
ldd: /etc: not regular file
ldd: /home: not regular file
ldd: /lib: not regular file
ldd: /lib32: not regular file
ldd: /lib64: not regular file
ldd: /libx32: not regular file
ldd: /lost+found: not regular file
ldd: /media: not regular file
ldd: /mnt: not regular file
ldd: /opt: not regular file
ldd: /proc: not regular file
ldd: /root: not regular file
ldd: /run: not regular file
ldd: /sbin: not regular file
ldd: /snap: not regular file
ldd: /srv: not regular file
ldd: /sys: not regular file
ldd: /tmp: not regular file
ldd: /usr: not regular file
ldd: /var: not regular file
ldd: /shellphish-qemu-cgc-base: No such file or directory
cp: omitting directory '/bin'
cp: omitting directory '/boot'
cp: omitting directory '/cdrom'
cp: omitting directory '/dev'
cp: omitting directory '/etc'
cp: omitting directory '/home'
cp: omitting directory '/lib'
cp: omitting directory '/lib32'
cp: omitting directory '/lib64'
cp: omitting directory '/libx32'
cp: omitting directory '/lost+found'
cp: omitting directory '/media'
cp: omitting directory '/mnt'
cp: omitting directory '/opt'
cp: omitting directory '/proc'
cp: omitting directory '/root'
cp: omitting directory '/run'
cp: omitting directory '/sbin'
cp: omitting directory '/snap'
cp: omitting directory '/srv'
cp: omitting directory '/sys'
cp: omitting directory '/tmp'
cp: omitting directory '/usr'
cp: omitting directory '/var'

When i use archr.arsenal.QEMUTracerBow this error happened, so i tracked the code and find the reason.

So, how to sovle this?

docker mount failed when creating angrProjectBow

Hi,

When I tried to run "python test/test_bow_angr.py", I got the following error:

mount: /tmp/archr_mounts/ad53a3e1edbb4216b30219b3673052f70833dc42fd2576dda50f2f10d64982c0: special device /home/johnson/dockers/overlay2/7953906f2439673aefed891c9c8d16681e6de7a9a512a8701618cfb0a7c330ea/merged does not exist.

umount: /tmp/archr_mounts/ad53a3e1edbb4216b30219b3673052f70833dc42fd2576dda50f2f10d64982c0: not mounted.
Traceback (most recent call last):
File "test_bow_angr.py", line 42, in
test_env_angr()
File "test_bow_angr.py", line 31, in test_env_angr
angr_checks(t)
File "test_bow_angr.py", line 14, in angr_checks
project = apb.fire()
File "/home/johnson/virtualenvs/angr-aeg/lib/python3.5/site-packages/archr-8.18.10.5-py3.5.egg/archr/arsenal/angr_project.py", line 30, in fire
self.project = angr.Project(the_binary, preload_libs=the_libs, lib_opts=lib_opts, main_opts=bin_opts, **kwargs)
File "/home/johnson/opensource/security/angr-dev/angr/angr/project.py", line 115, in init
raise Exception("Not a valid binary file: %s" % repr(thing))
Exception: Not a valid binary file: '/tmp/archr_mounts/ad53a3e1edbb4216b30219b3673052f70833dc42fd2576dda50f2f10d64982c0/usr/bin/env'

By debugging the program line by line, I found that this problem is caused by the following line in docker_target.py/mount_local function

os.system(_super_mount_cmd + "mount -o bind %s %s" % (self._merged_path, self.local_path))

My os is ubuntu 16.04 and docker version is 18.03.0-ce.

flaky test_cat_stderr CI test

Description

Somehow test_cat_stderr (test_dockertarget_simple.TestDockerTargetSimple) is flaky and it can block archr CI sometimes.
It might be an issue in the test itself or something wrong with the docker target.
I'll take a look later.

Steps to reproduce the bug

No response

Environment

No response

Additional context

No response

DockerImageTarget with pull=True should fail gracefully

It's often difficult to know if a docker image is a local target or from a registry, and therefore we don't know when to set pull=True.

pull=True should fail gracefully if it's unable to pull, in case it's a local docker image.

coredump load failed in shellphish-qemu-linux-arm with a KeyError

Description

I use rex to exploit a simple arm bug with dumbtracer
I add the following code in test_rex.py

def test_linux_armel_stacksmash_jump_dumb():
    path = bin_location
    bin_path = os.path.join(path, "vuln_stacksmash_withjump")
    ld_path = os.path.join(path, "ld-linux.so.3")
    libc_path = os.path.join(path,"libc.so.6")
    env = 'LD_PRELOAD='+ld_path
    lib_path = path
    inp = b"A" * 0x100

    input("Let's go to run "+str(bin_path))
    from rex.crash_tracer import TraceMode
    with archr.targets.LocalTarget([ld_path, '--library-path', lib_path, bin_path], bin_path, target_arch='arm').build().start() as target:
        print("Success")
        trace_mode = TraceMode.DUMB
        crash = rex.Crash(target, inp, trace_mode=trace_mode)

with load the coredump it failed

(rex-lastest) lxw@ubuntu2004:~/rex/tests$ python test_rex.py linux_armel_stacksmash_jump_dumb
Let's go to run /home/lxw/rex/tests/../../binaries/vuln_stacksmash_withjump
DEBUG    | 2023-08-20 20:08:33,767 | archr.targets  | target OS not specified, using `linux` by default
Success
DEBUG    | 2023-08-20 20:08:34,571 | archr.targets  | Running command: 'mkdir' '/tmp/tracer_target_l9qsjyud'
DEBUG    | 2023-08-20 20:08:34,576 | archr.targets  | Running command: 'chmod' '777' '/tmp/tracer_target_l9qsjyud'
DEBUG    | 2023-08-20 20:08:34,579 | archr.analyzers.qemu_tracer | launch QEMU with command: /tmp/archr_local_2e6snvhd/shellphish_qemu/fire /tmp/archr_local_2e6snvhd/shellphish_qemu/shellphish-qemu-linux-arm -C /tmp/tracer_target_l9qsjyud -d nochain,exec,page,strace -D /tmp/tracer-daqgestg.trace -- /home/lxw/rex/tests/../../binaries/ld-linux.so.3 --library-path /home/lxw/rex/tests/../../binaries /home/lxw/rex/tests/../../binaries/vuln_stacksmash_withjump
DEBUG    | 2023-08-20 20:08:34,579 | archr.targets  | Running command: '/tmp/archr_local_2e6snvhd/shellphish_qemu/fire' '/tmp/archr_local_2e6snvhd/shellphish_qemu/shellphish-qemu-linux-arm' '-C' '/tmp/tracer_target_l9qsjyud' '-d' 'nochain,exec,page,strace' '-D' '/tmp/tracer-daqgestg.trace' '--' '/home/lxw/rex/tests/../../binaries/ld-linux.so.3' '--library-path' '/home/lxw/rex/tests/../../binaries' '/home/lxw/rex/tests/../../binaries/vuln_stacksmash_withjump'
INFO     | 2023-08-20 20:08:34,581 | archr.analyzers | sleep for 15 seconds waiting for the target to initialize
DEBUG    | 2023-08-20 20:08:49,582 | archr.target.actions | [OpenChannelAction] openning channel: stdio
DEBUG    | 2023-08-20 20:08:49,583 | archr.target.actions | [SendAction] sending data to channel stdio: b'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'
INFO     | 2023-08-20 20:08:49,583 | archr.log      | ======= Sending 256 bytes =======

INFO     | 2023-08-20 20:08:49,583 | archr.log      | >> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%

DEBUG    | 2023-08-20 20:08:49,698 | archr.analyzers.qemu_tracer | Qemu tracer returned with code=-11 timed_out=False crashed=True signal=Signals.SIGSEGV
DEBUG    | 2023-08-20 20:08:49,740 | archr.analyzers.qemu_tracer | Detected the tainted fd to be 0
DEBUG    | 2023-08-20 20:08:49,741 | archr.analyzers.qemu_tracer | Detected the crashing address at 0x41414140
DEBUG    | 2023-08-20 20:08:49,741 | archr.analyzers.qemu_tracer | Trace consists of 19046 basic blocks
DEBUG    | 2023-08-20 20:08:49,741 | archr.targets  | Running command: 'rm' '/tmp/tracer-daqgestg.trace'
DEBUG    | 2023-08-20 20:08:49,745 | archr.targets  | Running command: 'rm' '-rf' '/tmp/tracer_target_l9qsjyud'
Traceback (most recent call last):
  File "test_rex.py", line 487, in <module>
    globals()['test_' + sys.argv[1]]()
  File "test_rex.py", line 463, in test_linux_armel_stacksmash_jump_dumb
    crash = rex.Crash(target, inp, trace_mode=trace_mode)
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/rex-0.2-py3.8.egg/rex/crash.py", line 618, in __init__
    self._initialize()
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/rex-0.2-py3.8.egg/rex/crash.py", line 1001, in _initialize
    self.concrete_trace()
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/rex-0.2-py3.8.egg/rex/crash.py", line 405, in concrete_trace
    self.trace_result, self.core_registers = self.tracer.concrete_trace(testcase, channel,
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/rex-0.2-py3.8.egg/rex/crash_tracer/dumb_tracer.py", line 189, in concrete_trace
    self.crash_addr, times = self._identify_crash_addr(testcase, channel, pre_fire_hook,
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/rex-0.2-py3.8.egg/rex/crash_tracer/dumb_tracer.py", line 171, in _identify_crash_addr
    project = self.angr_project_bow.fire(core_path=r.core_path)
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/archr/analyzers/angr_project.py", line 141, in fire
    self.project = angr.Project(core_path, main_opts=bin_opts, rebase_granularity=0x1000, **project_kwargs)
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/angr/project.py", line 147, in __init__
    self.loader = cle.Loader(self.filename, concrete_target=concrete_target, **load_options)
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle/loader.py", line 188, in __init__
    self.initial_load_objects = self._internal_load(
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle/loader.py", line 782, in _internal_load
    obj = self._load_object_isolated(main_spec)
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle/loader.py", line 986, in _load_object_isolated
    result = backend_cls(binary, binary_stream, is_main_bin=self._main_object is None, loader=self, **options)
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle/backends/elf/elfcore.py", line 66, in __init__
    self.__extract_note_info()
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle/backends/elf/elfcore.py", line 112, in __extract_note_info
    self.__parse_auxv(n_desc)
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle/backends/elf/elfcore.py", line 400, in __parse_auxv
    byte = self.__dummy_clemory[pos]
  File "/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle/memory.py", line 287, in __getitem__
    raise KeyError(k)
KeyError: 15

Steps to reproduce the bug

I upload all my environment and coredump file to google cloud
https://drive.google.com/drive/folders/15ms9I8QG-Pc477LfJWO7kRIBgx1TY2cl?usp=share_link

Environment

python -m angr.misc.bug_report
/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/angr/misc/bug_report.py:1: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
angr environment report
=============================
Date: 2023-08-20 21:27:08.696405
Running in virtual environment at /home/lxw/.virtualenvs/rex-lastest
/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/angr/misc/bug_report.py:88: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources  # pylint:disable=import-outside-toplevel
Platform: linux-x86_64
Python version: 3.8.10 (default, May 26 2023, 14:05:08)
[GCC 9.4.0]
######## angr #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/angr
Pip version angr 9.2.64
Couldn't find git info
######## ailment #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/ailment
Pip version ailment 9.2.64
Couldn't find git info
######## cle #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/cle
Pip version cle 9.2.64
Couldn't find git info
######## pyvex #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/pyvex
Pip version pyvex 9.2.64
Couldn't find git info
######## claripy #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/claripy
Pip version claripy 9.2.64
Couldn't find git info
######## archinfo #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/archinfo
Pip version archinfo 9.2.64
Couldn't find git info
######## z3 #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/z3
Pip version z3-solver 4.10.2.0
Couldn't find git info
######## unicorn #########
Python found it in /home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/unicorn
Pip version unicorn 2.0.1.post1
Couldn't find git info
######### Native Module Info ##########
angr: <CDLL '/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/angr/state_plugins/../lib/angr_native.so', handle 3175af0 at 0x7f3205e2fc10>
unicorn: <CDLL '/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/unicorn/lib/libunicorn.so.2', handle 2bf8710 at 0x7f3208ed3940>
pyvex: <cffi.api._make_ffi_library.<locals>.FFILibrary object at 0x7f3209ae3820>
z3: <CDLL '/home/lxw/.virtualenvs/rex-lastest/lib/python3.8/site-packages/z3/lib/libz3.so', handle 28b8bf0 at 0x7f320b9a7e20>

all the environment rex is the lasted and I installed with git folder

Additional context

No response

Unable to generate trace for certain CGC binaries using PoVs: BrokenPipeError

archr fails to generate the trace for some CGC binaries using their PoVs. The traceback is something like this:

Traceback (most recent call last):
  File "archr-repro.py", line 17, in <module>
    archr_trace_result = archr_tracer_bow.fire(testcase=b''.join(pov.writes))
  File "/home/dnivra/angr-dev/archr/archr/analyzers/__init__.py", line 50, in fire
    r.write(testcase)
  File "/home/dnivra/.virtualenvs/angr-dev/lib/python3.6/site-packages/nclib/netcat.py", line 687, in send
    s = s[self._send(s):]
  File "/home/dnivra/.virtualenvs/angr-dev/lib/python3.6/site-packages/nclib/netcat.py", line 513, in _send
    ret = self.sock.send(data)
  File "/home/dnivra/.virtualenvs/angr-dev/lib/python3.6/site-packages/nclib/simplesock.py", line 221, in send
    return self.child_send.send(data)
  File "/home/dnivra/.virtualenvs/angr-dev/lib/python3.6/site-packages/nclib/simplesock.py", line 160, in send
    return self.file.write(data)
BrokenPipeError: [Errno 32] Broken pipe

The binaries for which I was able to reproduce this error are CROMU_00028(binary, pov), NRFIN_00008(binary, pov), NRFIN_00012(binary, pov), NRFIN_00039(binary, pov) and KPRCA_00034(binary, pov; takes sometime to finish tracing). However, I am able to generate a trace using tracer fine(except for KPRCA_00034) so I am probably not using archr correctly to generate a trace(or it's an actual bug). Here is a small script to reproduce the issue:

import pathlib

import archr
import tracer

cgc_binary_name = "CROMU_00028"
# https://github.com/zardus/cgc-bins/blob/master/all_unpatched/CROMU_00028
binary = f"cgc-bins/all_unpatched/{cgc_binary_name}"
# From https://github.com/lungetech/cgc-challenge-corpus/blob/master/CROMU_00028/pov/POV_00000.xml
pov = tracer.TracerPoV(f"cgc-challenge-corpus/{cgc_binary_name}/pov/POV_00000.xml")
# Uncomment lines below to enable tracing using tracer
# print("Generating trace using tracer...", end="", flush=True)
# tracer_trace_result = tracer.QEMURunner(binary=str(binary), input=b''.join(pov.writes))
# print("done.")
print("Generating trace using archr...", end="", flush=True)
archr_target = archr.targets.LocalTarget([str(binary)], target_os='cgc')
archr_tracer_bow = archr.arsenal.QEMUTracerBow(archr_target)
archr_trace_result = archr_tracer_bow.fire(testcase=b''.join(pov.writes))
print("done.")

Any help/insights into what could be wrong would be helpful!

AttributeError: 'str' object has no attribute 'inject_path'

Python 3.8.1 (default, Jan 22 2020, 06:38:00)
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import rex

/home/finch/.local/lib/python3.8/site-packages/pysmt/walkers/generic.py:43: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working
if len(nodetypes) == 1 and isinstance(nodetypes[0], collections.Iterable):

rex.Crash("vuln_stacksmash", "A"*227).exploit().arsenal["rop_to_system"].script("x.py")

Traceback (most recent call last):
File "", line 1, in
File "/home/finch/.local/lib/python3.8/site-packages/rex-0.2-py3.8.egg/rex/crash.py", line 78, in init
File "/home/finch/.local/lib/python3.8/site-packages/archr-8.20.7.0-py3.8.egg/archr/arsenal/qemu_tracer.py", line 43, in init
super().init(target, **kwargs)
File "/home/finch/.local/lib/python3.8/site-packages/archr-8.20.7.0-py3.8.egg/archr/arsenal/init.py", line 20, in init
self.nock()
File "/home/finch/.local/lib/python3.8/site-packages/archr-8.20.7.0-py3.8.egg/archr/arsenal/init.py", line 28, in nock
self.target.inject_path(b, os.path.join(self.target.tmpwd, self.REQUIRED_ARROW))
AttributeError: 'str' object has no attribute 'inject_path'

fd leaks everywhere. what do we do...

the problem is that we use Popen internally under the hood of run_command. Each time it allocates some fd for the process. if we don't clean up the resource, there will be fd leak every time we call run_command.
This happens every time we do something like output = target.run_command().communicate()...

MacOS Support

Networking for docker containers on MacOS is bad. We either need to perform networking via another container on the same network as the target container, or forward ports to the host.

tracer example goes off track at getenv

Following your tracer test case for archer, it seems to setup correctly and trace part way into the libc. It then errors out:

In [18]: simgr.run()
---------------------------------------------------------------------------
TracerDesyncError                         Traceback (most recent call last)
<ipython-input-18-d7a0b3e79488> in <module>
----> 1 simgr.run()

~/angr-dev/angr/angr/sim_manager.py in run(self, stash, n, until, **kwargs)
    257         for _ in (itertools.count() if n is None else range(0, n)):
    258             if not self.complete() and self._stashes[stash]:
--> 259                 self.step(stash=stash, **kwargs)
    260                 if not (until and until(self)):
    261                     continue

~/angr-dev/angr/angr/misc/hookset.py in __call__(self, *args, **kwargs)
     73             current_hook = self.pending.pop()
     74             try:
---> 75                 result = current_hook(self.func.__self__, *args, **kwargs)
     76             finally:
     77                 self.pending.append(current_hook)

~/angr-dev/angr/angr/exploration_techniques/tracer.py in step(self, simgr, stash, **kwargs)
    223     def step(self, simgr, stash='active', **kwargs):
    224         simgr.drop(stash='missed')
--> 225         return simgr.step(stash=stash, **kwargs)
    226
    227     def step_state(self, simgr, state, **kwargs):

~/angr-dev/angr/angr/misc/hookset.py in __call__(self, *args, **kwargs)
     78             return result
     79         else:
---> 80             return self.func(*args, **kwargs)

~/angr-dev/angr/angr/sim_manager.py in step(self, stash, n, selector_func, step_func, successor_func, until, filter_func, **run_args)
    341
    342             pre_errored = len(self._errored)
--> 343             successors = self.step_state(state, successor_func=successor_func, **run_args)
    344
    345             # handle degenerate stepping cases here. desired behavior:

~/angr-dev/angr/angr/misc/hookset.py in __call__(self, *args, **kwargs)
     73             current_hook = self.pending.pop()
     74             try:
---> 75                 result = current_hook(self.func.__self__, *args, **kwargs)
     76             finally:
     77                 self.pending.append(current_hook)

~/angr-dev/angr/angr/exploration_techniques/tracer.py in step_state(self, simgr, state, **kwargs)
    273             # strict mode
    274             if len(succs) == 1:
--> 275                 self._update_state_tracking(succs[0])
    276             elif len(succs) == 0:
    277                 raise Exception("All states disappeared!")

~/angr-dev/angr/angr/exploration_techniques/tracer.py in _update_state_tracking(self, state)
    438             raise TracerDesyncError("Oops! angr did not follow the trace",
    439                                     deviating_addr=state.addr,
--> 440                                     deviating_trace_idx=idx+1)
    441
    442         if state.globals['sync_idx'] is not None:

TracerDesyncError: Oops! angr did not follow the trace

Looking at the state it went offline at:

 '<IRSB from 0x7ffff7a26778: 1 sat>',
 '<IRSB from 0x7ffff7a26785: 1 sat>',
 '<IRSB from 0x7ffff7a26778: 1 sat>',
 '<IRSB from 0x7ffff7a26785: 1 sat>',
 '<IRSB from 0x7ffff7a26778: 1 sat>',
 '<IRSB from 0x7ffff7a26785: 1 sat>',
 '<IRSB from 0x7ffff7a26778: 1 sat>']

In [23]: p
Out[23]: <Project /home/angr/work/ctf/gctf/2019/rev/flaggy_bird/cat>

In [24]: p.loader.describe_addr(0x7ffff7a26778)
Out[24]: 'getenv+0x98 in libc.so.6 (0x42778)'

Resolving what getenv string it was attempting to look at

In [38]: s2.mem[s2.regs.r13].string.concrete
Out[38]: b'CPATH'

CPATH does not appear to exist in the environment given to angr (nor the base env). Not sure at this point why it's diverging from the QEMU trace.

No binary file created?

Excited to play with this! This is me playing without a deep understanding of archr...could be operator error. Done on Ubuntu 16.04 64.

Really like the containerization of angr and wanted to continue that with archr, so...
docker run -it -v /var/run/docker.sock:/var/run/docker.sock [image id]

Cloned archr in the angr container
python setup.py build python setup.py install pip install pygdbmi sudo apt-get install gdb

From there, I wanted to test out the docker-cat example. I created the docker-cat image using Dockerfile...
from ubuntu:latest entrypoint ["/bin/cat"]

and...
docker build -t docker-cat [directory to find dockerfile]

Wrote the following and ran...
import pygdbmi.gdbcontroller
import archr
import angr
import os

t = archr.targets.DockerImageTarget('docker-cat').build().start()

dsb = archr.arsenal.DataScoutBow(t) apb = archr.arsenal.angrProjectBow(t, dsb) asb = archr.arsenal.angrStateBow(t, apb) project = apb.fire(use_sim_procedures=False) state = asb.fire(add_options={angr.sim_options.STRICT_PAGE_ACCESS}) # for now simgr = project.factory.simulation_manager(state) simgr.run()

Got the resulting error...
Exception: Not a valid binary file: '/tmp/archr_mounts/[docker-cat image]/bin/cat'

When I checked out the directory, no cat binary existed in the /tmp file.

Datascout bow fails to get argv when the target binary is in another directory

I get this error archr.errors.ArchrError: DataScout failed to get argv from the target process. stdout: b'' stderr: b'setarch: ././test_binaries/vulnerable: No such file or directory\n' in an angr project bow

The following short script reproduces the issue

#!/usr/bin/env python3
import archr

source = './test_binaries/vulnerable'
with archr.targets.LocalTarget([source]).build().start() as t:
    dsb = archr.arsenal.DataScoutBow(t)
    project = archr.arsenal.angrProjectBow(t, dsb).fire()

This issue does not appear when I copy the file to the same directory and use './vulnerable' instead as source.

Target injection does a recursive chown on /

When a bow injects some artifact into the target (e.g. strace), and the target's user is not root, it ends up doing a recursive chown on /.

This seems unnecessarily aggressive. It should probably recursively chown /tmp.

License

At the moment it's unclear how the source licensed. Could you please add a license statement? Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.