endgameinc/xori

Xori is an automation-ready disassembly and static analysis library for PE32, 32+ and shellcode

Xori - Custom disassembly framework

Xori is an automation-ready disassembly and static analysis library that consumes shellcode or PE binaries and provides triage analysis data.

Acknowledgements:

Xori wouldn't exist without inspiration and ideas from the open source community. We are indebted to the work of the Capstone engine and the LLVM Project.

Architectures:

• i386
• x86-64

File Formats

• PE, PE+
• Plain shellcode

Current Features

• Outputs json of the 1) Disassembly, 2) Functions, and 3) Imports.
• Manages Image and Stack memory.
• 2 modes:
  • Light Emulation - meant to enumerate all paths (Registers, Stack, Some Instructions).
  • Full Emulation - only follows the code’s path (Slow performance).
• Simulated TEB & PEB structures.
• Evaluates functions based on DLL exports.
• Displays strings based on referenced memory locations.
• Uses FLIRT style signatures (Fast Library Identification and Recognition Technology).
• Allows you to use your own exports for simulating the PEB.
• Will detect padding after a non-returning call.
• Will try to identify function references from offsets.

What it doesn't do yet:

• The engine is interactive.
• Does not dump strings.
• Does not process non-executable sections.
• TEB and PEB are not enabled for non-pe files.
• Only some x86 instructions are emulated, not all.
• Patching and assembling.
• No plugins or scripting.

Documentation

Requirements

rustc 1.27.0

Install rust for OSX & Linux Distros

curl https://sh.rustup.rs -sSf | sh

Install rust for Windows

First get the rustup.exe (the rust toolchain installer) from here.

This tool will install the rust compiler rustc, the rust package manager cargo and other usefull tools for development in rust.

• run the rustup.exe

Install rust prerequisites

in case you see this output, your Windows environment is missing the Build Tools for Visual Studio so keep reading, otherwise go here

• follow the link from the output, or click here

• cancel the rustup-init.exe

• back in browser, scroll down, expand the tab Tools for Visual Studio 2017 & download the Build Tools for Visual Studio 2017

• run the executable

choose the Visual C++ build tools & click "install", close the "Visual Studio Installer" after the installation

Install rust toolchain

run the rustup.exe & you will see the following output

after the successful installation you can see that the rust compiler rustc, rust package manager cargo and other tools were installed (under C:\Users\%username%\.cargo & C:\Users\%username%\.rustup)

• open a new "Command Prompt" & follow the xori build steps here

Installation

1. Build Xori

This command will also create other binaries such as pesymbols ans peinfo.

git clone https://github.com/endgameinc/xori.git
cd xori
cargo build --release

2. Create xori.json config file

cp xori.json.example xori.json
[edit if desired]

3. (Optional) Build the symbols files

If you want to create your own symbol files you need to set the dll folders to where you stored your windows dlls.

"function_symbol32": "./src/analysis/symbols/generated_user_syswow64.json",
"function_symbol64": "./src/analysis/symbols/generated_user_system32.json",
"symbol_server": {
	"dll_folder32": "./dlls/32bit",
	"dll_folder64": "./dlls/64bit"

Run pesymbols to overwrite the function_symbol json

 ./target/release/pesymbols

Run

./target/release/xori -f test.exe

Run all tests

cargo test

Browser GUI

Chrome	Firefox	Safari	IE	Opera
Latest ✔	Latest ✔	Latest ✔	x	Latest ✔

Requirements

nodejs

yarn (optional for UI dev)

• On Ubuntu 18.04 you may need to apt install the following: curl git libssl-dev pkg-config build-essential npm

Build

cd gui
npm install

Run

In one terminal

cd gui
node src/server.js

In another terminal

cd gui
npm start

It will open your default browser to http://localhost:3000/. The backend API is listening on localhost:5000.

xori's Issues

Info: Dependency list (Ubuntu 18.04 WKS default install)

curl git libssl-dev pkg-config build-essential npm

No repository description

No description, website, or topics provided.

The first phrase from README should go as a descritpion:

Xori is an automation-ready disassembly and static analysis library that consumes shellcode or PE binaries and provides triage analysis data.

issue with Browser GUI

I get an error when I try to parse a file :

I use Chrome on Windows 10.

Allow cross origin request from anything

For production use, it’s best not to allow all origins. Instead, create a white list of allowed domains, and check each request against the white list.

Fix remaining warnings in xori build

#39 fixes the majority of the build warnings in xori. There are a couple outstanding warnings that need further investigation. I see these warnings when building on macos 10.15.3, ubuntu 19.10, and windows server 2019.

   Compiling xori v0.0.1 (/Users/hulk/src/endgameinc/xori)
warning: use of deprecated item 'std::mem::uninitialized': use `mem::MaybeUninit` instead
   --> src/analysis/formats/pe.rs:523:3
    |
523 | /   do_parse!(input,
524 | |         major_linker_version: le_u8 >>
525 | |         minor_linker_version: le_u8 >>
526 | |         size_of_code: le_u32 >>
...   |
584 | |         }))
585 | |   )
    | |___^
    |
    = note: `#[warn(deprecated)]` on by default
    = note: this warning originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

warning: use of deprecated item 'std::mem::uninitialized': use `mem::MaybeUninit` instead
   --> src/analysis/formats/pe.rs:589:3
    |
589 | /   do_parse!(input,
590 | |         major_linker_version: le_u8 >>
591 | |         minor_linker_version: le_u8 >>
592 | |         size_of_code: le_u32 >>
...   |
653 | |             )
654 | |   )
    | |___^
    |
    = note: this warning originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

m

errors installing on ubuntu 16.04

Issue with install

failed to run custom build command for openssl v0.9.24 process didn't exit successfully:/root/xori/target/debug/build/openssl-3a8db21765a1c7e0/build-script-build(exit code: 101) --- stderr thread 'main' panicked at 'Unable to detect OpenSSL version', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-0.9.24/build.rs:16:14 note: Run withRUST_BACKTRACE=1for a backtrace.

WASM ?

Rather than having a react app why don't you just have a WASM app!!!

What is the purpose of emulation_enabled?

when I enable emulation_enabled in xori.json and give a sample for analysis. But it keeps on running no results will produce. Is there anything I missed? And what about the time to get the output after enabling emulation?

please give me a clickable link

just do this https://pages.github.com/
trying to read this is super annoying right now

capstone-rs backend?

I noticed yesterday that much of Xori's instruction decoding maps pretty directly to chunks of Capstone's decoding logic. Since that appears to be the case for so much of xori's decoder, does it make sense to use capstone-rs directly?

I noticed in your slides that you mention fixing a few capstone bugs in the process of rewriting, so maybe those fixes could be upstreamed? If the motivator for decoding in rust is avoiding memory bugs another thought might be to see about parcelling out Xori's x86 decoder and seeing about replacing the capstone decoder upstream?

Barring any of that, it would be good to know that Xori's x86 decoding very closely derives from Capstone's implementation, much like Capstone does in indicating its LLVM origins, especially if bugs were found in Xori's decoder to know that they should be upstreamed - even the disassembly mapping tables are the same as Capstone's and those do change from time to time!

(a nice part of capstone-rs as a backend means the bar for supporting any other Capstone-backed arch is a lot easier!)

slice exception for small sample

Using the small 7-byte sample with hex dump
00000000: eb 01 0f 31 c0 40 c3 ...1.@.

Running the command line analysis
xori -f [file]

results in following exception:

error: folder does not exist, using current_dir
error: config file does not exist, using default configurations.
IMAGE START: 1000
CODE START: 1000
ENTRYPOINT: 1000
ARCH: ArchX86
MODE: Mode32
thread 'main' panicked at 'index 11 out of range for slice of length 7', libcore/slice/mod.rs:1965:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: std::sys_common::backtrace::print
at libstd/sys_common/backtrace.rs:71
at libstd/sys_common/backtrace.rs:59
2: std::panicking::default_hook::{{closure}}
at libstd/panicking.rs:211
3: std::panicking::default_hook
at libstd/panicking.rs:227
4: std::panicking::rust_panic_with_hook
at libstd/panicking.rs:511
5: std::panicking::continue_panic_fmt
at libstd/panicking.rs:426
6: rust_begin_unwind
at libstd/panicking.rs:337
7: core::panicking::panic_fmt
at libcore/panicking.rs:92
8: core::slice::slice_index_len_fail
at libcore/slice/mod.rs:1965
9: xori::analysis::data_analyzer::check_if_padding
at /checkout/src/libcore/slice/mod.rs:2130
at /checkout/src/libcore/slice/mod.rs:1947
at src/analysis/data_analyzer.rs:33
10: xori::arch::x86::analyzex86::analyze_instructionx86
at src/arch/x86/analyzex86.rs:1247
11: xori::arch::x86::analyzex86::recurse_disasmx86
at src/arch/x86/analyzex86.rs:1387
12: xori::analysis::analyze::disassemble_init
at src/analysis/analyze.rs:421
13: xori::analysis::analyze::analyze
at src/analysis/analyze.rs:523
14: xori::main
at src/main.rs:131
15: std::rt::lang_start::{{closure}}
at /checkout/src/libstd/rt.rs:74
16: std::panicking::try::do_call
at libstd/rt.rs:59
at libstd/panicking.rs:310
17: __rust_maybe_catch_panic
at libpanic_unwind/lib.rs:105
18: std::rt::lang_start_internal
at libstd/panicking.rs:289
at libstd/panic.rs:392
at libstd/rt.rs:58
19: main
20: __libc_start_main
21: _start

---

Note: ubuntu 16.04
rustc 1.28.0
cargo 1.28.0

Suggestion: Provide pre-built binaries

It would be nice to provide pre-built binaries and make them available using releases, so users didn't have to install Rust and build it themselves.

Just a friendly suggestion :)

xori currently will not build on modern Linux distros

On modern Linux distros (Ubuntu 18.04, 19.10) that ship with openssl 1.1.x, cargo build produces this error:

error: failed to run custom build command for `openssl v0.9.24`

Caused by:
  process didn't exit successfully: `/home/hulk/rust/xori/target/release/build/openssl-62f9983eb3104bc7/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Unable to detect OpenSSL version', /home/hulk/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-0.9.24/build.rs:16:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

warning: build failed, waiting for other jobs to finish...
error: build failed

I'm pretty sure this is related to the version of the reqwestcrate we're currently using.

$ grep reqwest Cargo.toml
reqwest = "0.8.5"

Consider providing CFG as a part of output

As of now, CFG is displayed in GUI.
It would be of great help if that graph is outputted along with other output files. A lot of papers talk about malware detection using CFG analysis, so providing CFG (which you are generating anyway) is much helpful.
Along with disasm, functions and import files, you can output cfg file which has cfg in form of adjacency matrix or adjacency list.

Xori install error on Windows 10

I've attempted to install Xori on two versions of Windows 10; both the latest and greatest v 1803 as well as the previous version (1709 I believe). Per the requirements:
-Installed Rust from https://static.rust-lang.org/dist/rust-1.28.0-x86_64-pc-windows-gnu.msi
-Downloaded the zip from https://github.com/endgameinc/xori
-Tried installing in CMD window, both with and without Admin privileges using "cargo build --release".
-Also tried installing with the same command in a PowerShell window.

In all cases, I got the attached error message.

Can you help me troubleshoot this?

Consider integration with radeco-lib

Since your program is embeddable and written in Rust, your project (and radeco of course, haha) might benefit from reusing the crate:

https://github.com/radareorg/radeco and https://github.com/radareorg/radeco-lib

It is not yet finished, but should be ready for its first release in about a month.