Giter Site home page Giter Site logo

binarly-io / fwhunt-scan Goto Github PK

View Code? Open in Web Editor NEW
193.0 22.0 26.0 762 KB

Tools for analyzing UEFI firmware and checking UEFI modules with FwHunt rules

License: GNU General Public License v3.0

Python 99.64% Makefile 0.14% Dockerfile 0.22%
reverse-engineering uefi efi-protocols efi-guid radare2 uefi-firmware uefi-firmware-analysis

fwhunt-scan's Introduction

License: GPL v3 fwhunt-scan CI fwhunt-scan pypi

fwhunt Logo

FwHunt Community Scanner

Tools for analyzing UEFI firmware and checking UEFI modules with FwHunt rules.

Dependencies

rizin (v0.6.2)

Installation

Install with pip (tested on python3.6 and above):

$ python -m pip install fwhunt-scan

Install manually:

$ git clone https://github.com/binarly-io/fwhunt-scan.git && cd fwhunt-scan
$ python setup.py install

Example

With script

Analyze/scan separate module:

$ python3 fwhunt_scan_analyzer.py analyze-module {image_path} -o out.json
$ python3 fwhunt_scan_analyzer.py scan-module --rule {rule_path} {image_path}

Scan the entire firmware image:

$ python3 fwhunt_scan_analyzer.py scan-firmware -r rules/BRLY-2021-001.yml -r rules/BRLY-2021-004.yml -r rules/RsbStuffingCheck.yml test/fw.bin

With docker

To avoid installing dependencies, you can use the docker image.

You can build a docker image locally as follows:

docker build -t fwhunt_scan .

Or pull the latest image from ghcr.

Example of use:

docker run --rm -it -v {module_path}:/tmp/image:ro \
  fwhunt_scan analyze-module /tmp/image # to analyze EFI module

docker run --rm -it -v {module_path}:/tmp/image:ro -v {rule_path}:/tmp/rule.yml:ro \
  fwhunt_scan scan-module /tmp/image -r /tmp/rule.yml # to scan EFI module with specified FwHunt rule

docker run --rm -it -v {module_path}:/tmp/image:ro -v {rule_path}:/tmp/rule.yml:ro \
  fwhunt_scan scan-firmware /tmp/image -r /tmp/rule.yml # to scan firmware image with specified FwHunt rule

docker run --rm -it -v {module_path}:/tmp/image:ro -v {rules_directory}:/tmp/rules:ro \
  fwhunt_scan scan-firmware /tmp/image --rules_dir /tmp/rules # to scan firmware image with specified rules directory

All these steps are automated in the fwhunt_scan_docker.py script:

python3 fwhunt_scan_docker.py analyze-module {module_path} # to analyze EFI module

python3 fwhunt_scan_docker.py scan-module -r {rule_path} {module_path} # to scan EFI module with specified FwHunt rule

python3 fwhunt_scan_docker.py scan-firmware -r {rule_path} {firmware_path} # to scan firmware image with specified FwHunt rule

python3 fwhunt_scan_docker.py scan-firmware --rules_dir {rules_directory} {firmware_path} # to scan firmware image with specified rules directory

From code

UefiAnalyzer

Basic usage examples:

from fwhunt_scan import UefiAnalyzer

...
uefi_analyzer = UefiAnalyzer(image_path=module_path)
print(uefi_analyzer.get_summary())
uefi_analyzer.close()
from fwhunt_scan import UefiAnalyzer

...
with UefiAnalyzer(image_path=module_path) as uefi_analyzer:
    print(uefi_analyzer.get_summary())

On Linux platforms, you can pass blob for analysis instead of file:

from fwhunt_scan import UefiAnalyzer

...
with UefiAnalyzer(blob=data) as uefi_analyzer:
    print(uefi_analyzer.get_summary())

UefiScanner

from fwhunt_scan import UefiAnalyzer, UefiRule, UefiScanner

...
uefi_analyzer = UefiAnalyzer(module_path)

# rule1 and rule2 - contents of the rules on YAML format
uefi_rules = [UefiRule(rule1), UefiRule(rule2)]

scanner = UefiScanner(uefi_analyzer, uefi_rules)
result = scanner.result

fwhunt-scan's People

Contributors

hughsie avatar matrosov avatar phretor avatar rs38 avatar xorpse avatar yeggor avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fwhunt-scan's Issues

BrokenPipeError

With the new rizin code I'm getting occasionally:

  File "/home/hughsie/Code/lvfs-website/lvfs/queries/utils.py", line 43, in _query_run_shard_uefi_r2
    uefi_analyzer = UefiAnalyzer(blob=shard.blob)
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/uefi_r2/uefi_analyzer.py", line 83, in __init__
    self._rz.cmd("aaaa")
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/rzpipe/open_base.py", line 231, in cmd
    res = self._cmd(cmd, **kwargs)
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/rzpipe/open_sync.py", line 102, in _cmd_process
    self.process.stdin.write((cmd + "\n").encode("utf8"))
BrokenPipeError: [Errno 32] Broken pipe

any ideas? I can upload the PE binary that makes it crash somewhere if that helps.

AttributeError: 'str' object has no attribute 'get'

Crash with fwhunt-scan 2.1.0:

Using the input file https://github.com/binarly-io/VulnREsearch/blob/main/FwHunt/HP/BRLY-2021-040.yml

Backtrace is:

  File "/home/hughsie/Code/lvfs-website/lvfs/queries/utils.py", line 33, in _async_query_run
    _query_run(query)
  File "/home/hughsie/Code/lvfs-website/lvfs/queries/utils.py", line 177, in _query_run
    query.run_shard(query, md, shard)
  File "/home/hughsie/Code/lvfs-website/lvfs/queries/utils.py", line 47, in _query_run_shard_fwhunt_scan
    if scanner.results:
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/fwhunt_scan/uefi_scanner.py", line 1087, in results
    self._results = self._get_results()
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/fwhunt_scan/uefi_scanner.py", line 1075, in _get_results
    res = self._get_results_variants(uefi_rule.variants[variant])
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/fwhunt_scan/uefi_scanner.py", line 1049, in _get_results_variants
    res &= self._code_scanner(rule_variant.code)
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/fwhunt_scan/uefi_scanner.py", line 88, in code
    self._code = self._get_code()
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/fwhunt_scan/uefi_scanner.py", line 77, in _get_code
    pattern=c.get("pattern", None),
AttributeError: 'str' object has no attribute 'get'

Upload a new release to PyPI?

Before I can merge the LVFS side of the scanner I really need a proper release uploaded to pypi. The way I do this for python-cabarchive is:

make pkg
twine upload dist/*

You'll need to register an account here: https://pypi.org/account/register/

If you'd like me to do this let me know, although I think Alex would probably like @binarly-io to be in control of this.

ValueError: bytes_le is not a 16-char string

Currently testing fwhunt-scan on Kali Linux:

sudo python3 ./fwhunt_scan_analyzer.py scan /home/m1k3/firmware_stuff/Firmware_images/UEFI-Bios/bc0064.cap -r FwHunt/rules/Vulnerabilities/HP/BRLY-2021-007.yml -r FwHunt/rules/Vulnerabilities/HP/BRLY-2021-005.yml -r FwHunt/rules/Vulnerabilities/HP/BRLY-2022-010.yml -r FwHunt/rules/Vulnerabilities/HP/BRLY-2021-034.yml -r FwHunt/rules/Vulnerabilities/HP/BRLY-2021-040.yml -r <snip>

Was running into the following exception:

  File "/home/m1k3/git-repos/fwhunt-scan/fwhunt_scan/uefi_smm.py", line 269, in get_child_sw_smi_handler_bb
    handler_guid = str(uuid.UUID(bytes_le=guid_b)).upper()
  File "/usr/lib/python3.10/uuid.py", line 181, in __init__
    raise ValueError('bytes_le is not a 16-char string')
ValueError: bytes_le is not a 16-char string

Quick and dirty fix:

~/git-repos/fwhunt-scan/fwhunt_scan/uefi_smm.py - Line 269:
                try:
                    handler_guid = str(uuid.UUID(bytes_le=guid_b)).upper()
                except:
                    pass

Calling UefiAnalyzer(blob) on non-PE data takes a LOOONG time to finish

When you use UefiAnalyzer() on a small EFI binary it typically finishes in about 100ms. If you ask UefiAnalyzer() to analyse something that's not a PE file (e.g. a jpg, or a zip file, or a ROM image, or just some text) then it takes a long time to process the file. Like minutes per file.

echo "hello world" > foo2.efi
./env/bin/python ./foo.py

where foo.py is just:

from uefi_r2 import UefiAnalyzer
with open("./foo2.efi", "rb") as f:
    with UefiAnalyzer(blob=f.read()) as uefi_analyzer:
	print(uefi_analyzer.get_summary())

eventually it outputs:

{'core': {'file': 'shm://psm_07173507', 'fd': 3, 'size': 4294967295, 'humansz': '4.0G', 'iorw': False, 'mode': 'r-x', 'block': 256, 'format': 'any'}}

I assume it's just read 4G of data....

r2_get_p_guids is really, really slow

A DXE (in this case com.intel.Uefi.Application.ELabel) has a .data vsize of 258208 bytes. It takes a very, very, long time to find the protocol GUIDs. I left it running at 100% CPU for over 15 minutes and it was still going until I ctrl-c'd the job. I can provide the binary via email if required.

UefiAnalyzer cannot find BootService and RuntimeService

I followed the instruction in README and installed fwhunt_scan module, but the analyzer cannot find any services when calling get_summary.

Here is an example.

>>> from fwhunt_scan import UefiAnalyzer
>>> analyzer = UefiAnalyzer(image_path="EnglishDxe")
>>> analyzer.boot_services
[]
>>> analyzer.runtime_services
[]
>>> analyzer.get_summary()
{'core': {'file': 'EnglishDxe', 'fd': 3, 'size': 3136, 'humansz': '3.1K', 'iorw': False, 'mode': 'r-x', 'block': 256, 'format': 'pe64'}, 'bin': {'arch': 'x86', 'baddr': 65536, 'binsz': 3136, 'bintype': 'pe', 'bits': 64, 'retguard': False, 'class': 'PE32+', 'cmp.csum': '0x00007325', 'compiled': 'Thu Jan  1 08:00:00 1970 UTC+8', 'endian': 'LE', 'hdr.csum': '0x00000000', 'laddr': 0, 'lang': 'c', 'machine': 'AMD 64', 'maxopsz': 16, 'minopsz': 1, 'os': 'efi', 'overlay': False, 'cc': 'ms', 'pcalign': 0, 'signed': False, 'subsys': 'EFI Boot Service Driver', 'stripped': False, 'crypto': False, 'havecode': True, 'va': True, 'sanitiz': False, 'static': True, 'linenum': False, 'lsyms': False, 'canary': False, 'PIE': False, 'RELROCS': False, 'NX': False}, 'g_bs': [67568], 'g_rt': [], 'g_smst': [], 'bs_list': [], 'rt_list': [], 'protocols': [], 'nvram_vars': [], 'p_guids': [{'value': 'A4C751FC-23AE-4C3E-92E94964CF63F349', 'name': 'EFI_UNICODE_COLLATION2_PROTOCOL_GUID', 'address': 67328}, {'value': '1D85CD7F-F43D-11D2-9A0C0090273FC14D', 'name': 'EFI_UNICODE_COLLATION_PROTOCOL_GUID', 'address': 67344}]}
>>>

The rizin version is as follow:

(fwhunt) ➜  fwhunt-scan git:(master) ✗ rizin -v
rizin 0.4.1 @ linux-x86-64
commit: 9023f8b997db210cef3b9a25cf1748fbc94942ed, build: 2022-09-10__02:22:47

I tried to find out the problem, it seems like the comparison here returns not equal.

(g_bs_area_insn["type"] == "ucall")

In rizin, the call instruction has a ircall type rather than ucall.

[0x00010362]> pdb
│           0x00010349      and   qword [var_28h], 0
│           0x0001034f      lea   rax, [0x00010758]
│           0x00010356      mov   qword [var_20h], rax
│           0x0001035b      lea   r9, section..data                    ; 0x10700
│           0x00010362      mov   rax, qword [0x000107f0]              ; [0x107f0:8]=0
│           0x00010369      lea   r8, [0x00010720]
│           0x00010370      lea   rdx, [0x00010710]
│           0x00010377      lea   rcx, [0x000107e0]
│           0x0001037e      call  qword [rax + 0x148]                  ; 328
│           0x00010384      add   rsp, 0x38
└           0x00010388      ret
[0x00010362]> pdj 1 @ 0x1037e
[{"offset":66430,"ptr":328,"esil":"0x148,rax,+,[8],rip,8,rsp,-=,rsp,=[],rip,=","refptr":false,"fcn_addr":66208,"fcn_last":66435,"size":6,"opcode":"call qword [rax + 0x148]","disasm":"call qword [rax + 0x148]","bytes":"ff9048010000","family":"cpu","type":"ircall","reloc":false,"type_num":402653188,"type2_num":0}]
[0x00010362]>

The issue was fixed when changing all these comparisons to ircall.

UefiAnalyzer(blob) seems broken in git

Using master, this works:

uefi_analyzer = UefiAnalyzer(image_path=image_path)

but using this failed:

blob = open(image_path, "rb").read()
uefi_analyzer = UefiAnalyzer(blob=blob)

It doesn't produce any output... interestingly in the lvfs-website self tests I get:

  File "/home/hughsie/Code/lvfs-website/plugins/uefi_scanner/__init__.py", line 105, in _run_test_on_shard
    with UefiAnalyzer(blob=shard.blob, rizinhome=rizinhome) as uefi_analyzer:
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/uefi_r2/uefi_analyzer.py", line 89, in __init__
    self._rz.cmd("aaaa")
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/rzpipe/open_base.py", line 244, in cmd
    res = self._cmd(cmd, **kwargs)
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.10/site-packages/rzpipe/open_sync.py", line 104, in _cmd_process
    self.process.stdin.write((cmd + "\n").encode("utf8"))
BrokenPipeError: [Errno 32] Broken pipe

crash with HP firmware

I've just scanned a ton of firmware of the LVFS with 1.0.2, and managed to get:

  File "/home/hughsie/Code/lvfs-website/env/lib/python3.9/site-packages/uefi_r2/uefi_scanner.py", line 459, in _get_result
    result &= self._compare_guids()
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.9/site-packages/uefi_r2/uefi_scanner.py", line 417, in _compare_guids
    for guid_analyzer in self._uefi_analyzer.protocol_guids:
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.9/site-packages/uefi_r2/uefi_analyzer.py", line 540, in protocol_guids
    self._protocol_guids = self._get_protocol_guids()
  File "/home/hughsie/Code/lvfs-website/env/lib/python3.9/site-packages/uefi_r2/uefi_analyzer.py", line 514, in _get_protocol_guids
    for section in self.sections:
TypeError: 'NoneType' object is not iterable

I think we always need to check if we have no sections. i.e. either change the mypy type from sections(self) -> List[Any] to sections(self) -> Optional[List[Any]] and then fix up all the callers to be able to do something sane on None -- or, perhaps better to check the self._r2.cmdj("aflj") output, and if None then return [].

I'm happy to produce a PR if you tell me which you'd rather.

click upgrade causes problems on LVFS

The conflict is caused by:
The user requested click==7.1.2
celery 5.1.2 depends on click<8.0 and >=7.0
click-didyoumean 0.3.0 depends on click>=7
click-repl 0.2.0 depends on click
flask 2.0.2 depends on click>=7.1.2
uefi-r2 1.2.0 depends on click~=8.0.0

Can we just change the click to be unversioned? or make it >=7 perhaps?

Not easy to get the metadata out of UefiRule

At the moment I'm using it like this:

            uefi_rule = UefiRule(rule_fn)
            uefi_rule_name = uefi_rule._uefi_rule['meta']['name']
            uefi_rule_description = uefi_rule._uefi_rule['meta']['description']

This is a bit cumbersome. Perhaps we could have some nice property helpers like uefi_rule.name?

TypeError: 'NoneType' object is not iterable

When running on the LVFS I get:

  File ".../env/lib/python3.10/site-packages/fwhunt_scan/uefi_scanner.py", line 834, in _get_bounds
    funcs = list(self._uefi_analyzer._rz.cmdj("aflqj"))
TypeError: 'NoneType' object is not iterable

firmware.zip

I've attached a file that reproduces the issue, rules are all of FwHunt/rules/Vulnerabilities/** -- the unzip password is lvfs

Help message says to use non-existing "scan" command

Steps to reproduce: run fwhunt_scan_analyzer.py scan-firmware -d FwHunt/rules dump.bin

Result: [I] Specify volume_guids in IntelAlderLakeLeak or use scan command

Expected result: running with scan instead of scan-firmware works.

Actual result: no such scan command.

Work-around: use a combination of extract and analyze_module on every extracted module, but it's tedious to do manually.

Expected fix: either provide an actual scan command, fix scan-firmware to do the right thing, or update the warning to avoid mentioning a non-existing solution.

Regression after adding TE support

Commit e6d1c68 regressed mypy, and there looks to be at least one actual bug:

./env/bin/mypy uefi_r2
uefi_r2/uefi_te.py:23: error: Call to untyped function "TerseExecutableError" in typed context
uefi_r2/uefi_te.py:31: error: Call to untyped function "TerseExecutableError" in typed context
uefi_r2/uefi_te.py:33: error: Incompatible types in assignment (expression has type "None", variable has type "int")
uefi_r2/uefi_te.py:34: error: Incompatible types in assignment (expression has type "None", variable has type "int")
uefi_r2/uefi_te.py:35: error: Incompatible types in assignment (expression has type "None", variable has type "int")
uefi_r2/uefi_te.py:36: error: Incompatible types in assignment (expression has type "None", variable has type "int")
uefi_r2/uefi_te.py:37: error: Incompatible types in assignment (expression has type "None", variable has type "int")
uefi_r2/uefi_te.py:38: error: Incompatible types in assignment (expression has type "None", variable has type "int")
uefi_r2/uefi_te.py:39: error: Incompatible types in assignment (expression has type "None", variable has type "int")
uefi_r2/uefi_te.py:60: error: Incompatible return value type (got "str", expected "int")

not compatible with python 3.6.x

Python 3.8.x and 3.9.x worked fine. 3.6.x errored our with the following. May want to update the readme

C:\uefi_r2>python setup.py install
Traceback (most recent call last):
File "setup.py", line 2, in
from uefi_r2 import author, email, version
File "C:\uefi_r2\uefi_r2_init_.py", line 11, in
from .uefi_analyzer import UefiAnalyzer, UefiAnalyzerError
File "C:\uefi_r2\uefi_r2\uefi_analyzer.py", line 13, in
from multiprocessing import shared_memory
ImportError: cannot import name 'shared_memory'

C:\uefi_r2>python --version
Python 3.6.6

uefi_extractor function is different in github version and pypi version

Hi, I found that parameter "file_guids" in function UefiExtractor is not the same in github version and pypi version.
Github version is <self._file_guids: List[str] = [g.lower() for g in file_guids]>, and pypi is <self._file_guid: str = file_guid.lower()>.
The datatype of the input parameter "file_guids" is different, the definition as well.

Please let us create UefiRule and UefiAnalyzer using a blob, not a file

On the LVFS we currently write hundreds of thousands of temporary files to run each uefi_r2 rule. Rather than do:

with tempfile.NamedTemporaryFile(
    mode="wb", prefix="uefi_scanner_", suffix=".efi", dir=None, delete=True
) as f:
    f.write(shard.blob)
    f.flush()
    uefi_analyzer = UefiAnalyzer(image_path=f.name)

what I really want to do is:

uefi_analyzer = UefiAnalyzer(blob=shard.blob)

... and the same thing for UefiRule (although we only call that one once...) -- many thanks.

Error : mapping values are not allowed here

Hi everyone, in the output after execution, "mapping values ​​are not allowed here", how to solve these errors? This is the first time I use this software, did I overlook something? Please help, grateful!

> # python3 fwhunt_scan_analyzer.py scan --rule rules/BRLY-2021-043.yml test/fw.BIN
> 
> Error: ScannerError(None, None, 'mapping values are not allowed here', <yaml.error.Mark object at 0x7f32a7fa75b0>)
> Scanner result None (variant: default) FwHunt rule has been triggered and threat detected! (test/fw.BIN)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.