mandiant/ghidrathon

The FLARE team's open-source extension to add Python 3 scripting to Ghidra.

ghidrathon's Issues

remove isinstance and issubclass workarounds

remove workaround added in #22 after Jep releases ninia/jep#440

undocumented: the move to separate state removed access to the `GhidraScript.state` variable

The splitting out of Python state is cool and all, and make sense.

It had one undocumented side-effect that bit me: previously, state would grab the GhidraState state member from GhidraScript directly and let me access the state that you.

It can be substituted by getState(), which is the Ghidra provided API for it, I suppose. I had to learn that the hard way because, until now, I'd never actually needed to use the API. (Jython provides the same direct access to that variable.)

It may be worth a late-breaking addition to your release notes to save the next person who runs into this the same confusion. (or not, but I figured I'd mention it.)

PS: on the plus side, they upgrade saved a bug report about an interpreter crash that has been fixed along the way, so good work there.

make installation easier

Despite being familiar with Ghidrathon development, there have been times that I've avoided using Ghidrathon because its a pain to install. When I visit a project and see that I have to compile software from source I am much less likely to experiment with it (probably due to past experiences when the compilation failed, and I'm stuck debugging the build process).

We should investigate how to make Ghidrathon easier to setup and use.

In a perfect world, I think I'd want to see: users download a single file from GH releases page, put it in a specific directory, and everything just works. How close can we get to this ideal?

I can't build ghidrathon.

Tried couple of things:
ghidra is in ~/Documents/ghidra_10.3.1_PUBLIC
set an environment GHIDRA_INSTALL_DIR="~/Documents/ghidra_10.3.1_PUBLIC" it DOES NOT FIND it when i use this.
And if i use -PGHIDRA_INSTALL_DIR somehow it outputs this error, I cant seem to fix.
Few more information:
Ubuntu 22.04
jep 4.1.1 installed
OpenJDK 17
ghidra 10.3.1
gradle 8.1.1
my gradle is in ~/Documents as well

error looks like this:

> Configure project :
Using Ghidra install directory ~/Documents/ghidra_10.3.1_PUBLIC/

FAILURE: Build failed with an exception.

* Where:
Build file '/home/anzo/Documents/Ghidrathon-2.1.0/build.gradle' line: 71

* What went wrong:
A problem occurred evaluating root project 'Ghidrathon-2.1.0'.
> Could not read script '/home/anzo/.gradle/daemon/8.1.1/~/Documents/ghidra_10.3.1_PUBLIC/support/buildExtension.gradle' as it does not exist.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 332ms

Please help.

Trouble disabling Jython

The installation instructions suggest disabling Jython. However, when I browse to Configure > Ghidra Core, I don't see PythonPlugin listed. Am I looking in the wrong place? Thank you.

Using Ghidra v10.1.5

Installing and running on Windows

First off, great work on the development and architecture of this plugin I'm looking forward to Ghidra + Python3 scripts!

I wanted to document my process on attempting to getting this plugin running:

Prerequisites notes

Python

Python 3.10 was installed through the Windows Store
i.e. installed to:
%LocalAppData%\Packages\PythonSoftwareFoundation.Python.3.10_...\
and therefore pip --user install jep installed to:
%LocalAppData%\Packages\PythonSoftwareFoundation.Python.3.10_...\LocalCache\local-packages\Python310\site-packages\jep

Java

Eclipse Adoptium OpenJDK installed to:
C:\Program Files\Eclipse Adoptium\jdk-11.0.16.101-hotspot
with both bin\client\jvm.dll and bin\server\jvm.dll

Building

Build succeeded with a small deprecation warning, observable with this diff:

diff --git a/build.gradle b/build.gradle
index 618e1e7..0aa608b 100644
--- a/build.gradle
+++ b/build.gradle
@@ -53,6 +53,10 @@ tasks.matching { it.name != 'copyJepNativeBinaries' && it.name != 'installJep' }
     task.dependsOn copyJepNativeBinaries
 }
 
+tasks.withType(JavaCompile) {
+    options.deprecation = true
+}
+
 // from here we use the standard Gradle build provided by Ghidra framework
 
 //----------------------START "DO NOT MODIFY" SECTION------------------------------

See: #9

And the Jep binaries were correctly discovered and bundled with the Ghidra plugin under dist/

Installing

I get a Ghidra Exception on loading the plugin after a restart:

%USERPROFILE%\.ghidra\.ghidra_10.1.5_PUBLIC\Extensions\Ghidrathon\os\win_x86_64\jep.dll: Can't find dependent libraries
java.lang.UnsatisfiedLinkError: %USERPROFILE%\.ghidra\.ghidra_10.1.5_PUBLIC\Extensions\Ghidrathon\os\win_x86_64\jep.dll: Can't find dependent libraries

in investigating the jep.dll with ldd in git bash Thanks Windows

ldd ./.ghidra/.ghidra_10.1.5_PUBLIC/Extensions/Ghidrathon/os/win_x86_64/jep.dll
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffc5eea0000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffc5d3c0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffc5c380000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffc5ed90000)
        jep.dll => %USERPROFILE%/.ghidra/.ghidra_10.1.5_PUBLIC/Extensions/Ghidrathon/os/win_x86_64/jep.dll (0x7ffc50b60000)
        ucrtbase.dll => /c/Windows/System32/ucrtbase.dll (0x7ffc5cb70000)
        vcruntime140.dll => /c/Windows/System32/vcruntime140.dll (0x7ffc35bd0000)
        jvm.dll => not found
        python310.dll => not found
        api-ms-win-crt-heap-l1-1-0.dll => /c/Program Files/Eclipse Adoptium/jdk-11.0.16.101-hotspot/bin/api-ms-win-crt-heap-l1-1-0.dll (?)
        api-ms-win-crt-runtime-l1-1-0.dll => /c/Program Files/Eclipse Adoptium/jdk-11.0.16.101-hotspot/bin/api-ms-win-crt-runtime-l1-1-0.dll (?)
        api-ms-win-crt-stdio-l1-1-0.dll => /c/Program Files/Eclipse Adoptium/jdk-11.0.16.101-hotspot/bin/api-ms-win-crt-stdio-l1-1-0.dll (?)
        api-ms-win-crt-convert-l1-1-0.dll => /c/Program Files/Eclipse Adoptium/jdk-11.0.16.101-hotspot/bin/api-ms-win-crt-convert-l1-1-0.dll (?)
        api-ms-win-crt-string-l1-1-0.dll => /c/Program Files/Eclipse Adoptium/jdk-11.0.16.101-hotspot/bin/api-ms-win-crt-string-l1-1-0.dll (?)

There are two unreachable DLLs

• jvm.dll could not be found out of the Eclipse Adoptium OpenJDK install directory (or maybe not able to disambiguate between the two?)
• python310.dll seems to not be distributed with the windows store build (perhaps a static build?)

I understand that these are likely issues specific to Jep itself however I wanted to report general usability issues to either make more meaningful Ghidra error messages and to upstream these issues to Jep is called for.

Thanks again for the project!

Operating System: Windows 11 21H2 22000.856
Java Version: OpenJDK 11.0.16.1
Ghidra Version: 10.1.5
Gradle Version: 7.5.1
Python Version: 3.10.7
Jep Version: 4.0.3

Java packages overshadow Python modules when resolving imports

Hi @0ddc0de - thank you for reaching out. TLDR; this unfortunately appears to be a bug in Ghidrathon. In theory, after fixing this bug what you are trying to do should work, although, I am uncertain without testing how dropping into a REPL may mess with headless execution flow.

Long winded explanation and notes:

Based on ImportError: java.lang.ClassNotFoundException: pdb.Pdb it appears Ghidra's ghidra.app.util.bin.format.pdb package is overshadowing Python's pdb module. This is likely due to Jep's import hook giving preference to Java over Python when resolving imports.

We should give preference to Python when executing Python code but still allow access to the overshadowed Java package. This likely requires wrapping Jep's ClassEnquirer as discussed here or modifying Jep's java_import_hook.py.

Originally posted by @mike-hunhoff in #37 (comment)

getScriptArgs API behavior differs from standard Ghidra in headless mode

Description

If you use Ghidrathon in headless mode with both a -preScript and -postScript, the call to getScriptArgs() does not work as expected.

If you launch an analysis with the following command line:

analyzeHeadless /tmp/out out -import /bin/ls -scriptPath -preScript script.py arg_a arg_b -postScript script.py arg_x arg_y

The API call getScriptArgs() will return arguments arg_a and arg_b for both the preScript and postScript.

Of course it gets worse if the user does not use preScript since the argument won't be visible to postScript at all.

Expected behavior

The API call getScriptArgs() should return ["arg_a", "arg_b"] for preScript and ["arg_x", "arg_y"] for postScript.

Environment Info

• OS: Linux (Debian 11)
• Python version: 3.10.12
• Ghidra version: 10.3
• Ghidrathon version: 2.2.1
• openjdk version 17.0.7" 2023-04-18

Reproduction Steps

rm -rf /tmp/out && mkdir /tmp/out
echo "print([x for x in getScriptArgs()])" > /tmp/script.py
analyzeHeadless /tmp/out out -import /bin/ls -scriptPath -preScript script.py arg_a arg_b -postScript script.py arg_x arg_y

Command line output

openjdk version "17.0.7" 2023-04-18
OpenJDK Runtime Environment (build 17.0.7+7-Debian-1deb11u1)
OpenJDK 64-Bit Server VM (build 17.0.7+7-Debian-1deb11u1, mixed mode)
INFO  Using log config file: jar:file:/opt/ghidra/Ghidra/Framework/Generic/lib/Generic.jar!/generic.log4j.xml (LoggingInitialization)  
INFO  Using log file: /app/.ghidra/.ghidra_10.3_PUBLIC/application.log (LoggingInitialization)  
INFO  Loading user preferences: /app/.ghidra/.ghidra_10.3_PUBLIC/preferences (Preferences)  
INFO  Searching for classes... (ClassSearcher)  
INFO  Class search complete (869 ms) (ClassSearcher)  
INFO  Initializing SSL Context (SSLContextInitializer)  
INFO  Initializing Random Number Generator... (SecureRandomFactory)  
INFO  Random Number Generator initialization complete: NativePRNGNonBlocking (SecureRandomFactory)  
INFO  Trust manager disabled, cacerts have not been set (ApplicationTrustManagerFactory)  
INFO  HEADLESS Script Paths:
    /opt/ghidra/Ghidra/Features/SystemEmulation/ghidra_scripts
    /opt/ghidra/Ghidra/Features/GnuDemangler/ghidra_scripts
    /opt/ghidra/Ghidra/Processors/DATA/ghidra_scripts
    /opt/ghidra/Ghidra/Processors/JVM/ghidra_scripts
    /opt/ghidra/Ghidra/Features/MicrosoftCodeAnalyzer/ghidra_scripts
    /opt/ghidra/Ghidra/Features/Python/ghidra_scripts
    /opt/ghidra/Ghidra/Features/VersionTracking/ghidra_scripts
    /opt/ghidra/Ghidra/Features/Base/ghidra_scripts
    /opt/ghidra/Ghidra/Processors/Atmel/ghidra_scripts
    /opt/ghidra/Ghidra/Features/Decompiler/ghidra_scripts
    /opt/ghidra/Ghidra/Features/FunctionID/ghidra_scripts
    /opt/ghidra/Ghidra/Features/PDB/ghidra_scripts
    /opt/ghidra/Ghidra/Processors/8051/ghidra_scripts
    /opt/ghidra/Ghidra/Processors/PIC/ghidra_scripts
    /tmp
    /opt/ghidra/Ghidra/Features/FileFormats/ghidra_scripts
    /opt/ghidra/Ghidra/Extensions/ghidrathon/ghidra_scripts
    /opt/ghidra/Ghidra/Features/BytePatterns/ghidra_scripts (HeadlessAnalyzer)  
INFO  HEADLESS: execution starts (HeadlessAnalyzer)  
INFO  Creating project: /tmp/out/out (HeadlessAnalyzer)  
INFO  Creating project: /tmp/out/out (DefaultProject)  
INFO  REPORT: Processing input files:  (HeadlessAnalyzer)  
INFO       project: /tmp/out/out (HeadlessAnalyzer)  
INFO  IMPORTING: /bin/ls (HeadlessAnalyzer)  
INFO  Using Loader: Executable and Linking Format (ELF) (AutoImporter)  
INFO  Using Language/Compiler: x86:LE:64:default:gcc (AutoImporter)  
INFO  IMPORTING: Loaded 0 additional files (HeadlessAnalyzer)  
INFO  SCRIPT: /tmp/script.py (HeadlessAnalyzer)  
['arg_a', 'arg_b']
INFO  ANALYZING all memory and code: /bin/ls (HeadlessAnalyzer)  
INFO  DWARF external debug information found: ExternalDebugInfo [filename=61a544c35b9dc1d172d1a1c09043e487326966.debug, crc=d165fd8a, hash=6461a544c35b9dc1d172d1a1c09043e487326966] (ExternalDebugFilesService)  
INFO  Unable to find DWARF information, skipping DWARF analysis (DWARFAnalyzer)  
INFO  hit non-returning function, restarting decompiler switch analyzer later (DecompilerSwitchAnalyzer)  
INFO  Packed database cache: /tmp/analysis-Ghidra/packed-db-cache (PackedDatabaseCache)  
INFO  Applied data type archive: generic_clib_64 (ApplyDataArchiveAnalyzer)  
INFO  -----------------------------------------------------
    ASCII Strings                              0.421 secs
    Apply Data Archives                        0.227 secs
    Call Convention ID                         0.012 secs
    Call-Fixup Installer                       0.015 secs
    Create Address Tables                      0.055 secs
    Create Address Tables - One Time           0.021 secs
    Create Function                            0.042 secs
    DWARF                                      0.008 secs
    Data Reference                             0.053 secs
    Decompiler Switch Analysis                 0.089 secs
    Decompiler Switch Analysis - One Time      2.851 secs
    Demangler GNU                              0.021 secs
    Disassemble Entry Points                   1.020 secs
    ELF Scalar Operand References              0.326 secs
    Embedded Media                             0.019 secs
    External Entry References                  0.000 secs
    Function Start Search                      0.043 secs
    Function Start Search After Code           0.012 secs
    Function Start Search After Data           0.009 secs
    GCC Exception Handlers                     0.412 secs
    Non-Returning Functions - Discovered       0.240 secs
    Non-Returning Functions - Known            0.005 secs
    Reference                                  0.159 secs
    Shared Return Calls                        0.112 secs
    Stack                                      1.741 secs
    Subroutine References                      0.046 secs
    Subroutine References - One Time           0.000 secs
    x86 Constant Reference Analyzer            1.973 secs
-----------------------------------------------------
     Total Time   9 secs
-----------------------------------------------------
 (AutoAnalysisManager)  
INFO  REPORT: Analysis succeeded for file: /bin/ls (HeadlessAnalyzer)  
INFO  SCRIPT: /tmp/script.py (HeadlessAnalyzer)  
['arg_a', 'arg_b']
INFO  ANALYZING changes made by post scripts: /bin/ls (HeadlessAnalyzer)  
INFO  REPORT: Post-analysis succeeded for file: /bin/ls (HeadlessAnalyzer)  
INFO  REPORT: Save succeeded for: /ls (out:/ls) (HeadlessAnalyzer)  
INFO  REPORT: Import succeeded (HeadlessAnalyzer) 

Make GHIDRA_INSTALL_DIR less ambiguous

    Thank you @as0ni . Out of curiosity do you have an environment variable named `GHIDRA_INSTALL_DIR` set? I'm wondering if the following code from Ghidrathon's gradle script is causing gradle to reference an older Ghidra install directory during the build: https://github.com/mandiant/Ghidrathon/blob/83515552de0b8694a0a1c8b26b9a8303f41046be/build.gradle#L61-L66

Originally posted by @mike-hunhoff in #28 (comment)

add MacOS support

see #4

• x86_64
• M1 ARM

remove stream workaround

see if we can use JepConfig built-in redirection added in v4.0:

https://github.com/ninia/jep/blob/b926ca4af1df7a496ea0a8cb7d030bbe310de898/src/main/java/jep/JepConfig.java#L123-L151

Headless Ghidrathon extension installation

After building the extension, Ghidra's documentation tells us to unzip it in the $GHIDRA_INSTALL_DIR/Extensions diretory; however, putting it in this directory only gives it visibility to perform the actual installation via the gui. To avoid having to configure the extension in the gui, we must extract the zip to ~/.ghidra/.ghidra_<version>/Extensions

note: The Extensions folder will not be present in the hidden directories (.ghidra*), so we must manually create it before/ during the extraction process

Invocation of the headlessAnalyzer should now be using Ghidrathon to process python scripts instead of the default Jython plugin.

ux: interpreter REPL multi-line support broken in Python 3.8

New Issue using Conda

I am starting to get this issue with my plugin after installing ghidrathon.

ImportError: PyO3 modules may only be initialized once per interpreter process

Ghidrathon Incompatible with Python Threads

Hi there Ghidrathon team,

First, I want to say thank you for open-sourcing such a cool tool! It's awesome. I've tried a lot of Python3 -> Ghidra plugins, and this is an epic one. I found a limitation that I thought might be worth mentioning in your README. To preface, I have read the thread limitations section of the JEP project before submitting this.

Whenever you attempt to use the Ghidrathon Python3 interpreter from another Python thread, two critical problems happen:

1. JEP, which was used to make a single Java Interpreter flips out because a thread inside the interpreter is still alive after the main thread has died
2. All objects that used to be in the globals, as well as connected classes, are outdated.

To better demonstrate what I mean, here is a POC you can run:

from threading import Thread
import time

def loc_printer():
    i = 0 
    while i < 10: 
        i += 1
        time.sleep(1)
        print(f"Current location: {str(currentAddress)}") 

if __name__ == "__main__":
    t = Thread(target=loc_printer)
    t.daemon = True
    t.start()
    print("running thread!")

Place this code in a file and run it from the script manager. You will notice you get a scary-looking error from JEP:

2022-08-29 | 21:34:10 | ERROR | (ConsoleComponentProvider) Unexpected Exception: jep.JepException: All threads must be stopped before closing Jep. java.lang.RuntimeException: jep.JepException: All threads must be stopped before closing Jep.

You will also notice that the printed value of currentAddress never actually changes even if you click around your decompiler. This second bug is something I've tracked across a few projects, so I know its not something you can likely fix. It has something to do with the state passed through the Python interpreter.

Unfortunately, I come only bringing problems and no solutions :(. I thought it would be at least worth mentioning since it is unclear that Python threads will not work. In JEP docs, it seems as though having multiple Interpreter objects in Java would be the only issue, not having multiple threads in the Python Interpreter.

for loop throws Python TypeError for some Iterable objects

Description

Most iterable objects returned by Ghidra API calls are iterable via classic for loops; however, there are some cases in which the iterable object is not compatible.

Steps to Re-create

def extract_function_loop(fh: FunctionHandle):
    f: ghidra.program.database.function.FunctionDB = fh.inner

    edges = []
    for block in SimpleBlockIterator(BasicBlockModel(currentProgram()), f.getBody(), monitor()):  # type: ignore [name-defined] # noqa: F821
        dests = block.getDestinations(monitor())  # type: ignore [name-defined] # noqa: F821
        s_addrs = block.getStartAddresses()

        while dests.hasNext():  # For loop throws Python TypeError
            for addr in s_addrs:
                edges.append((addr.getOffset(), dests.next().getDestinationAddress().getOffset()))

    if loops.has_loop(edges):
        yield Characteristic("loop"), AbsoluteVirtualAddress(f.getEntryPoint().getOffset())

https://github.com/mandiant/capa/blob/70d36ab6406a46b5bc9c2cac3866a80a8c8ea38d/capa/features/extractors/ghidra/function.py#L28C1-L42C1

Expected Behavior

dests should be compatible to use with a classic for loop i.e. for dest in dests:

Actual Behavior

Using a classic for loop throws a Python TypeError

Version Info

Ghidra v10.3.2
Ghidrathon v3.0.0

Could not build plugin for Ghidra 10.2

Plugin is not able to build for Ghidra 10.2:

$ gradle -PGHIDRA_INSTALL_DIR=../ghidra_10.2_PUBLIC

> Task :installJep
Requirement already satisfied: jep in /home/user/venvs/main/lib/python3.10/site-packages (4.1.0)

[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip

> Task :copyJepNativeBinaries
2022-11-06 13:28:22,584 INFO     Searching for Jep Python module directory
2022-11-06 13:28:22,584 INFO     Found Jep Python module directory at /home/user/venvs/main/lib/python3.10/site-packages/jep
2022-11-06 13:28:22,584 INFO     Copying libjep.so and /home/user/venvs/main/lib/python3.10/site-packages/jep/jep-4.1.0.jar to extension folders
2022-11-06 13:28:22,586 INFO     Done

> Task :compileJava FAILED
/home/user/srcs/ghidra-related/Ghidrathon/src/main/java/ghidrathon/GhidrathonScriptProvider.java:37: error: getScriptInstance(ResourceFile,PrintWriter) in GhidrathonScriptProvider cannot override getScriptInstance(ResourceFile,PrintWriter) in GhidraScriptProvider
        public GhidraScript getScriptInstance(ResourceFile sourceFile, PrintWriter writer)
                            ^
  overridden method does not throw IllegalAccessException
/home/user/srcs/ghidra-related/Ghidrathon/src/main/java/ghidrathon/GhidrathonPlugin.java:53: warning: [removal] ProgramPlugin(PluginTool,boolean,boolean,boolean) in ProgramPlugin has been deprecated and marked for removal
                super(tool, true, true, true);
                ^
1 error
1 warning

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
3 actionable tasks: 3 executed

Unable to install on Intel MacOS - java.lang.UnsatisfiedLinkError of libjep.so

After building and installing the latest version, I get the following error:

Can't load library: /Users/jeroen/.ghidra/.ghidra_10.3.3_PUBLIC/Extensions/Ghidrathon-3.0.1/os/mac_x86_64/libjep.so
java.lang.UnsatisfiedLinkError: Can't load library: /Users/jeroen/.ghidra/.ghidra_10.3.3_PUBLIC/Extensions/Ghidrathon-3.0.1/os/mac_x86_64/libjep.so
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2398)
	at java.base/java.lang.Runtime.load0(Runtime.java:785)
	at java.base/java.lang.System.load(System.java:1979)
	at jep.MainInterpreter.initialize(MainInterpreter.java:126)
	at jep.MainInterpreter.getMainInterpreter(MainInterpreter.java:101)
	at jep.Jep.<init>(Jep.java:133)
	at jep.SharedInterpreter.<init>(SharedInterpreter.java:60)
	at ghidrathon.interpreter.GhidrathonInterpreter.<init>(GhidrathonInterpreter.java:72)
	at ghidrathon.interpreter.GhidrathonInterpreter.get(GhidrathonInterpreter.java:232)
	at ghidrathon.GhidrathonConsoleInputThread.run(GhidrathonConsoleInputThread.java:66)

---------------------------------------------------
Build Date: 2023-Aug-29 1442 EDT
Ghidra Version: 10.3.3
Java Home: /Library/Java/JavaVirtualMachines/jdk-18.0.2.jdk/Contents/Home
JVM Version: Oracle Corporation 18.0.2
OS: Mac OS X 12.6 x86_64

The file is there though...

(.venv) ➜  mac_x86_64 ls -lah
total 1016
drwxr-x---  4 jeroen  staff   128B Sep  9 14:22 .
drwxr-x---  5 jeroen  staff   160B Sep  9 14:22 ..
-rw-r--r--  1 jeroen  staff   235B Sep  9 14:22 README.txt
-rwxr-xr-x  1 jeroen  staff   502K Sep  9 14:22 libjep.so
(.venv) ➜  mac_x86_64 file libjep.so 
libjep.so: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64Mach-O 64-bit bundle x86_64] [arm64:Mach-O 64-bit bundle arm64Mach-O 64-bit bundle arm64]
libjep.so (for architecture x86_64):	Mach-O 64-bit bundle x86_64
libjep.so (for architecture arm64):	Mach-O 64-bit bundle arm64
(.venv) ➜  mac_x86_64 pwd
/Users/jeroen/.ghidra/.ghidra_10.3.3_PUBLIC/Extensions/Ghidrathon-3.0.1/os/mac_x86_64

[Feat] Documentation translation section in README.md

Can we add a translation section within the README file ?
This section will provide translated versions of the content in different languages.
If yes, kindly assign to me under GSSOC'23

If interactive interpreter of Ghidrathon can be used in other text editor or IDEs.

I would like to know if the Ghidrathon Python interpreter can be used in Pycharm or other IDEs. I find it very difficult to use the interpreter in the GUI.

isinstance() not working for Ghidra types

Seems like isinstance() is broken for Ghidra types:

   _____ _     _     _           _   _                 
  / ____| |   (_)   | |         | | | |                
 | |  __| |__  _  __| |_ __ __ _| |_| |__   ___  _ __  
 | | |_ | '_ \| |/ _` | '__/ _` | __| '_ \ / _ \| '_ \ 
 | |__| | | | | | (_| | | | (_| | |_| | | | (_) | | | |
  \_____|_| |_|_|\__,_|_|  \__,_|\__|_| |_|\___/|_| |_|
                                                       
Python 3.10.6 Interpreter for Ghidra. Developed by FLARE.

>>> from ghidra.program.database.data import *
>>> dm = currentProgram.getDataTypeManager()
>>> dt = next(x for x in dm.getAllDataTypes() if x.getName() == 'CreateProcessW')
>>> print(type(dt))
<class 'ghidra.program.database.data.FunctionDefinitionDB'>
>>> print(FunctionDefinitionDB)
class ghidra.program.database.data.FunctionDefinitionDB
>>> isinstance(dt, FunctionDefinitionDB)
Traceback (most recent call last):
  File "/Users/pieceofsummer/.ghidra/.ghidra_10.1.5_PUBLIC/Extensions/Ghidrathon-1.0.0/data/python/jepeval.py", line 66, in jepeval
    more_input_needed = _jepeval(line)
  File "/Users/pieceofsummer/.ghidra/.ghidra_10.1.5_PUBLIC/Extensions/Ghidrathon-1.0.0/data/python/jepeval.py", line 49, in _jepeval
    exec(compile(line, "<string>", "single"), globals(), globals())
  File "<string>", line 1, in <module>
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union

fix build warning for Ghidra 10.2 + JDK 17+

...\GhidrathonPlugin.java:53: warning: [removal] ProgramPlugin(PluginTool,boolean,boolean,boolean) in ProgramPlugin has been deprecated and marked for removal
                super(tool, true, true, true);
                ^
1 warning

automatically print environment details upon stack trace

We should consider installing a top level exception handler that logs basic environmental details, such as OS, python version, ghidra version, when displaying an exception and stack trace. This will make it easier to create bug reports with sufficient information, like in #65

update installation documentation to include extracting directly to Ghidra/Extensions folder

pending #27

doc: https://ghidra-sre.org/InstallationGuide.html#Extensions

add ARM support

Thanks for this project! I am trying to deploy this in a Docker container (linux+arm64) without access to a GUI to turn on the extension, and have been debugging trying to find out why it doesn't seem to be loading after I clone it.

Ghidra's documentation here seems to indicate that I should be able to simply clone this into the Ghidra extensions directory, so given a local installation path of /home/nonroot/ghidra, I would expect that to be /home/nonroot/ghidra/Ghidra/Extensions. The extension is unzipped and at /home/nonroot/ghidra/Ghidra/Extensions/Ghidrathon-2.0.0, and I've made sure to buildNatives, so I'd think I'd be good to go from there.

Still pretty new, so it's likely there's something I'm missing here. Attempting to run a headless analysis seems to be unable to parse the example script as Python 3, though:

INFO  SCRIPT: /home/nonroot/ghidra/Ghidra/Extensions/Ghidrathon-2.0.0/ghidra_scripts/ghidrathon_example.py (HeadlessAnalyzer)
  File "/home/nonroot/ghidra/Ghidra/Extensions/Ghidrathon-2.0.0/ghidra_scripts/ghidrathon_example.py", line 30
    f"Function {func.getName()} @ {hex(func.getEntryPoint().getOffset())}: {block_count} blocks, {insn_count} instructions"
    ^
SyntaxError: no viable alternative at input '"Function {func.getName()} @ {hex(func.getEntryPoint().getOffset())}: {block_count} blocks, {insn_count} instructions"'

Please let me know what context would be helpful here, if any.

sys.exit() closes Ghidra instead of terminating the script

Hello,
In the default Python interpreter of Ghidra, invoking sys.exit() or just exit() stops the script execution. This is handy if you want to prevent your script from proceeding with the execution. When I ported my script to Ghidrathon, it took me some time to figure out why Ghidra was crashing when executing the script: sys.exit() exits Ghidra, which is not the expected behavior (terminating the script). The only way of stopping scripts prematurely is to raise an error, as returning from a non-function is not allowed.

TLDR: sys.exit() should quit the current script/Python environment instead of closing Ghidra

Unhandled exception when clicking empty autocomplete option in interpreter window

I am using a fairly recent commit of ghidra (NationalSecurityAgency/ghidra@9e83921) and the latest (b7bedaf) of Ghidrathon

Autocomplete provides no completions for me:

and clicking it gives me an NPE (although this might be a Ghidra issue) :

Cannot invoke "java.util.List.get(int)" because "this.list" is null
java.lang.NullPointerException: Cannot invoke "java.util.List.get(int)" because "this.list" is null
	at ghidra.app.plugin.core.interpreter.CodeCompletionListSelectionModel.setSelectionInterval(CodeCompletionWindow.java:378)
	at java.desktop/javax.swing.JList.setSelectionInterval(JList.java:2104)
	at java.desktop/javax.swing.plaf.basic.BasicListUI$Handler.adjustSelection(BasicListUI.java:2931)
	at java.desktop/javax.swing.plaf.basic.BasicListUI$Handler.mousePressed(BasicListUI.java:2887)
	at java.desktop/java.awt.AWTEventMulticaster.mousePressed(AWTEventMulticaster.java:288)
	at java.desktop/java.awt.AWTEventMulticaster.mousePressed(AWTEventMulticaster.java:287)
	at java.desktop/java.awt.Component.processMouseEvent(Component.java:6623)
	at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3389)

Are there any possible mistakes/omissions I might have made while building/installing to look out for? It appears (#5) that autocomplete isn't a thing yet. Are there any potential problems if I run the Jython side-by-side for autocomplete and disable it before running scripts?

doc: disable Jython interpreter w/out using Ghidra UI

Can we disable the Jython extension without using Ghidra's UI? This is useful when configuring Ghidrathon where UI access is not available.

Ensure sys.executable set correctly

@Forsworns oh this is an interesting one. Based on the output that you provided I tracked the offending lines of code to packaging.types: https://github.com/pypa/packaging/blob/3030822b0b06a43fbbb2710da7b0846d1bebd2ba/src/packaging/tags.py#L399-L410

The code calls subprocess.run where the target executable is pulled from sys.executable. sys.executable is set correctly in my Linux test environment:

>>> import sys
>>> sys.executable
'/home/spring/dragon/.env/bin/python3`

Can you run the code listed above from your Ghidrathon interpreter? I'm guessing sys.executable is set to the Java executable used to run Ghidra in your Mac environment...

Originally posted by @mike-hunhoff in #59 (comment)

doc: no support for subclassing Java classes using Python classes

Example 1:

from ghidra.app.tablechooser import StringColumnDisplay

class Example(StringColumnDisplay):
    def __init__(self, title, index):
        self.title = title
        self.index = index

    def getColumnName(self):
        return self.title
# Error: TypeError: cannot create 'jep.PyJClass' instances

Example 2:

from ghidra.app.tablechooser import StringColumnDisplay

class Example(StringColumnDisplay.__pytype__):
    def __init__(self, title, index):
        self.title = title
        self.index = index

    def getColumnName(self):
        return self.title
# Error: TypeError: cannot create 'Example' instances

I'm not certain what is happening here, I'm thinking it's an inheritance limitation of JEP.

However, the old python interface using Python 2 this is useful for creating GUI interface elements, which to my knowledge works just fine.

https://github.com/pombredanne/findcrypt-ghidra-python/blob/ccf00d3acbe3c31446da8cf92302cf363e96b915/findcrypt.py#L2134-L2142

If you extend the class as is, you get the error TypeError: cannot create 'jep.PyJClass' instances and when you extend it using the exposed __pytype__ you can create the class but you cannot create an instance of it.

Any thoughts on this? is there a workaround perhaps?

User Story: Code Completion / Discoverability

I would like to develop Ghidra API Python scripts with the ability to discover the API through automatic code completion; both in Ghidrathon's REPL, and, preferably, also when using an external IDE. I am extremely enthusiastic about Ghidrathon, but without the ability to leverage code completion / content assist features, I would still chose Java over Python because the integration with Eclipse works well and feels indispensable when dealing with Ghidra's massively nested API.

Notably, code completion in Eclipse is essentially the only appealing feature of Java here. I would vastly prefer to develop in Python due to the rich ecosystem of libraries and the fact that other toolchains (IDA Pro, Binary Ninja) also use Python as their main driver for plugins and scripts.

update documentation to reflect Ghidra 10.2 build requirements

• extension build requires >= Gradle 7.3
• pending #15, update Ghidra requirement >= 10.2
• JDK 17 required (users should have already installed if using Ghidra 10.2)

Crash on consecutive script runs

Bug Description
Over the last few days, Ghidra proceeds to crash entirely when I attempt to run a script for the second time in a row. Once the app is opened again, It proceeds to run the script once, and then crashes on attempt #2. It is regardless of what the script entails or what is being analyzed, something is simple as printing 'hello world' is enough to trigger it. I've tried reinstalling Ghidra to see if it solves the problem and have also tried using an older version to no avail.

To Reproduce

1. Open any script
2. Run the script
3. Run the script again

Once the crash happens, Ghidra closes entirely and a .log crash report is created in my home directory which I've included here.

Host OS: Windows 11
Gurst OS: Ubuntu 22.04
Java version: 17.05
Ghidra version: 10.2.3
Origin: Official github distro

Crash Log

A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x00007fee762a9d78, pid=3814, tid=4824

 JRE version: OpenJDK Runtime Environment (17.0.5+8) (build 17.0.5+8-Ubuntu-2ubuntu122.04)
 Java VM: OpenJDK 64-Bit Server VM (17.0.5+8-Ubuntu-2ubuntu122.04, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
 Problematic frame:
 C  [_multiarray_umath.cpython-310-x86_64-linux-gnu.so+0x2a9d78]  PyArray_Item_INCREF+0x28

 Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /home/happydragon/core.3814)

 If you would like to submit a bug report, please visit:
  Unknown
 The crash happened outside the Java Virtual Machine in native code.
 See problematic frame for where to report the bug.


---------------  S U M M A R Y ------------

Command Line: -Djava.system.class.loader=ghidra.GhidraClassLoader -Dfile.encoding=UTF8 -Duser.country=US -Duser.language=en -Duser.variant= -Dsun.java2d.opengl=false -Djdk.tls.client.protocols=TLSv1.2,TLSv1.3 -Dcpu.core.limit= -Dcpu.core.override= -Dfont.size.override= -Dpython.console.encoding=UTF-8 -Xshare:off --add-opens=java.desktop/sun.awt.image=ALL-UNNAMED -Dsun.java2d.pmoffscreen=false -Dsun.java2d.xrender=true -Dsun.java2d.uiScale=1 -Dawt.useSystemAAFontSettings=on ghidra.Ghidra ghidra.GhidraRun

Host: 11th Gen Intel(R) Core(TM) i7-1195G7 @ 2.90GHz, 4 cores, 7G, Ubuntu 22.04.1 LTS
Time: Fri Mar  3 12:12:57 2023 CST elapsed time: 550.652183 seconds (0d 0h 9m 10s)

Cannot use transformers package

Thanks for your great work on this plugin.

My env:

MacOS 12.5 (x86), Python 3.11, Ghidrathon master branch, Ghidra 10.4

Background

Recently I'm trying to write a script like Gepetto to analyze bytecodes with llm. The langchain pacakge seems work well with Ghidrathon. But I want to use a local llm (after all, it's free, lol). So I played with Hugging Face and try to use transformers in Ghidrathon.

Problem

But I cannot import transformers, just try this in Ghidrathon:

from transformers import AutoTokenizer

and I got a CPython error

File "/usr/local/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/transformers/data/__init__.py", line 26, in <module>
    from .metrics import glue_compute_metrics, xnli_compute_metrics
  File "/usr/local/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/transformers/data/metrics/__init__.py", line 19, in <module>
    from scipy.stats import pearsonr, spearmanr
  File "/usr/local/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/scipy/stats/__init__.py", line 608, in <module>
    from ._stats_py import *
  File "/usr/local/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/scipy/stats/_stats_py.py", line 37, in <module>
    from numpy.testing import suppress_warnings
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/usr/local/Cellar/[email protected]/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/jep/shared_modules_hook.py", line 134, in exec_module
    self.sharedImporter.sharedImport(fullname)
TypeError: CalledProcessError.__init__() missing 1 required positional argument: 'cmd'

I guess it was due to the lazy module loading in transformers pakage. And I don't know if I need to place some package under PYTHON_SHARED_MODULES or PYTHON_INCLUDE_PATHS. Or if this related to a known limitation of JEP.

clearListing Exception

I use clearListing API will raise an Exception, but it work normal in jpython

>> clearListing(toAddr(0))
Traceback (most recent call last):
  File "ghidra_10.3.1_PUBLIC/Ghidra/Extensions/Ghidrathon-2.2.1/data/python/jepeval.py", line 66, in jepeval
    more_input_needed = _jepeval(line)
  File "ghidra_10.3.1_PUBLIC/Ghidra/Extensions/Ghidrathon-2.2.1/data/python/jepeval.py", line 49, in _jepeval
    exec(compile(line, "<string>", "single"), globals(), globals())
  File "<string>", line 1, in <module>
RuntimeError: ghidra.framework.model.DomainObjectException caused by: ghidra.util.exception.ClosedException: File is closed

explore alternatives to cherry-picking Jep files

original comment: #4 (comment)
follow-up: #4 (comment)

ux: inform user when Python interpreter is busy

A long running Python script/command holds the GIL preventing other Python code from executing in the same process until the script/command finishes. Inform user that the interpreter is busy when this happens.

Building Ghidraton on ci

Trying to avoid setting up gradle on my system,
I created a GitHub workflow that builds Ghidraton on ci.

CI result: https://github.com/madebr/Ghidrathon/actions/runs/5313362030

The resulting archive installs fine, but does not run.
There are no logs shown anywhere.

I run Fedora Linux, so I installed python3-jep.

Is there something I am missing?

Also, I can't disable the original Jython plugin in File -> Configure -> Ghidra Core.
There is no item with python there.

Could not build extension for Ghidra 10.2.2

Having an issue building for 10.2.2.

I did notice this similar closed ticket: #18. However, I tried the same build command below for 10.2.1 and 10.2 and received the same error in both cases. Am I missing something?

Thanks for any help.

C:\Tools\Ghidrathon\Ghidrathon-2.0.0>gradle -PGHIDRA_INSTALL_DIR="C:\Program Files (x86)\ghidra_10.2.2_PUBLIC"

> Task :installJep
Requirement already satisfied: jep in c:\users\analyst\appdata\local\programs\python\python39\lib\site-packages (4.0.3)

> Task :copyJepNativeBinaries
2023-01-06 17:37:51,049 INFO     Searching for Jep Python module directory
2023-01-06 17:37:51,049 INFO     Found Jep Python module directory at C:\Users\analyst\AppData\Local\Programs\Python\Python39\lib\site-packages\jep
2023-01-06 17:37:51,049 INFO     Copying jep.dll and C:\Users\analyst\AppData\Local\Programs\Python\Python39\lib\site-packages\jep\jep-4.0.3.jar to extension folders
2023-01-06 17:37:51,049 INFO     Done

> Task :compileJava FAILED
C:\Tools\Ghidrathon\Ghidrathon-2.0.0\src\main\java\ghidrathon\GhidrathonScriptProvider.java:19: error: cannot find symbol
import ghidra.app.script.GhidraScriptLoadException;
                        ^
  symbol:   class GhidraScriptLoadException
  location: package ghidra.app.script
C:\Tools\Ghidrathon\Ghidrathon-2.0.0\src\main\java\ghidrathon\GhidrathonScriptProvider.java:39: error: cannot find symbol
                        throws GhidraScriptLoadException {
                               ^
  symbol:   class GhidraScriptLoadException
  location: class GhidrathonScriptProvider
C:\Tools\Ghidrathon\Ghidrathon-2.0.0\src\main\java\ghidrathon\GhidrathonPlugin.java:53: error: no suitable constructor found for ProgramPlugin(PluginTool)
                super(tool);
                ^
    constructor ProgramPlugin.ProgramPlugin(PluginTool,boolean,boolean,boolean) is not applicable
      (actual and formal argument lists differ in length)
    constructor ProgramPlugin.ProgramPlugin(PluginTool,boolean,boolean) is not applicable
      (actual and formal argument lists differ in length)
C:\Tools\Ghidrathon\Ghidrathon-2.0.0\src\main\java\ghidrathon\GhidrathonScriptProvider.java:48: error: cannot find symbol
                        throw new GhidraScriptLoadException("Unable to instantiate: " + e.getMessage(), e);
                                  ^
  symbol:   class GhidraScriptLoadException
  location: class GhidrathonScriptProvider
4 errors

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s
3 actionable tasks: 3 executed

doc: no support for CPython faulthandler module

The CPython faulthandler module installs fault handlers for the SIGSEGV, SIGFPE, SIGABRT, SIGBUS, and SIGILL signals. Unfortunately, our experience shows that this level of OS/process manipulation destabilizes the Java process that is running Ghidra + Ghidrathon, often resulting in a crash (SIGSEGV). For the time being, we need to document that Ghidrathon does not support the faulthandler module (and likely never will). Python modules that use the faulthandler module, e.g. pytest, should be configured to disable it, if possible. If the faulthandler module cannot be disabled then Ghidrathon does not support the Python module(s) that use it. We should also note that, in general, Ghidrathon's behavior for this level of OS/process manipulation is undefined, at best.

Known Python module using faulthandler:

Module	How to disable?
pytest	use -p no:faulthandler at invocation; see here

expose current `GhidraScript` instance to Python

ghidra.util.Msg has some useful "popup" functions that I wanted to use, specifically the ones that take a Java Throwable for displaying Java exception details; for example:

public static void showError​(java.lang.Object originator, java.awt.Component parent, java.lang.String title, java.lang.Object message)

That first argument, originator, is supposed to be the Java thing that is displaying the message; typical use would be Msg.showError(this, ...) in Java. I'd like to do the same in Ghidrathon, but there is no easy way I can see to access the current GhidraScript derivative.

I can get the current script by calling jepwrappers.get_script(), which returns the ghidrathon.GhidrathonScript instance, if I want, but it isn't exposed any other way that I can see.

I'd suggest adding getCurrentScript() or something very similar as a "public" version of that to the globals of the python script environment?

Ghidraton doesnt compil against ghidra master since commit 9235902

the script error handling api has been slightly modified in ghidra and Ghidraton is not compatible atm, i think a PR is a bit early at this point so i m posting this small diff that can be applied to fix compilation :

diff --git a/src/main/java/ghidrathon/GhidrathonScriptProvider.java b/src/main/java/ghidrathon/GhidrathonScriptProvider.java
index f72d4f4..d49c1dc 100644
--- a/src/main/java/ghidrathon/GhidrathonScriptProvider.java
+++ b/src/main/java/ghidrathon/GhidrathonScriptProvider.java
@@ -16,6 +16,7 @@ import generic.jar.ResourceFile;
 
 import ghidra.app.script.GhidraScript;
 import ghidra.app.script.GhidraScriptProvider;
+import ghidra.app.script.GhidraScriptLoadException;
 
 public class GhidrathonScriptProvider extends GhidraScriptProvider {
 
@@ -35,15 +36,24 @@ public class GhidrathonScriptProvider extends GhidraScriptProvider {
 
 	@Override
 	public GhidraScript getScriptInstance(ResourceFile sourceFile, PrintWriter writer)
-			throws ClassNotFoundException, InstantiationException, IllegalAccessException {
-
-		Class<?> clazz = Class.forName(GhidrathonScript.class.getName());
-		GhidraScript script = (GhidraScript) clazz.newInstance();
-		
-		script.setSourceFile(sourceFile);
-		
-		return script;
+			throws GhidraScriptLoadException {
+		try{
+			Class<?> clazz = Class.forName(GhidrathonScript.class.getName());
+			GhidraScript script = (GhidraScript) clazz.newInstance();
+			script.setSourceFile(sourceFile);
 		
+			return script;
+		}
+		catch (InstantiationException e) {
+			throw new GhidraScriptLoadException("Unable to instantiate: " + e.getMessage(), e);
+		}
+		catch (ClassNotFoundException e) {
+			throw new GhidraScriptLoadException("Unable to locate class: " + e.getMessage(), e);
+		}
+		catch (IllegalAccessException e) {
+			throw new GhidraScriptLoadException(
+				"The class or its default constructor is not accessible: " + e.getMessage(), e);
+ 		}
 	}
 
 	@Override

Drop into interactive mode from analyzeHeadless

Hi,

First of all, thanks for this project!

I was wondering if dropping into pdb/ipdb, or a REPL from Ghidra's headless mode would be possible using Ghidrathon.

Just naively trying ipdb results in:

[...snip...]
INFO  SCRIPT: /src/checker/ghidra.py (HeadlessAnalyzer)
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/jep/java_import_hook.py", line 45, in __getattr__
    return super(module, self).__getattribute__(name)
AttributeError: module 'pdb' has no attribute 'Pdb'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.ghidra/.ghidra_10.2.3_PUBLIC/Extensions/ghidrathon/data/python/jeprunscript.py", line 31, in jep_runscript
    exec(compile(source, path, "exec"), {**globals(), **additional_globals})
  File "/src/checker/ghidra.py", line 8, in <module>
    from IPython import embed
  File "/usr/lib/python3/dist-packages/IPython/__init__.py", line 55, in <module>
    from .core.application import Application
  File "/usr/lib/python3/dist-packages/IPython/core/application.py", line 25, in <module>
    from IPython.core import release, crashhandler
  File "/usr/lib/python3/dist-packages/IPython/core/crashhandler.py", line 27, in <module>
    from IPython.core import ultratb
  File "/usr/lib/python3/dist-packages/IPython/core/ultratb.py", line 112, in <module>
    from IPython.core import debugger
  File "/usr/lib/python3/dist-packages/IPython/core/debugger.py", line 123, in <module>
    from pdb import Pdb as OldPdb
  File "/usr/local/lib/python3.10/dist-packages/jep/java_import_hook.py", line 57, in __getattr__
    clazz = forName('{0}.{1}'.format(self.__name__, name))
ImportError: java.lang.ClassNotFoundException: pdb.Pdb
[...snip...]

Using

import code
code.interact(local={**globals(), **locals()})

results in a prompt, but the code is only interpreted when I leave the interactive session (e.g., CTRL+D):

[...snip...]
INFO  SCRIPT: /src/checker/ghidra.py (HeadlessAnalyzer)
>>> print("foobar")
>>> Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

now exiting InteractiveConsole...
foobar
[...snip...]

Is there any way to do this?

numpy usage causes Ghidra to crash

This is likely a result of Ghidrathon's use of Jep's SubInterpreter v. SharedInterpreter (see Jep discussion here).

We could:

1. change Ghidrathon to use Jep's SharedInterpreter
2. continue using SubInterpreter but include numpy in the list of shared modules in the JepConfig, as suggested by Jep developers

Additional discussion from Jep developers on which interpreter to use:

Use SharedInterpreters if
- you are brand new to Jep and are not sure. It is easier to go from using SharedInterpreters to SubInterpreters than the other way around.
- you want to use a lot of CPython extension modules
Use SubInterpreters if
- you used Jep instances in Jep 3.x and want to retain the same behavior
- you need your interpreters as isolated as possible

close interpreter window when sys.exit called

This behavior would more closely match that of the standard Python interpreter.

Virtualenv under Windows

I followed the installation instructions from the readme and would like to use a virtual environment. When activating that venv (with /path/to/venv/Scripts/activate.bat), and executing ghidraRun.bat, Ghidra shows the following exception when I try to open the Ghidrathon window:

java.lang.RuntimeException: jep.JepException: <class 'ModuleNotFoundError'>: No module named 'jep'
	at ghidrathon.interpreter.GhidrathonInterpreter.get(GhidrathonInterpreter.java:215)
	at ghidrathon.GhidrathonConsoleInputThread.run(GhidrathonConsoleInputThread.java:59)
Caused by: jep.JepException: <class 'ModuleNotFoundError'>: No module named 'jep'
	at <string>.<module>(<string>:1)
	at jep.Jep.eval(Native Method)
	at jep.Jep.eval(Jep.java:312)
	at jep.Jep.configureInterpreter(Jep.java:179)
	at jep.Jep.<init>(Jep.java:159)
	at jep.Jep.<init>(Jep.java:128)
	at jep.SubInterpreter.<init>(SubInterpreter.java:63)
	at ghidrathon.interpreter.GhidrathonInterpreter.<init>(GhidrathonInterpreter.java:74)
	at ghidrathon.interpreter.GhidrathonInterpreter.get(GhidrathonInterpreter.java:210)
	... 1 more

Operating System: Windows 10
Java Version: OpenJDK 11.0.16
Ghidra Version: 10.1.5
Python Version: 3.9.1
Jep Version: 4.0.3

If I don't use a venv but just install jep into the system-wide Python installation, Ghidrathon works as expected. So I assume it has something to do with ghidraRun.bat not correctly passing the environment variables from the currently active venv?

ghidra exits abnormally？Is it a bug of Ghidrathon Or Jep?

Here is the log.

hs_err_pid121091.log

ClosedException is triggered when accessing AddressRangeMapDB in headless mode.

Environment Info

• OS: Linux (Debian 11)
• Python version: 3.10.12
• Ghidra version: 10.3
• Ghidrathon version: 2.2.1
• openjdk version 17.0.7" 2023-04-18

Setup Info

I have a python script running as a postScript to Ghidra running in headless mode.

The script does something like this:

FUNC_NAMES_LIST = ["printf"]

functions = list(currentProgram.getFunctionManager().getFunctions(True))

target_addresses = [
    func.getEntryPoint() for func in functions if func.getName() in FUNC_NAMES_LIST
]

references = itertools.chain.from_iterable(
    getReferencesTo(target_address) for target_address in target_addresses
)

I wanted to port it to Python3 using Ghidrathon but I'm getting ClosedException whenever I try to interact with the AddressRangeMadDB to get x-refs.

I'm getting the following stacktrace from Ghidra:

ERROR Unexpected Exception: File is closed (AddressRangeMapDB) ghidra.util.exception.ClosedException: File is closed
	at db.buffers.BufferMgr.getCachedBufferNode(BufferMgr.java:883)
	at db.buffers.BufferMgr.getBufferNode(BufferMgr.java:792)
	at db.buffers.BufferMgr.getBuffer(BufferMgr.java:956)
	at db.NodeMgr.getFixedKeyNode(NodeMgr.java:292)
	at db.Table.getFieldKeyNode(Table.java:166)
	at db.Table$ShortDurationFieldKeyIterator.initialize(Table.java:3852)
	at db.Table$ShortDurationFieldKeyIterator.<init>(Table.java:3826)
	at db.Table$FieldKeyIterator.<init>(Table.java:3370)
	at db.Table.fieldKeyIterator(Table.java:2005)
	at db.FieldIndexTable$PrimaryKeyIterator.<init>(FieldIndexTable.java:493)
	at db.FieldIndexTable.keyIterator(FieldIndexTable.java:410)
	at db.Table.indexIterator(Table.java:1569)
	at ghidra.program.database.util.AddressRangeMapDB.getAddressSet(AddressRangeMapDB.java:345)
	at ghidra.program.database.symbol.NamespaceManager.getAddressSet(NamespaceManager.java:292)
	at ghidra.program.database.symbol.NamespaceManager.getAddressSet(NamespaceManager.java:270)
	at ghidra.program.database.function.FunctionDB.getBody(FunctionDB.java:345)
	at ghidra.program.database.function.FunctionManagerDB.getFunctionContaining(FunctionManagerDB.java:534)
	at ghidra.program.database.ListingDB.getFunctionContaining(ListingDB.java:454)
	at ghidra.program.flatapi.FlatProgramAPI.getFunctionContaining(FlatProgramAPI.java:1042)
	at jep.Jep.invoke(Native Method)
	at jep.Jep.invoke(Jep.java:259)
	at ghidrathon.interpreter.GhidrathonInterpreter.runScript(GhidrathonInterpreter.java:426)
	at ghidrathon.GhidrathonScript.run(GhidrathonScript.java:41)
	at ghidra.app.script.GhidraScript.executeNormal(GhidraScript.java:397)
	at ghidra.app.script.GhidraScript.doExecute(GhidraScript.java:252)
	at ghidra.app.script.GhidraScript.execute(GhidraScript.java:230)
	at ghidra.app.util.headless.HeadlessAnalyzer.runScript(HeadlessAnalyzer.java:579)
	at ghidra.app.util.headless.HeadlessAnalyzer.runScriptsList(HeadlessAnalyzer.java:912)
	at ghidra.app.util.headless.HeadlessAnalyzer.analyzeProgram(HeadlessAnalyzer.java:1060)
	at ghidra.app.util.headless.HeadlessAnalyzer.processFileWithImport(HeadlessAnalyzer.java:1544)
	at ghidra.app.util.headless.HeadlessAnalyzer.processWithImport(HeadlessAnalyzer.java:1662)
	at ghidra.app.util.headless.HeadlessAnalyzer.processWithImport(HeadlessAnalyzer.java:1709)
	at ghidra.app.util.headless.HeadlessAnalyzer.processWithImport(HeadlessAnalyzer.java:1727)
	at ghidra.app.util.headless.HeadlessAnalyzer.processLocal(HeadlessAnalyzer.java:448)
	at ghidra.app.util.headless.AnalyzeHeadless.launch(AnalyzeHeadless.java:127)
	at ghidra.GhidraLauncher.launch(GhidraLauncher.java:77)
	at ghidra.Ghidra.main(Ghidra.java:54)
ghidra.framework.model.DomainObjectException caused by: ghidra.util.exception.ClosedException: File is closed
Traceback (most recent call last):
--snip--
RuntimeError: ghidra.framework.model.DomainObjectException caused by: ghidra.util.exception.ClosedException: File is closed

No method : VarnodeTranslator

Tried to use

_translator = ghidra.program.model.pcode.VarnodeTranslator()

But it told me that:

    _translator = ghidra.program.model.pcode.VarnodeTranslator()
NameError: No such Method.

Build suggestions

I just installed Ghidrathon-1.0.0 and I ran into 2 very minor installation issues that were trivial to correct, but could potentially make the experience a bit more streamlined if addressed:

1. By default on Ubuntu 22.04, python is not on the path but python3 is. Perhaps you could test for both (probably starting with python3 before failing.
2. Also by default, pip was not installed. The error from Gradle didn't convey this...I had to run the python3 -m pip install jep command manually to figure out what was going on. Perhaps that error messages can be relayed by Gradle somehow.