Giter Site home page Giter Site logo

a1ext / labeless Goto Github PK

View Code? Open in Web Editor NEW
500.0 42.0 81.0 130.45 MB

Labeless is a multipurpose IDA Pro plugin system for labels/comments synchronization with a debugger backend, with complex memory dumping and interactive Python scripting capabilities.

License: Other

Batchfile 0.07% C 40.02% Python 23.91% C++ 35.85% QMake 0.04% SWIG 0.12%
ida memorydumping python-script x64dbg ollydbg labeless x86 x86-64 x64 x64dbg-plugin

labeless's Introduction

Contributed By Check Point Software Technologies LTD.

PyPI Build status Total Downloads Telegram Channel Twitter URL Donate to this project using Patreon

Features

1. Seamless synchronization of labels, function names, comments and global variables (w/wo demangling)

  • Synchronization modes
    • On demand
    • On rename (update on-the-fly)
  • Supports image base-independent synchronization

2. Dynamic dumping of debugged process memory regions

It can be useful in the following cases:

  • When debugged process has extracted/temporary/injected module which doesn't appear in modules list
  • When it doesn't have a valid PE header
  • When it has corrupted import table, etc.
  • When it contains unpacked memory regions inside a binary, you can easily merge these new memory regions with the ones that are already present in your database
  • When reconstructing chain of memory chunks which are used by malware (and not only, if you know what we mean) so that the picture of its behaviour is complete

3. Python scripting

We support the following list of debug backends for now:

Overview

Labeless is a multipurpose IDA Pro plugin system for labels/comments synchronization with a debug backend, with complex memory dumping and interactive Python scripting capabilities. It consists of two parts: IDA plugin and debug backend's plugin.

Labeless significantly reduces time that researcher spends on transferring already reversed/documented code information from IDA (static) to debugger (dynamic). It saves time, preventing from doing the same job twice. Also, you can document and add data to the IDB on the fly and your changes will be automatically propagated to debug backend, even if you will restart the virtual machine or instance of debug backend will crash. So, you will never lose your research.

This solution is highly upgradable. You can implement any helper scripts in Python on debug backend's side and then just call them from IDA with one line of code, parsing the results and automatically propagating changes to IDB.

We can take that memory region and put it in the IDB, fixing imports 'on-the-fly', using debug backend's functionality. No more need in ImpRec or BinScylla, searching for the regions in memory that contain the real IAT, because we get that information dynamically from the debugged process itself.

As a result we have a lot of memory regions that may represent even different modules (if the unpacking process if multistage) with valid references between them, which gives us a possibility to build a full control flow graph of the executable. Basically, we will end up with one big IDB, containing all the info on the specific case.

Virus Bulletin 2015

Videos

Installation

Usage of precompiled binaries (release version)

Debug-backend setup:

If you want to use both x86 and x86_64 targets, then you should do the following steps for each python distro.

  • Set up Python 2.7 (x86/x86_64)
  • Copy deploy directory to target machine, where you want to use a debugger backend
  • Set up protobuf 2.6.1 using the following commands:
cd c:\deploy
c:\Python27\python.exe setup_protobuf.py
  • Install labeless python module, there are two ways to archive that, the first one is to use PyPI in case you have an Internet connection on the debug machine:
pip install --upgrade labeless

In case you don't have an Internet connection, you could install prebuilt module from release archive:

cd c:\deploy
c:\Python27\Scripts\pip.exe install labeless-1.1.2.65-py2.py3-none-any.whl

Note: If you have already used Labeless before and you want to update it, don't forget to reinstall python module each time you have new release

  • Configure your debugger backend: set up plugins directory

Configuring of your IDA PRO:

Labeless supports Windows and Linux (starting from 6.9.5 version of) IDA PRO. Labeless handles only PE/AMD64 binaries. Labeless requires IDAPython plugin python.[plw|p64|plx|plx64] (it ships with IDA PRO, but make sure it works well).

There are plugins:

IDA[XX]\plugins\labeless_ida.plw - for IDA for Windows, handles 32-bit binaries (used with idaq)
IDA[XX]\plugins\labeless_ida.p64 - for IDA for Windows, handles 64-bit binaries (used with idaq64)
IDA[XX]\plugins\labeless_ida.plx - for IDA for Linux, handles 32-bit binaries (used with idaq)
IDA[XX]\plugins\labeless_ida.plx64 - for IDA for Linux, handles 64-bit binaries (used with idaq64)

Copy Labeless plugins to your IDA's plugins directory, for example c:\IDA68\plugins

  • In case you have IDA for Windows, please, use .plw/.p64 plugins.

  • If you have IDA for Linux, please, use .plx/.plx64 plugins. Also, copy IDA[XX]/libprotobuf.so.9 to your IDA home directory (for example /home/alex/ida695/), it's an important library.

Configuring of debug backends

1. OllyDbg 1.10

You may find prepared debugger in the following directory OllyDbg110. (Note!: Don't forget to set up debugger's plugins directory).

2. DeFixed 1.10 (FOFF's team mod)

Copy DeFixed110\plugins\labeless_olly_foff.dll to DeFixed plugins directory (Note!: Don't forget to set up debugger's plugins directory)

3. OllyDbg 2.01

You may find prepared debugger in the following directory OllyDbg201. (Note!: Don't forget to set up debugger's plugins directory).

4. x64dbg (x32,x64)

You may find prepared debugger in the following directory x64dbg.

Checking if everything works

  • Start debug backend (debugger) and check for Labeless item presence in the Plugins menu. If there is any problem, then check Olly's log window for details. Open the log window and check for LL: ok, binded at <IP>:<PORT> message, its presence means that debug backend-side plugin is initialized successfully. Note: if you start many debuggers, you may see that the following message appears

port_auto_select.png

Also, you may see the firewall alert

allow_firewall.png

If you want to access the debug backend from another computer, you should allow the backend to listen by this dialog or manually.

  • Start working with existing IDA database or use Labeless -> Load stub database... from the menu
  • Open Labeless settings dialog using any of the following actions:
    • menu Edit -> Plugins -> Labeless
    • main menu Labeless -> Settings...
    • hotkey Alt+Shift+E
  • Enter IP address and port of the guest machine (where debug backend is set up), then click on 'Test connection' button
  • If IDA displays the message Successfully connected!, then configuration is done correctly.

Development

  • Set up Python 2.7 (x86/x86_64)
  • protobuf 2.6.1
  • Visual Studio 2010 + Qt 4.8.4 (built with "QT" namespace) - required by IDA's 6.8 plugin (to proper use IDA's Qt). You can configure Qt by yourself with the following command:

configure -platform win32-msvc2010 -shared -release -no-webkit -opensource -no-qt3support -no-phonon -no-phonon-backend -opengl desktop -nomake demos -nomake examples -nomake tools -no-script -no-scripttools -no-declarative -qtnamespace QT

How to use

  • If you want to enable synchronization of labels (names) and comments from IDA to Olly you should check 'Enable labels & comments sync' in Labeless settings dialog in IDA. There is one required field called 'Remote module base', which should be set to the current module base of the analyzed application. You can find out that information in the debugger
  • Select needed features, like Demangle name, Local labels, Non-code names
  • Select comments synchronization type:
    • <Disabled>
    • Non-repeatable
    • Repeatable
    • All

Repeatable - are comments, which IDA shows in any referenced place.

  • If you want to sync labels right now - press 'Sync now' button. Labeless will sync all found names in your IDB with Olly. Settings dialog will be automatically closed, while saving all settings
  • Also, you may use Labeless -> Sync labels now from IDA's main menu
  • If you want to customize settings for IDADump engine, do it in the 'IDADump' tab.
  • To save changed settings, click on 'Save & Close'

Things automatically performed in the background

  • If you enabled 'Enable labels & comments sync' option, then Labeless will automatically synchronize all the data on any rename operation in IDA

Troubleshooting

Issue with Python 2.7.11 is described here, so avoid usage of this version. The latest stable supported version is 2.7.10.

  • Q: Labeless for x64dbg x32 is works, but x64 doesn't. Why?
  • A: Please, recheck that you have installed protobuf and 'labeless' module for Python x64

Download

Credits

labeless's People

Contributors

a1ext avatar gitter-badger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

labeless's Issues

Ubuntu 14.04 compatibily

Hi guys!

Even though I have installed an updated GCC version from the one available for Ubuntu 14.04.5 using the PPA ubuntu-toolchain-r/test, I'm still getting this error:
dlopen(/opt/ida-6.95/plugins/labeless_ida_690.plx): /opt/ida-6.95/plugins/labeless_ida_690.plx: symbol _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERjj, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

I guess this is compilation/ABI-related, so not sure if there is an easy solution for this besides me upgrading to a newer Linux distro?

Thank you in advance.
Best.

IDA 7.0 x64 support

Now with the new IDA finally being x64 bit could we hope for a release that has this top plugin will support the new version?

Binary File Sync

Is it possible to add sync support for Bynary files like Shellcodes? That would be useful when debugging them without the worry to convert them to executables.
image

x64 plugin for x64dbg is not loading

Latest release will not load up the labeless plugin in x64dbg debugger. This goes for the the x64 only. The 32 bit version is working as it should.

nice project!

looks cool, but the project description is still " No description or website provided. "?

cheers.

WinDbg support

It will be great if you will add support for WinDbg debugger.

The IDA7.0 plug-in test version, error

@a1ext
Debug 32 bit programs in 64-bit IDA7.0(ida.exe):
Just tested, ida7.0 prompts the connection to fail, but does not affect the use.
It's a strange question, and I almost give up using it

new problem:
Debug 64 bit programs in 64-bit IDA7.0(ida64.exe):
prompts connection failure,Click hand sync.
No response,The program card to death

Debug 32 bit programs in 64-bit IDA7.0(ida64.exe):
prompts connection failure,Click hand sync.
Log prompt Received data, but not displayed

ida7 x64dbg sync failing

The environment:
win10x64
labeless1.1.2.65 - release binary
ida7.1
x64dbg - Jul 1 2018

The main issue is nothing is syncing properly. I don't see any specific errors when running the sync manually, i just don't see anything flowing over to x64dbg.

Here's an example of what i see in the output window in ida:
Labeless: do sync all now... Labels: 142, comments: 3 RPC Thread: OK, tasks left: 0
in x64dbg log window
LL: Labeless::onClientSockAccept: Connected from: 127.0.0.1:51869 LL: socket 00000848 accepted LL: Labeless::onClientSockRead: Received 8 bytes of 8. LL: Labeless::onClientSockRead: Received 89 bytes of 89. LL: Labeless::onClientSockBufferReceived: new request pushed { jobId: 9, bkg: 0, finished: 0 } LL: Labeless::serverThread: jobId 9 socket 00000848 Response sent, len: 0x0000000A LL: Labeless::serverThread: jobId 9 is removed

When running the "test connection" button inside of the ida plugin, i get: "Test failed, error: Invalid version response:"

I've setup and installed this for both x64 and x86 ida/x64dbg and i get the same result in both setups.

One other thing to note. The 1.1.2.65 release code doesn't have the setup.py in the deploy directory so i just pulled that from master to run the python setup.py install.

Labeless not showing in x64DBG

Hi, first of all thanks for the great plugin. I tried to use x64DBG butI noticed that labeless plugin is not showing in the plugin toolbar. I used the x64DBG included in your package but it doesn't work.
OS: Windows 7 SP1

"On rename" bug

With turned on "Enable labels & comment sync", when user dumps new region IDA spams with rename notifications. Background sync of labels "on rename" should be inactive when IDADump working.

IDADump randomly fail to read memory regions

#IDADump issue

Environment

labeless v1.1.4.0, 64 bit x64dbg, python2,

IDA-side information

in PyOlly stderr logs, sometimes, VirtualQueryEx failed with error 998, however the memory regoin was surely accessable on x64dbg.

Problem Description

Describe the problem/bug as clearly as possible.

Steps to Reproduce

1.try IDAdump by regions, then check out stderr log , then compare the dumped result in IDA hex view to the x64dbg memory view.

Possible Solutions

I think the problem could be a bug in python2 ctypes (python3 seems no such issues), however we could handle it carefully.
I found two APIs were referred,
VirtualProtectEx and VirtualQueryEx, both there are SIZE_T in prototype.
see

g['ok'] = VirtualProtectEx(h_process, C.c_void_p(g['ea'] + off), GRANULARITY,

and
if VirtualQueryEx(h_process, p_addr, C.byref(mbi), C.sizeof(mbi)):

example, look at C.sizeof(mbi), gave a 32 bits int 0x30, but should be a SIZE_T in prototype which is 64 bits on x64,
or, ctypes will finally call on VirtualQueryEx with a undefined high 32bits value like 0x????????00000030,
after changing to C.c_size_t(C.sizeof(mbi)) and C.c_size_t(GRANULARITY) , IDADump works perfectly.

Structures support?

Is it supported by debuggers, is it possible to use/sync structs in olly, x64dbg, etc.?

problem

ida 6.8:
Labeless::loadImportTable: storage::loadExternSegData() failed

Olly part doesn't show up in the plugin menu

First of all, I appreciated for your work!

I download the link on the bottom of the Github page and try to install it.
But the plugin doesn't show up in the plugin menu.
I also checked the log, but nothing showed up.

I've tried ollydbg 108b, 110 and 201 but all of them don't work either.

So here comes my questions:

  1. What version of ollydbg is compatible with Labeless?
  2. If I only want to use (instead of build) the plugin, is Visual Studio 2012 (or newer) necessary?

IDA/debugger scripts history keeping

Environment

IDA-side information

Question Answer
IDA-side OS version Win 7 x64
IDA PRO bitness (not IDB/I64) both
IDA plugin bitness both
IDA plugin version 1.1.2.85

Debugger-side information

Not related

Problem Description

Labeless scripting in IDA and in Olly lacks of Python script history. It would be useful if you occasionally close PyOlly tab or IDA crashes.

Labeless for linux missing

When I download the release I cannot find the labeless_ida.plx and labeless_ida.plx64 files. Where have they gone?

Labeless just works with x32dbg but not x64dbg

Labeless in the latest release "[x64dbg] SDK and testing bundle are updated to snapshot on 2016-09-18_01-22"(labeless_release_full_1.1.0.9.7z) just works with x32dbg but not x64dbg.

screenshot - 9_28_2016 5_26_30 pm
screenshot - 9_28_2016 5_28_35 pm

problem

x64dbg x64:
[PLUGIN] Failed to load plugin: labeless_x64dbg.dp64

x64dbg x32:

LL: Labeless::logInitPythonFail: "import labeless as ll" failed.
LL: pluginit: Labeless::initPython() failed
[PLUGIN] pluginit failed for plugin: labeless_x64dbg.dp32

labeless vesion:

labeless_release_full_1.1.0.7.7z

x64dbg vesion :

snapshot_2016-09-01_07-38.7z

Automatic update of the "Remote module base" when changed in the debugger

Currently, whenever the debuggee is restarted in the debugger and changes its base address, we need to manually update Labeless from IDA with the new base address.

image

I suggest, if possible, that Labeles will get triggered when the debuggee is restarted and changes its address and will update the "Remote module base" automatically.

Linux version?

Hi,

Great work, I love the plugin. I was wondering if you were thinking about support the Linux version of IDA? Since you have remote debugging, I assume this is possible?

Can't asyn struct member variable name?

Hello!
My Enviroment: Windows10, IDA7.0, x64dbg 2018.5.17

x64dbg:

004017D0 |  | mov ecx,dword ptr ss:[esp+10]                | ecx:start
004017D4 |  | mov eax,dword ptr ss:[esp+4]                 |
004017D8 |  | mov dword ptr ds:[4040B4],ecx                | ecx:start
004017DE |  | mov ecx,<c2dcad_1.g_obj3>                    | ecx:start
004017E3 |  | mov dword ptr ds:[4040AC],eax                |
004017E8 |  | call <c2dcad_1.CObj3_MyCreateWindow>         |
004017ED |  | mov ecx,<c2dcad_1.g_obj3>                    | ecx:start
004017F2 |  | call <c2dcad_1.CObj3_MyMessageLoop>          |
004017F7 |  | ret 10                                       |

DA7.0:

.text:004017D0                 mov     ecx, [esp+nShowCmd]
.text:004017D4                 mov     eax, [esp+hInstance]
.text:004017D8                 mov     g_obj3.m_nShowCmd, ecx
.text:004017DE                 mov     ecx, offset g_obj3
.text:004017E3                 mov     g_obj3.m_hInstance, eax
.text:004017E8                 call    CObj3_MyCreateWindow
.text:004017ED                 mov     ecx, offset g_obj3
.text:004017F2                 call    CObj3_MyMessageLoop
.text:004017F7                 retn    10h
.text:004017F7 _WinMain@16     endp

Is a method asyn g_obj3.m_nShowCmd to 4040B4 at 004017D8?
Thanks!

cannot work with BreakModule

I've setup protobuf & labeless with python 2.7 32bit.
Then I copied the corresponding plugin folder to ollydbg and x64dbg
I can safely use ollydbg with labeless as well as the given x64dbg.
But whenever I run my own x64dbg(32bit) it just stuck and the crash.
Deleting labeless_x64dbg.dp32 solves that problem but obviously we cannot do that as a workaround.
So any ideas? I would appreciate every help.
Thank you

Label synced from IDA makes bp command doesn't work with the label name

Hi, I see a problem when I try set a breakpoint with bp command in OllyDbg with the Command Bar plugin, if the labels are synced from IDA, I cannot set breakpoints with labels synced. I must edit label (:) to remove the arguments in the label to be able to set the breakpoint at the label. I think you could give an option to sync only the label name without the arguments in the labels :) .

screenshot - 7_4_2016 6_07_53 pm

screenshot - 7_4_2016 6_08_54 pm

screenshot - 7_4_2016 6_09_30 pm

screenshot - 7_4_2016 6_13_12 pm

Is it possible to run labeless on IDA PRO demo?

Does this tool work with IDA Pro demo version? I can see that after the build phase there are no IDA plugins files to copy (IDA[XX]\plugins\labeless_ida.plw, IDA[XX]\plugins\labeless_ida.p64) but I was rather thinking whether it is possible to make this work with the demo version of full one is required?

IDA7 Linux plugin

Hi,
In the docs it's mentioned that Linux is supported but couldn't find any Linux IDA plugins in release files or notes on how to build the plugin on Linux?
Is the Linux IDA supported with the current version and could you provide some notes on how to build the plugin?

Thanks!

update to the latest x64dbg

The latest x64dbg has a working update checker again. Also, there should be no breaking API changes (ever) 👍

Thread EIPs as start points

It will be great, if you will add every thread EIP as start_xxx points, so it will be possible to see what every thread does.

load plugin fails.

LL: Labeless::logInitPythonFail: "import labeless as ll" failed.
LL: pluginit: Labeless::initPython() failed
[插件] 插件函数pluginit失败: labeless_x64dbg.dp32

load fails, how does it work? none-English character not support?

Automatic following EA x64dbg - IDA; not a feature yet?

Hi, I've been trying to get the same experience as the video on the "Wiki" page, with x64dbg and IDAPro. I haven't been able to get IDA to jump automatically to the x64dbg EA, but I can get it to jump manually (through the menu).
It might be worth mentioning that it's across VM's, and all of the other features are present.

And many thanks for a wonderful plugin, it's going to help me a lot in the future!

Standalone distro

For now installation requires protobuf downloading, but not every vm has internet connection/vm tools to download it.

Wrong dumped segment selector set

Environment

IDA-side information

Question Answer
IDA-side OS version Win 7 x64
IDA PRO bitness (not IDB/I64) 64
IDA plugin bitness 32
IDA plugin version 1.1.2.97

Debugger-side information

Doesn't matter

Problem Description

After dumping part of dynamically resolved API table, code references get broken:

image

Steps to Reproduce

  1. Open Trickbot sample
  2. trace it in debugger until APIs are resolved
  3. Perform Keep and import dumping of resolved APIs memory region (416CAC - 416F8C):
    image

Possible Solutions

The following script fixes the problem:

seg = idaapi.getseg(0x416CAC)
seg.sel = idaapi.setup_selector(2)
seg.update()

image

Function local vars

Hi, I recently update labeless to the latest version and see that the local vars of functions are not synced from IDA to OllyDbg like the previous versions of labeless. I think it's more convenient to sync all function local vars (not only the structs only ones).

screenshot - 10_1_2016 9_48_38 am

Failed to load plugin for x64dbg (32-bit only) on 1.1.0.7 w/Python 2.7

After following the instructions on the target machine with the latest version (1.1.0.7), the 32-bit version of x64dbg does not load the plug-in properly. Meanwhile, the 64-bit version has no issue.

You can see both the log screen output for both versions here (32-bit on left, 64-bit on right):
image

I had this same issue with a previous version attempted earlier (1.1.0.3 specifically).

I've been struggling with trying to get this working to no avail.

I am running on Python 2.7 within a Windows 7 VM.

Thank you for your work on this great set of plugins - I appreciate the time and effort you have put into this valuable project.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.