smvv/pypy-traceview

Static HTML viewer for PyPy trace logs

PyPy traceview

This tool allows you to find out what assembly code is generated by PyPy's JIT. It shows the Python bytecode, RPython IR and its corresponding assembly code.

Example usage

Let's say we have the following simple Python code (stored in file: add.py):

def add(a, b):
    return a + b


def main():
    tmp = 0
    for i in range(100000):
        tmp += add(i, i % 17) % 100000
    return tmp


if __name__ == "__main__":
    main()

What assembly code does PyPy generate for the loop and method add? In order to answer this question, we need to generate a PyPy trace log file:

PYPYLOG=jit:pypy.log pypy3 add.py

Next, we generate a static HTML file that allows us to see what's in the trace log:

PYTHONPATH=. pipenv run ./pypy-traceview pypy.log

The default output file is output.html. You can open this HTML file in a browser, and you will see the following:

The left panel shows one-line Python code snippets and the matching Python bytecode. The middle panel shows RPython IR, and the right panel shows assembly output. Clicking on a Python code snippet will jump to the matching RPython IR.

Other options

Using --help, it is possble to show the customization options:

$ PYTHONPATH=. pipenv run ./pypy-traceview --help
usage: pypy-traceview [-h] [--output OUTPUT] [--profile]
                      [--mnemonics MNEMONICS]
                      FILE

Convert PyPy JIT log file to HTML.

positional arguments:
  FILE                  PyPy JIT log filename

optional arguments:
  -h, --help            show this help message and exit
  --output OUTPUT, -o OUTPUT
                        HTML output filename (default: output.html)
  --profile             Dump CPU profiler info
  --mnemonics MNEMONICS
                        Disassemble using "intel" or "att" (default) syntax

Note: the flag --profile is for internal use. It profiles the tool and dumps a textual representation of where CPU time is spent on stdout.

pypy-traceview's Issues

error: ImportError: cannot import name '__all__'

Hi,

I'm trying to use pypy-traceview. On Python 3.6, I get:

Traceback (most recent call last):
  File "./pypy-traceview", line 12, in <module>
    from pypy_traceview.renderer.html import render
  File "/home/staff/gback/python/pypy-traceview/pypy_traceview/renderer/html.py", line 2, in <module>
    from jinja2 import Template
  File "/home/staff/gback/python/venv/lib/python3.6/site-packages/jinja2/__init__.py", line 8, in <module>
    from .environment import Environment as Environment
  File "/home/staff/gback/python/venv/lib/python3.6/site-packages/jinja2/environment.py", line 17, in <module>
    from . import nodes
  File "/home/staff/gback/python/venv/lib/python3.6/site-packages/jinja2/nodes.py", line 5, in <module>
    import inspect
  File "/usr/lib64/python3.6/inspect.py", line 36, in <module>
    import dis
  File "/usr/lib64/python3.6/dis.py", line 9, in <module>
    from opcode import __all__ as _opcodes_all
ImportError: cannot import name '__all__'

I get the same error on Python 3.10:

$ PYTHONPATH=. ../venv/bin/pipenv run ./pypy-traceview ../logfile 
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Traceback (most recent call last):
  File "/home/gback/icpc/icpc-fall-2022/pickupsticks/pypy-traceview/./pypy-traceview", line 4, in <module>
    import pstats
  File "/usr/lib/python3.10/pstats.py", line 31, in <module>
    from dataclasses import dataclass
  File "/usr/lib/python3.10/dataclasses.py", line 5, in <module>
    import inspect
  File "/usr/lib/python3.10/inspect.py", line 38, in <module>
    import dis
  File "/usr/lib/python3.10/dis.py", line 9, in <module>
    from opcode import __all__ as _opcodes_all
ImportError: cannot import name '__all__' from 'opcode' (/home/gback/icpc/icpc-fall-2022/pickupsticks/pypy-traceview/pypy_traceview/opcode.py)

It seems that you are importing dis which in turn imports opcode but due to the PYTHONPATH setting picks up your version instead of the system version. I note that /usr/lib/python3.10/opcode.py has an __all__ export.