Official Debian package

[alexmyczko]

Official Debian package

[pablogsal]

Hi @alexmyczko and thanks for opening the issue. Looking at the build failures I am seeing the following:

x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -g -fwrapv -O2 -g -O2 -ffile-prefix-map=/var/www/debian/memray/memray-1.11.0=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -D_FILE_OFFSET_BITS=64 -DUSE_MEMRAY_TLS_MODEL=1 -Isrc/memray/_memray -I/usr/include/x86_64-linux-gnu -Isrc -I/var/www/debian/memray/memray-1.11.0/src/vendor/libbacktrace/install/include -I/usr/include/python3.11 -c src/memray/_memray.cpp -o build/temp.linux-x86_64-cpython-311/src/memray/_memray.o -std=c++17 -Wall -flto -fvisibility=hidden
src/memray/_memray.cpp: In function ‘int __pyx_pf_6memray_7_memray_7Tracker___cinit__(__pyx_obj_6memray_7_memray_Tracker*, PyObject*, PyObject*, bool, unsigned int, bool, bool, __pyx_t_6memray_7_memray_FileFormat)’:
src/memray/_memray.cpp:11950:48: error: too few arguments to function ‘std::unique_ptr<memray::tracking_api::RecordWriter> memray::tracking_api::createRecordWriter(std::unique_ptr<memray::io::Sink>, const std::string&, bool, FileFormat, bool)’
11950 |     __pyx_t_8 = memray::api::createRecordWriter(cython_std::move<std::unique_ptr<memray::io::Sink> >(__pyx_t_7), __pyx_v_command_line, __pyx_v_native_traces, ((enum memray::tracking_api::FileFormat)__pyx_v_file_format));
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/memray/_memray.cpp:859:
src/memray/_memray/record_writer.h:61:1: note: declared here
   61 | createRecordWriter(
      | ^~~~~~~~~~~~~~~~~~
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

This doesn't make any sense to me, it looks like you have not regenerated the Python sources. I can build memray locally without any problems and the CI can build it as well so this must be some problem in your Debian rules file or the way you are building the package. Unfortunately we cannot help more here unless you provide us with a reproducer that doesn't use any Debian infrastructure.

1.11.0 should build, and 1.10.0 should have no warnings and errors

Regarding the warnings, I am afraid we won't be able to help more here as these are aspects of how Debian wants to package projects. We are opened to considered Pull Requests but otherwise it looks like if you want to help some of these you will need to maintain your own patches.

[pablogsal]

For more details, notice that src/memray/_memray.cpp is generated during the build process and these are the lines generated from the latest version:

    __pyx_t_9 = memray::api::createRecordWriter(cython_std::move<std::unique_ptr<memray::io::Sink> >(__PYX_STD_MOVE_IF_SUPPORTED(__pyx_t_8)), __pyx_v_command_line, __pyx_v_native_traces, ((enum memray::tracking_api::FileFormat)__pyx_v_file_format), __pyx_v_trace_python_allocators);

as you can see there, the call is totally different and includes __pyx_v_trace_python_allocators which is not in your build logs. This hints that you are somehow producing an outdated Cython file in your build process.

[godlygeek]

E: memray: arch-independent-package-contains-binary-or-object [usr/lib/python3/dist-packages/memray/_inject.abi3.so]

This error in particular indicates that you're trying to produce an architecture-independent package for Memray. That's wrong: memray includes native code, and needs to be compiled for each architecture. We support x86-64 (aka amd64) and aarch64 (aka arm64). So at least one problem that you've got here is that you've declared that the package should be architecture-independent, rather than architecture-dependent.

[pablogsal]

This looks like it is because you seem to be using textual 0.1.13 while our requirement (check setup.py) is textual >= 0.34.0

[godlygeek]

Ooh, that's a mistake on our part... it looks like we actually require at least Textual 0.41 now. Can you try with that version, @alexmyczko?

[godlygeek]

Alternatively, if you aren't able to upgrade Textual to (at least) 0.41 yet, you should be able to build and run Memray 1.10 (one version older than the newest) with Textual 0.34. The dependency on features that require Textual 0.41 came with Memray 1.11.

[godlygeek]

Correction: it looks like the minimum Textual version we support from Memray 1.11 on is now Textual 0.41, not Textual 0.39 (I've corrected my two comments above).

[pablogsal]

and src/memray/reporters will it still be useful for live/cli only mode?

Removing the reporters directory will make memray unusable and broken and is certainly not supported. Also, the live cli mode is a reporter as well. Please, do not package memray that way

and i'd like to also de-vendor src/vendor/libbacktrace, is that supported?

Not only it is not supported but it also won't work because we keep some patches on top of libbacktrace so it's not just a plain vendoring but a fork.

[alexmyczko]

Hi @pablogsal that hint about src/memray/*.cpp was already helpful, thank you.

[alexmyczko]

The ITP link is https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009970

[alexmyczko]

Ok now when I try to run it get this:

memray -h
Traceback (most recent call last):
  File "/usr/bin/memray", line 33, in <module>
    sys.exit(load_entry_point('memray==1.11.0', 'console_scripts', 'memray')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/memray", line 25, in importlib_load_entry_point
    return next(matches).load()
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/memray/__main__.py", line 3, in <module>
    from memray.commands import main
  File "/usr/lib/python3/dist-packages/memray/commands/__init__.py", line 19, in <module>
    from . import attach
  File "/usr/lib/python3/dist-packages/memray/commands/attach.py", line 19, in <module>
    from .live import LiveCommand
  File "/usr/lib/python3/dist-packages/memray/commands/live.py", line 7, in <module>
    from memray.reporters.tui import TUIApp
  File "/usr/lib/python3/dist-packages/memray/reporters/tui.py", line 30, in <module>
    from textual.app import ComposeResult
ImportError: cannot import name 'ComposeResult' from 'textual.app' (/usr/lib/python3/dist-packages/textual/app.py)
root@phd-sid:/var/www/debian/memray# ls -la /usr/lib/python3/dist-packages/textual/app.py
-rw-r--r-- 1 root root 18220 Jan  5  2022 /usr/lib/python3/dist-packages/textual/app.py
root@phd-sid:/var/www/debian/memray# dpkg -S /usr/lib/python3/dist-packages/textual/app.py
python3-textual: /usr/lib/python3/dist-packages/textual/app.py
root@phd-sid:/var/www/debian/memray# dpkg -l python3-textual
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version      Architecture Description
+++-===============-============-============-=================================================================================
ii  python3-textual 0.1.13-1     all          TUI (Text User Interface) framework for Python inspired by modern web development

[alexmyczko]

that is what i thought, what was confirmed yesterday on irc. and why i posted the version ;)

so trying 0.34.0 unfortunately that didn't help:
ii python3-textual 0.34.0-0.1 all
from https://sid.ethz.ch/debian/memray/python3-textual/

ok the error msg is a bit different

Traceback (most recent call last):
  File "/usr/bin/memray", line 33, in <module>
    sys.exit(load_entry_point('memray==1.11.0', 'console_scripts', 'memray')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/memray", line 25, in importlib_load_entry_point
    return next(matches).load()
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/memray/__main__.py", line 3, in <module>
    from memray.commands import main
  File "/usr/lib/python3/dist-packages/memray/commands/__init__.py", line 28, in <module>
    from . import tree
  File "/usr/lib/python3/dist-packages/memray/commands/tree.py", line 9, in <module>
    from memray.reporters.tree import TreeReporter
  File "/usr/lib/python3/dist-packages/memray/reporters/tree.py", line 33, in <module>
    from textual.widgets import TextArea
  File "/usr/lib/python3/dist-packages/textual/widgets/__init__.py", line 96, in __getattr__
    raise ImportError(f"Package 'textual.widgets' has no class '{widget_class}'")
ImportError: Package 'textual.widgets' has no class 'TextArea'

[alexmyczko]

0.41.0 is no problem: https://sid.ethz.ch/debian/memray/python3-textual/

can confirm memray runs now.

[godlygeek]

I don't think there's anything left to do for this particular issue.

Let me know if I'm wrong, @alexmyczko.

[alexmyczko]

well i'd really like to get this packaged for debian main, but i think i'll have to remove
docs, tests, and src/memray/reporters will it still be useful for live/cli only mode? is building/running it that way supported somehow?

and i'd like to also de-vendor src/vendor/libbacktrace, is that supported?
appears to be https://github.com/ianlancetaylor/libbacktrace not yet packaged https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1008135

[alexmyczko]

perfect answer. so there must be a way to selfhost the html stuff. and thanks for the info on libbacktrace

mind if i keep the issue open? for visibility for others