friedrichfroebel/python-djvulibre

Python interface to DjVuLibre

python-djvulibre's Issues

Windows test?

I'm trying package this for NixOS but it seems like it always runs the Windows test djvu/dllpath.py. It also fails when running python setup.py test -v

Is there anyway to disable this test?

$ python setup.py test -v
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing python_djvulibre.egg-info/PKG-INFO
writing dependency_links to python_djvulibre.egg-info/dependency_links.txt
writing requirements to python_djvulibre.egg-info/requires.txt
writing top-level names to python_djvulibre.egg-info/top_level.txt
reading manifest file 'python_djvulibre.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'doc/COPYING'
warning: no files found matching 'doc/credits'
adding license file 'COPYING'
writing manifest file 'python_djvulibre.egg-info/SOURCES.txt'
running build_ext
dllpath (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: dllpath (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: dllpath
Traceback (most recent call last):
  File "/nix/store/y0bgyc2q1wdy29ya89ykhsqys1mrjk5z-python3-3.9.16/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/dansbandit/python-djvulibre/djvu/dllpath.py", line 24, in <module>
    raise ImportError('This module is for Windows only')
ImportError: This module is for Windows only


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>

Test examples

At the moment, the examples have been left out from further refactorings. This should be changed:

• Drop Python 2 support.
• Fix code formatting.
• Add basic tests for CI to ensure they continue to work as expected.

Determine coverage

Determine the test coverage by generating a special build and running the tests on it: https://cython.readthedocs.io/en/latest/src/tutorial/profiling_tutorial.html#enabling-coverage-analysis

Setting the macro definition seems to be a simple CLI parameter: python setup.py build_ext --inplace --define CYTHON_TRACE.

Fix for Cython 3.0

Building the package currently fails due to changes introduced in Cython 3.0:

warning: build/temp.linux-x86_64-3.9/src/config.pxi:1:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: build/temp.linux-x86_64-3.9/src/config.pxi:2:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:60:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:108:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:114:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:131:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:145:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:165:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:172:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:181:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:185:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: djvu/sexpr.pyx:213:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310

Error compiling Cython file:
------------------------------------------------------------
...
        self.buffer = []
        self.exc = None
        IF HAVE_MINIEXP_IO_T:
            cexpr_io_init(&self.cio)
            self.cio.data[0] = <void*>self
            self.cio.getc = _myio_getc
                            ^
------------------------------------------------------------

djvu/sexpr.pyx:148:28: Cannot assign type 'int (cexpr_io_t *) except? -1' to 'int (*)(cexpr_io_t *) noexcept'

Error compiling Cython file:
------------------------------------------------------------
...
        self.exc = None
        IF HAVE_MINIEXP_IO_T:
            cexpr_io_init(&self.cio)
            self.cio.data[0] = <void*>self
            self.cio.getc = _myio_getc
            self.cio.ungetc = _myio_ungetc
                              ^
------------------------------------------------------------

djvu/sexpr.pyx:149:30: Cannot assign type 'int (cexpr_io_t *, int) except? -1' to 'int (*)(cexpr_io_t *, int) noexcept'

Error compiling Cython file:
------------------------------------------------------------
...
        IF HAVE_MINIEXP_IO_T:
            cexpr_io_init(&self.cio)
            self.cio.data[0] = <void*>self
            self.cio.getc = _myio_getc
            self.cio.ungetc = _myio_ungetc
            self.cio.puts = _myio_puts
                            ^
------------------------------------------------------------

djvu/sexpr.pyx:150:28: Cannot assign type 'int (cexpr_io_t *, const char *) except? -1' to 'int (*)(cexpr_io_t *, char *) noexcept'
building 'djvu.sexpr' extension

The deprecation warnings should be easy enough to replace, see jwilk/python-afl@afeb42a and the _openmp_helpers stuff at https://github.com/scikit-learn/scikit-learn/pull/25654/files#diff-c2e0bcc6a9647883a5ef4120b283b15d3dd1d0cb1dfe768ea0bebaac0ea138c3 for example.

Nevertheless, I am less confident about the solution to the actual compilation errors. I assume that they are related to https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#exception-values-and-noexcept.

Push updates to PyPI

The package cannot be installed from PyPI at the moment as we have

python-djvulibre/pyproject.toml

Line 5 in 181ed40

'Cython>=0.20; python_version>="3"',

This will install Cython 3.0 with its breaking changes, thus requiring manual modifications to the file to fix the build.

For this reason, a new package version would have to be pushed to PyPI.

Broken unittests?

It seems like the unittests are broken.

$ LC_ALL=C python -m unittest discover tests -v 
test_const (unittest.loader._FailedTest.test_const) ... ERROR
test_decode (unittest.loader._FailedTest.test_decode) ... ERROR
test_sexpr (unittest.loader._FailedTest.test_sexpr) ... ERROR

======================================================================
ERROR: test_const (unittest.loader._FailedTest.test_const)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_const
Traceback (most recent call last):
  File "/home/dansbandit/.local/share/pyenv/versions/3.11.4/lib/python3.11/unittest/loader.py", line 407, in _find_test_path
    module = self._get_module_from_name(name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dansbandit/.local/share/pyenv/versions/3.11.4/lib/python3.11/unittest/loader.py", line 350, in _get_module_from_name
    __import__(name)
  File "/home/dansbandit/djvu/python-djvulibre-FriedrichFroebel/tests/test_const.py", line 16, in <module>
    from djvu.const import (
  File "/home/dansbandit/djvu/python-djvulibre-FriedrichFroebel/djvu/const.py", line 20, in <module>
    import djvu.sexpr
ModuleNotFoundError: No module named 'djvu.sexpr'


======================================================================
ERROR: test_decode (unittest.loader._FailedTest.test_decode)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_decode
Traceback (most recent call last):
  File "/home/dansbandit/.local/share/pyenv/versions/3.11.4/lib/python3.11/unittest/loader.py", line 407, in _find_test_path
    module = self._get_module_from_name(name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dansbandit/.local/share/pyenv/versions/3.11.4/lib/python3.11/unittest/loader.py", line 350, in _get_module_from_name
    __import__(name)
  File "/home/dansbandit/djvu/python-djvulibre-FriedrichFroebel/tests/test_decode.py", line 24, in <module>
    from djvu.decode import (
ModuleNotFoundError: No module named 'djvu.decode'


======================================================================
ERROR: test_sexpr (unittest.loader._FailedTest.test_sexpr)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_sexpr
Traceback (most recent call last):
  File "/home/dansbandit/.local/share/pyenv/versions/3.11.4/lib/python3.11/unittest/loader.py", line 407, in _find_test_path
    module = self._get_module_from_name(name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dansbandit/.local/share/pyenv/versions/3.11.4/lib/python3.11/unittest/loader.py", line 350, in _get_module_from_name
    __import__(name)
  File "/home/dansbandit/djvu/python-djvulibre-FriedrichFroebel/tests/test_sexpr.py", line 24, in <module>
    from djvu.sexpr import (
ModuleNotFoundError: No module named 'djvu.sexpr'


----------------------------------------------------------------------
Ran 3 tests in 0.001s

does not build on s390x and likely other big endian platform

Hi,

We noticed while building the package for Debian that DJVULibre likely has endianness-related issues.

You might find this interresting, or state that you don't want to support S390x because there are likely no real-world user.

Greetings :-)

https://buildd.debian.org/status/package.php?p=python-djvulibre&suite=sid

Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_djvu/build/tests/test_decode.py", line 531, in test_decode
    self.assertEqual(s, b'\xFF\xFF\xFF\x00' * 4)
AssertionError: b'\x00\xff\xff\xff\x00\xff\xff\xff\x00\xff\xff\xff\x00\xff\xff\xff' != b'\xff\xff\xff\x00\xff\xff\xff\x00\xff\xff\xff\x00\xff\xff\xff\x00'

----------------------------------------------------------------------
Ran 109 tests in 0.091s

FAILED (SKIP=3, errors=2, failures=1)
E: pybuild pybuild:391: test: plugin distutils failed with: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_djvu/build; python3.11 -m nose -v tests
	rm -fr -- /tmp/dh-xdg-rundir-saNTo7j9
dh_auto_test: error: pybuild --test --test-nose -i python{version} -p "3.12 3.11" returned exit code 13
make[1]: *** [debian/rules:17: override_dh_auto_test] Error 25
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:11: build-arch] Error 2

Change language level to 3

At the moment, the Cython files use language level 2. This should be upgraded to use level 3.

This requires further fixes for successful compilation, as this will change the meaning of string literals without prefixes: https://cython.readthedocs.io/en/latest/src/tutorial/strings.html#string-literals

Windows build not working properly

I cloned the repo and run "pip install ." but the install complains of no pkg-config installed and certain djvu-libre header missing. Under posix the cure would be "well, install pkg-config". But in windows It doesn't seem fit. I've built dvju-libre from source without problem bit it does not produce a PC file that pkg-config (had I installed it) would use. There seems to be a missing step for your setup.py to find my compiled djvu libre files. I tried setting INCLUDE/LIB env vars to no avail. Djvu-libre can be installed via WinGet or Scoop package managers (at least the later has the development files (headers, libs, etc, still no .pc file)), but setup.py is not aware. This is kind of a problem in windows since much of the djvu tooling depends on this foundation package. I would prefer to avoid going the msys2 or cygwin routes.

Initial cleanup

• Drop support for Python 2
• Drop compatibility code for old package versions.
• Use flake8 for code style.
• Drop Windows support?
• Set language level to 3
  • Requires further fixes for successful compilation, as this will change the meaning of string literals without prefixes: https://cython.readthedocs.io/en/latest/src/tutorial/strings.html#string-literals