olokelo/jxlpy

Cython bindings and Pillow plugin for JPEG XL

jxlpy's Issues

Request to support installing from pypi for python3.11

Python 3.11 has been released last year, I tested jxlpy (install from github) works well.
But I cannot install it from pypi as there is no wheel (built distribution) for python3.11 and the source distrubution (.tar.gz) seems not work.
Please consider publish a wheel for python3.11.

JXLPyEncoder object has no attribute 'do_cleanup'

Installed from the main branch on Ubuntu 20.04 a few minutes ago, everything builds perfectly. The verify_installed.py script works but the encoding examples don't run.

The following exception is thrown for encode_simple.py:

Traceback (most recent call last):
  File "examples/encode_simple.py", line 6, in <module>
    enc = jxlpy.JXLPyEncoder(quality=100, colorspace='RGB', size=size, effort=7)
  File "_jxlpy/_jxl.pyx", line 699, in _jxlpy.JXLPyEncoder.__init__
    self.do_cleanup()
AttributeError: '_jxlpy.JXLPyEncoder' object has no attribute 'do_cleanup'

Adding an empty do_cleanup() method doesn't seem to help. Is something out of sync with the upstream library?

Memory leak in decoding

The following code causes the python process to use unbounded memory:

for i in range(10000):PIL.Image.open(BytesIO(img))

where img is a random JXL image.

I think the issue is that img._decoder.close() is never called.

for i in range(10000):
    img = PIL.Image.open(BytesIO(img))
    img._decoder.close()

works fine (barely any memory use)

Exif support

Does it copy the exif and color space metadata to jpeg xl from the original input?

Not working with recent Pillow (10.1.0)

This plugin isn't working with a recent version of Pillow (10.1.0)

    im = Image.open(args.image)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/PIL/Image.py", line 3284, in open
    im = _open_core(fp, filename, prefix, formats)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/PIL/Image.py", line 3270, in _open_core
    im = factory(fp, filename)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/PIL/ImageFile.py", line 117, in __init__
    self._open()
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/jxlpy/JXLImagePlugin.py", line 31, in _open
    self.mode = self.rawmode = self._decoder.get_colorspace()

Reverting Pillow to 10.0.1 or 9.5.0 fixes the problem, both tested on Python 3.10.

pip install Pillow==9.5.0

or

pip install Pillow==10.0.1

Probably related to this API change.

Allow negative quality values

jxl allows "quality" values between -inf and 100. trying to use a negative quality value with jxlpy gives this error:

_jxlpy.JxlPyArgumentInvalid: quality (should be in range 0..100)

pip

Even if alpha - could you add this package to pypi.org?

Access Violation on Windows Build when encoding

When building this wheel on Windows using:

vcpkg install libjxl --triplet x64-windows

pip wheel --use-feature=in-tree-build ./jxlpy

delvewheel repair -v --add-path C:\tools\vcpkg\installed\x64-windows\bin --no-dll msvcp140.dll --no-mangle-all jxlpy-0.9.1-cp39-cp39-win_amd64.whl

pip uninstall jxlpy -y
pip install wheelhouse/jxlpy-0.9.1-cp39-cp39-win_amd64.whl

python decode_simple.py

python encode_decode_pillow.py

echo %errorlevel%

This gives an access violation on calling im.save, with the call stack

Exception thrown at 0x00007FFFDF941372 (jxl.dll) in python.exe: 0xC0000005: Access violation reading location 0x0000000000003C80.
[Inline Frame] jxl.dll!hwy::N_AVX2::LoadDup128(hwy::N_AVX2::Simd<float,8>) Line 1994
	at C:\tools\vcpkg\installed\x64-windows\include\hwy\ops\x86_256-inl.h(1994)
jxl.dll!jxl::N_AVX2::Symmetric5Interior<jxl::WrapUnchanged>(const jxl::Plane<float> & in, const jxl::Rect & rect, const __int64 ix, const __int64 iy, const jxl::WeightsSymmetric5 & weights, float * row_out) Line 95
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(95)
jxl.dll!jxl::N_AVX2::Symmetric5Row<jxl::WrapUnchanged>(const jxl::Plane<float> & in, const jxl::Rect & rect, const __int64 iy, const jxl::WeightsSymmetric5 & weights, float * row_out) Line 131
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(131)
[Inline Frame] jxl.dll!jxl::N_AVX2::Symmetric5::__l2::<lambda_89f43947037a03667304206744c598ea>::operator()(const int) Line 960
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(960)
jxl.dll!jxl::ThreadPool::RunCallState<jxl::Status __cdecl(unsigned __int64),<lambda_89f43947037a03667304206744c598ea>>::CallDataFunc(void * jpegxl_opaque, unsigned int value, unsigned __int64 thread_id) Line 89
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(89)
jxl_threads.dll!jpegxl::ThreadParallelRunner::Runner(void * runner_opaque, void * jpegxl_opaque, int(*)(void *, unsigned __int64) init, void(*)(void *, unsigned int, unsigned __int64) func, unsigned int start_range, unsigned int end_range) Line 72
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\threads\thread_parallel_runner_internal.cc(72)
[Inline Frame] jxl.dll!jxl::ThreadPool::Run(unsigned int) Line 51
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(51)
[Inline Frame] jxl.dll!jxl::ThreadPool::Run(unsigned int) Line 60
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(60)
[Inline Frame] jxl.dll!jxl::RunOnPool(jxl::ThreadPool *) Line 117
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\base\data_parallel.h(117)
jxl.dll!jxl::N_AVX2::Symmetric5(const jxl::Plane<float> & in, const jxl::Rect & rect, const jxl::WeightsSymmetric5 & weights, jxl::ThreadPool * pool, jxl::Plane<float> * out) Line 964
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\convolve.cc(964)
jxl.dll!jxl::GaborishInverse(jxl::Image3<float> * in_out, float mul, jxl::ThreadPool * pool) Line 62
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\gaborish.cc(62)
jxl.dll!jxl::DefaultEncoderHeuristics::LossyFrameHeuristics(jxl::PassesEncoderState * enc_state, jxl::ModularFrameEncoder * modular_frame_encoder, const jxl::ImageBundle * original_pixels, jxl::Image3<float> * opsin, jxl::ThreadPool * pool, jxl::AuxOut * aux_out) Line 865
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\enc_heuristics.cc(865)
jxl.dll!jxl::LossyFrameEncoder::ComputeEncodingData(const jxl::ImageBundle * linear, jxl::Image3<float> * opsin, jxl::ThreadPool * pool, jxl::ModularFrameEncoder * modular_frame_encoder, jxl::BitWriter * writer, jxl::FrameHeader * frame_header) Line 497
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\enc_frame.cc(497)
jxl.dll!jxl::EncodeFrame(const jxl::CompressParams & cparams_orig, const jxl::FrameInfo & frame_info, const jxl::CodecMetadata * metadata, const jxl::ImageBundle & ib, jxl::PassesEncoderState * passes_enc_state, jxl::ThreadPool * pool, jxl::BitWriter * writer, jxl::AuxOut * aux_out) Line 1187
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\enc_frame.cc(1187)
jxl.dll!JxlEncoderStruct::RefillOutputByteQueue() Line 94
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\encode.cc(94)
jxl.dll!JxlEncoderProcessOutput(JxlEncoderStruct * enc, unsigned char * * next_out, unsigned __int64 * avail_out) Line 458
	at C:\tools\vcpkg\buildtrees\libjxl\src\v0.6.1-d6e48761e1.clean\lib\jxl\encode.cc(458)
_jxlpy.cp39-win_amd64.pyd!__pyx_pf_6_jxlpy_12JXLPyEncoder_4get_output(__pyx_obj_6_jxlpy_JXLPyEncoder * __pyx_v_self) Line 5423
	at C:\Users\ErwindeHaan\jxlpy\jxlpy\_jxlpy\_jxl.cpp(5423)
python39.dll!method_vectorcall_NOARGS(_object * func, _object * const * args, unsigned __int64 nargsf, _object * kwnames) Line 435
	at \objects\descrobject.c(435)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
[Inline Frame] python39.dll!PyObject_Vectorcall(_object * nargsf, _object * const *) Line 127
	at \include\cpython\abstract.h(127)
python39.dll!call_function(_ts * tstate, _object * * * pp_stack, __int64 oparg, _object * kwnames) Line 5072
	at \python\ceval.c(5072)
python39.dll!_PyEval_EvalFrameDefault(_ts * tstate, _frame * f, int throwflag) Line 3505
	at \python\ceval.c(3505)
[Inline Frame] python39.dll!_PyEval_EvalFrame(_ts *) Line 40
	at \include\internal\pycore_ceval.h(40)
python39.dll!_PyEval_EvalCode(_ts * tstate, _object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4327
	at \python\ceval.c(4327)
[Inline Frame] python39.dll!_PyFunction_Vectorcall(_object * stack, _object * const *) Line 396
	at \objects\call.c(396)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
[Inline Frame] python39.dll!PyObject_Vectorcall(_object * nargsf, _object * const *) Line 127
	at \include\cpython\abstract.h(127)
python39.dll!call_function(_ts * tstate, _object * * * pp_stack, __int64 oparg, _object * kwnames) Line 5072
	at \python\ceval.c(5072)
python39.dll!_PyEval_EvalFrameDefault(_ts * tstate, _frame * f, int throwflag) Line 3519
	at \python\ceval.c(3519)
[Inline Frame] python39.dll!_PyEval_EvalFrame(_ts *) Line 40
	at \include\internal\pycore_ceval.h(40)
python39.dll!_PyEval_EvalCode(_ts * tstate, _object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4327
	at \python\ceval.c(4327)
[Inline Frame] python39.dll!_PyFunction_Vectorcall(_object * stack, _object * const *) Line 396
	at \objects\call.c(396)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
python39.dll!method_vectorcall(_object * method, _object * const * args, unsigned __int64 nargsf, _object * kwnames) Line 53
	at \objects\classobject.c(53)
[Inline Frame] python39.dll!_PyObject_VectorcallTstate(_ts *) Line 118
	at \include\cpython\abstract.h(118)
[Inline Frame] python39.dll!PyObject_Vectorcall(_object * nargsf, _object * const *) Line 127
	at \include\cpython\abstract.h(127)
python39.dll!call_function(_ts * tstate, _object * * * pp_stack, __int64 oparg, _object * kwnames) Line 5072
	at \python\ceval.c(5072)
python39.dll!_PyEval_EvalFrameDefault(_ts * tstate, _frame * f, int throwflag) Line 3536
	at \python\ceval.c(3536)
[Inline Frame] python39.dll!_PyEval_EvalFrame(_ts *) Line 40
	at \include\internal\pycore_ceval.h(40)
python39.dll!_PyEval_EvalCode(_ts * tstate, _object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4327
	at \python\ceval.c(4327)
python39.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object * locals, _object * const * args, __int64 argcount, _object * const * kwnames, _object * const * kwargs, __int64 kwcount, int kwstep, _object * const * defs, __int64 defcount, _object * kwdefs, _object * closure, _object * name, _object * qualname) Line 4366
	at \python\ceval.c(4366)
python39.dll!PyEval_EvalCodeEx(_object * _co, _object * globals, _object * locals, _object * const * args, int argcount, _object * const * kws, int kwcount, _object * const * defs, int defcount, _object * kwdefs, _object * closure) Line 4382
	at \python\ceval.c(4382)
python39.dll!PyEval_EvalCode(_object * co, _object * globals, _object * locals) Line 832
	at \python\ceval.c(832)
python39.dll!run_eval_code_obj(_ts * tstate, PyCodeObject * co, _object * globals, _object * locals) Line 1220
	at \python\pythonrun.c(1220)
python39.dll!run_mod(_mod * mod, _object * filename, _object * globals, _object * locals, PyCompilerFlags * flags, _arena * arena) Line 1241
	at \python\pythonrun.c(1241)
python39.dll!pyrun_file(_iobuf * fp, _object * filename, int start, _object * globals, _object * locals, int closeit, PyCompilerFlags * flags) Line 1138
	at \python\pythonrun.c(1138)
python39.dll!pyrun_simple_file(_iobuf * fp, _object * filename, int closeit, PyCompilerFlags * flags) Line 452
	at \python\pythonrun.c(452)
python39.dll!PyRun_SimpleFileExFlags(_iobuf * fp, const char * filename, int closeit, PyCompilerFlags * flags) Line 483
	at \python\pythonrun.c(483)
python39.dll!PyRun_AnyFileExFlags(_iobuf * fp, const char * filename, int closeit, PyCompilerFlags * flags) Line 92
	at \python\pythonrun.c(92)
python39.dll!pymain_run_file(const PyConfig * config, PyCompilerFlags * cf) Line 379
	at \modules\main.c(379)
python39.dll!pymain_run_python(int * exitcode) Line 605
	at \modules\main.c(605)
python39.dll!Py_RunMain() Line 685
	at \modules\main.c(685)
python39.dll!pymain_main(_PyArgv * args) Line 1106
	at \modules\main.c(1106)
python39.dll!Py_Main(int argc, wchar_t * * argv) Line 1118
	at \modules\main.c(1118)

So it seems to happen on the call here.

/* "_jxlpy/_jxl.pyx":804
 *         while self.status == JXL_ENC_NEED_MORE_OUTPUT:
 * 
 *             self.status = JxlEncoderProcessOutput(             # <<<<<<<<<<<<<<
 *                 self.encoder, &next_out, &avail_out
 *             )
 */

Do note to make this work you need to add the follwing to setup.py

jxlpy_ext = Extension(
    name="_jxlpy",
    sources=["_jxlpy/_jxl.pyx"],
    libraries=['jxl','jxl_threads'],
    extra_compile_args=['-Ox','-Zi'], # For debugging only, use O2 for release build
    extra_link_args=['-debug:full'], # For debugging only
    language='c++'
)
from sys import platform
if platform == "win32":
    jxlpy_ext.include_dirs=['C:\\tools\\vcpkg\\installed\\x64-windows\\include']
    jxlpy_ext.library_dirs=['C:\\tools\\vcpkg\\installed\\x64-windows\\lib']

Any ideas?

Support for L colorspace and extra channels.

For the record in case people are looking: I have implemented both these features in my own branch at https://github.com/alexjc/jxlpy.

Just need to merge #13 first, then I'll port them over!

Installation issue on mac os

Hi, out-of-the box installation fails for me:

(pipeline) ➜  ~ pip install jxlpy
Collecting jxlpy
  Using cached jxlpy-0.9.1.tar.gz (73 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/axelr/system1/envs/pipeline/bin/python3 /Users/axelr/system1/envs/pipeline/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/tmpt85aclzc
       cwd: /private/var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/pip-install-uxoxexbg/jxlpy_f0fdeceeee594351831466bb8cfd84aa
  Complete output (22 lines):
  Traceback (most recent call last):
    File "/Users/axelr/system1/envs/pipeline/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/Users/axelr/system1/envs/pipeline/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/axelr/system1/envs/pipeline/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/private/var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/pip-build-env-36rlrw9c/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "/private/var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/pip-build-env-36rlrw9c/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
      self.run_setup()
    File "/private/var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/pip-build-env-36rlrw9c/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 158, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 37, in <module>
      ext_modules=cythonize([jxlpy_ext]),
    File "/private/var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/pip-build-env-36rlrw9c/overlay/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 966, in cythonize
      module_list, module_metadata = create_extension_list(
    File "/private/var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/pip-build-env-36rlrw9c/overlay/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 816, in create_extension_list
      for file in nonempty(sorted(extended_iglob(filepattern)), "'%s' doesn't match any files" % filepattern):
    File "/private/var/folders/m7/d0p8pv5s6x5g09z3rwkgp24m0000gn/T/pip-build-env-36rlrw9c/overlay/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 114, in nonempty
      raise ValueError(error_msg)
  ValueError: '_jxlpy/_jxl.pyx' doesn't match any files

Environment:

Python 3.9.0
pip 21.3.1
mac os 11.5.2

"Couldn't recognize the image format for file" on big-endian (PowerPC 32-bit)

I have compiled libjxl (with libhwy built with SIMD instructions disabled) on my Powerbook G4 running Debian. The library works fine on my little-endian systems (x86, ARM) but fails on the big endian one. The standalone cjxl and djxl tools are working on the powerbook, so I know that my library build is functional.

My error (in MComix3, but my own fork that has imports for jxlpy in appropriate locations) is [ImageHandler] ERROR: Could not load pixbuf for page <n>: GLib.Error('Couldn't recognize the image format for file ""', 'gdk-pixbuf-error-quark', 3)`.

My guess (without looking at the code) is that the library is doing a 'magic word' check for some hexadecimal number, which has the bytes in opposite order on big endian hardware.

I think that something like what's being done here may be in order? Not really sure.

Implementing multithreading

I took a music cover and ran the 2x example test, generating a 13 MB JXL.

This takes a VERY long time to load via jxlpy

In comparison for example with nomacs, which takes less than five times what jxlpy does due to using multiple threads.

Would it be feasible to make this library use multiple threads? I was trying to use an application with Pillow and this extension, and loading this image made the startup time very long.

New release

Please make a new release, as the current version does not contain important fixes, such b5457c8 and 6df0993

Can't install via pip on windows

Hi, I tried to install jxlpy on windows using pip install jxlpy and got this error: ValueError: '_jxlpy/_jxl.pyx' doesn't match any files

Latest libjxl requires JxlEncoderCloseFrames to output valid .jxl

There's a new API function in the latest libjxl that requires JxlEncoderCloseFrames to be called with the encoder before creating the output. Otherwise, the encoder does not output a well-formed .jxl file, and it fails to decode without --allow_partial_files.

I put this call at the top of get_output() in the Cython file and it seems to work!