mozilla/mozjpeg

Improved JPEG encoder.

mozjpeg's Issues

How to generate Javadoc as a standalone Jar for usage with mozjpeg?

I'm using mozjpeg with java maven project. Under windows with cMake and MinGW it compiles perfectly, but there is no java documentation in the resulting Jar.
Could you please help by explaining how to generate Javadoc?

cjpeg -dct float produces bad quality output after trellis branch merge

bad quality sample attached.
command-line: cjpeg-static.exe -q 98 -dct float -outfile Lenna.jpg Lenna.bmp

Trouble for build on iOS

Hi,

I'm trying to build on iOS, but I have some trouble, I am on Maverick 10.9.2 + xcode 5.1, I reinstall an older version of xcode (Xcode 4.5) and call sudo xcode-select -switch /Applications/Xcode45.app/

Following the BUILDING.txt, I try to build for armv7, I just change the IOS_PLATFORMDIR with IOS_PLATFORMDIR="/Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform"

When I call sh configure with all parameter, I have this error :
configure: error: C preprocessor "/lib/cpp

What am I doing wrong ??

------ CONSOLE ------
sh /Users/aliot/Desktop/actisku/engine/software/sdk/mozjpeg/source/configure --host arm-apple-darwin10 --enable-static --disable-shared CC=/Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 LD=/Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 CFLAGS="-mfloat-abi=softfp -isysroot /Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS6.0.sdk -O3 -march=armv7 -mcpu=cortex-a8 -mtune=cortex-a8 -mfpu=neon" LDFLAGS="-mfloat-abi=softfp -isysroot /Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS6.0.sdk -march=armv7 -mcpu=cortex-a8 -mtune=cortex-a8 -mfpu=neon"
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for arm-apple-darwin10-strip... no
checking for strip... strip
checking for a thread-safe mkdir -p... /Users/aliot/Desktop/actisku/engine/software/sdk/mozjpeg/source/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for arm-apple-darwin10-gcc... /Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 accepts -g... yes
checking for /Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 option to accept ISO C89... none needed
checking whether /Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 understands -c and -o together... yes
checking dependency style of /Applications/Xcode45.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2... gcc3
checking how to run the C preprocessor... /lib/cpp
configure: error: in /Users/aliot/Desktop/actisku/engine/software/sdk/mozjpeg/temp': configure: error: C preprocessor "/lib/cpp" fails sanity check Seeconfig.log' for more details

mozjpeg jpegtran segfaults

same issue happens in both Linux and Win32.

~/mozjpeg$ gdb --args ./jpegtran -verbose -optimize -outfile 2.jpg.new e52e7a1c-a9e4-11e3-92e5-4043847afcf0.jpg
GNU gdb (GDB) CentOS (7.0.1-45.el5.centos)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /root/mozjpeg/jpegtran...done.
(gdb) r
Starting program: /root/mozjpeg/jpegtran -verbose -optimize -outfile 2.jpg.new e52e7a1c-a9e4-11e3-92e5-4043847afcf0.jpg
libmozjpeg version 1.0.0 (build 20140312)
Copyright (C) 1991-2012 Thomas G. Lane, Guido Vollbeding
Copyright (C) 1999-2006 MIYASAKA Masaru
Copyright (C) 2009 Pierre Ossman for Cendio AB
Copyright (C) 2009-2013 D. R. Commander
Copyright (C) 2009-2011 Nokia Corporation and/or its subsidiary(-ies)

Emulating The Independent JPEG Group's software, version 6b  27-Mar-1998

Start of Image
Define Quantization Table 0  precision 0
Start Of Frame 0xc2: width=2001, height=1250, components=4
    Component 67: 1hx1v q=0
    Component 77: 1hx1v q=0
    Component 89: 1hx1v q=0
    Component 75: 1hx1v q=0
Define Huffman Table 0x00
Start Of Scan: 4 components
    Component 67: dc=0 ac=0
    Component 77: dc=0 ac=0
    Component 89: dc=0 ac=0
    Component 75: dc=0 ac=0
  Ss=0, Se=0, Ah=0, Al=0
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 67: dc=0 ac=0
  Ss=1, Se=8, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 77: dc=0 ac=0
  Ss=1, Se=8, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 89: dc=0 ac=0
  Ss=1, Se=8, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 75: dc=0 ac=0
  Ss=1, Se=8, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 67: dc=0 ac=0
  Ss=9, Se=63, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 77: dc=0 ac=0
  Ss=9, Se=63, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 89: dc=0 ac=0
  Ss=9, Se=63, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 75: dc=0 ac=0
  Ss=9, Se=63, Ah=0, Al=2
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 67: dc=0 ac=0
  Ss=1, Se=63, Ah=2, Al=1
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 77: dc=0 ac=0
  Ss=1, Se=63, Ah=2, Al=1
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 89: dc=0 ac=0
  Ss=1, Se=63, Ah=2, Al=1
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 75: dc=0 ac=0
  Ss=1, Se=63, Ah=2, Al=1
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 67: dc=0 ac=0
  Ss=1, Se=63, Ah=1, Al=0
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 77: dc=0 ac=0
  Ss=1, Se=63, Ah=1, Al=0
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 89: dc=0 ac=0
  Ss=1, Se=63, Ah=1, Al=0
Define Huffman Table 0x10
Start Of Scan: 1 components
    Component 75: dc=0 ac=0
  Ss=1, Se=63, Ah=1, Al=0
End Of Image

Program received signal SIGSEGV, Segmentation fault.
0x0806a79b in copy_buffer (cinfo=0xbffff628, scan_idx=-452701905) at jcmaster.c:576
576       unsigned long size = master->scan_size[scan_idx];
(gdb) p *master
$1 = {pub = {prepare_for_pass = 0x806a3d9 <prepare_for_pass>, pass_startup = 0x806a741 <pass_startup>, finish_pass = 0x806b301 <finish_pass_master>, call_pass_startup = 0, is_last_pass = 1},
  pass_type = huff_opt_pass, pass_number = 33, total_passes = 34, scan_number = 16, scan_buffer = {0x80a0b88 "\377", <incomplete sequence \333>, 0x8094b78 "\377", <incomplete sequence \304>,
    0x809ab88 "\377", <incomplete sequence \304>, 0x80b0b90 "\377", <incomplete sequence \304>, 0x80c0ba8 "\377", <incomplete sequence \304>, 0x80d0bb0 "\377", <incomplete sequence \304>,
    0x80b4b98 "\377", <incomplete sequence \304>, 0x80b8ba0 "\377", <incomplete sequence \304>, 0x80ecbc8 "\377", <incomplete sequence \304>, 0x80fcbd0 "\377", <incomplete sequence \304>,
    0x80e4bc0 "\377", <incomplete sequence \304>, 0x810cbd8 "\377", <incomplete sequence \304>, 0x8114be0 "\377", <incomplete sequence \304>, 0x8124bf0 "\377", <incomplete sequence \304>, 0x0,
    0x0, 0x0, 0xf8ec179e <Address 0xf8ec179e out of bounds>, 0x76e647a1 <Address 0x76e647a1 out of bounds>, 0x372fbef9 <Address 0x372fbef9 out of bounds>,
    0x5446f6e9 <Address 0x5446f6e9 out of bounds>, 0xba4e5bbe <Address 0xba4e5bbe out of bounds>, 0x96c074fa <Address 0x96c074fa out of bounds>, 0xfa52ca08 <Address 0xfa52ca08 out of bounds>,
    0x857ed0af <Address 0x857ed0af out of bounds>, 0x7afafae6 <Address 0x7afafae6 out of bounds>, 0x8c6304b5 <Address 0x8c6304b5 out of bounds>, 0xf8c2cbfc <Address 0xf8c2cbfc out of bounds>,
    0xc359431f <Address 0xc359431f out of bounds>, 0x3133bcc5 <Address 0x3133bcc5 out of bounds>, 0x2d000092 <Address 0x2d000092 out of bounds>, 0xbe39edac <Address 0xbe39edac out of bounds>,
    0xcdfded8f <Address 0xcdfded8f out of bounds>, 0xb282cc14 <Address 0xb282cc14 out of bounds>, 0x86b6bfbe <Address 0x86b6bfbe out of bounds>, 0x79769ba2 <Address 0x79769ba2 out of bounds>,
    0xfad6c3ec <Address 0xfad6c3ec out of bounds>, 0x4a5e473f <Address 0x4a5e473f out of bounds>, 0x3c104066 <Address 0x3c104066 out of bounds>, 0x37f3987f <Address 0x37f3987f out of bounds>,
    0xcbe6a2de <Address 0xcbe6a2de out of bounds>, 0xea6f19ed <Address 0xea6f19ed out of bounds>, 0x4eeb75ca <Address 0x4eeb75ca out of bounds>, 0x9f4309bd <Address 0x9f4309bd out of bounds>,
    0x382a3a0d <Address 0x382a3a0d out of bounds>, 0xa3677cf5 <Address 0xa3677cf5 out of bounds>, 0xeab4ba27 <Address 0xeab4ba27 out of bounds>, 0xe639f6eb <Address 0xe639f6eb out of bounds>,
    0x996510ca <Address 0x996510ca out of bounds>, 0x81cc396 <Address 0x81cc396 out of bounds>, 0x80c4 <Address 0x80c4 out of bounds>, 0x0, 0x0, 0x0, 0x0, 0x0,
    0xbc262708 <Address 0xbc262708 out of bounds>, 0xf2715758 <Address 0xf2715758 out of bounds>, 0x3dfd6df8 <Address 0x3dfd6df8 out of bounds>, 0xecec3c7f <Address 0xecec3c7f out of bounds>,
    0xcd9ed1ec <Address 0xcd9ed1ec out of bounds>, 0xe12cb391 <Address 0xe12cb391 out of bounds>, 0x38460020 <Address 0x38460020 out of bounds>, 0x26c3ebb4 <Address 0x26c3ebb4 out of bounds>},
  scan_size = {52143, 15594, 11285, 14558, 34778, 34435, 12556, 16105, 34925, 36719, 20999, 26056, 30513, 36719, 251658384, 4081655505, 674001968, 4242353149, 1272850508, 533353962, 108,
    16777216, 784332148, 2163761075, 0, 0, 0, 0, 0, 1445003264, 2582576883, 10108645, 0, 3709582596, 3752415228, 2043257159, 4221079371, 3129618532, 1870583133, 4260950473, 3802721146, 76748896,
    1777693943, 2127601658, 605284231, 2762842978, 2313157048, 2360473329, 1691094123, 957547648, 4138784573, 4002348564, 2533590259, 2259327899, 1588042658, 4208305969, 549799727, 4286062592,
    1038557952, 922004303, 1280073343, 1361887723, 2591713950, 2391674538}, best_cost = 69213, best_freq_split_idx_luma = 1921132695, best_freq_split_idx_chroma = 0, best_Al_luma = 159409110,
  best_Al_chroma = 0, interleave_chroma_dc = 0, saved_dest = 0x808c220}
(gdb) bt
#0  0x0806a79b in copy_buffer (cinfo=0xbffff628, scan_idx=-452701905) at jcmaster.c:576
#1  0x0806b084 in select_scans (cinfo=0xbffff628, next_scan_number=14) at jcmaster.c:731
#2  0x0806b3db in finish_pass_master (cinfo=0xbffff628) at jcmaster.c:810
#3  0x0804dd15 in jpeg_finish_compress (cinfo=0xbffff628) at jcapimin.c:186
#4  0x08049a5c in main (argc=6, argv=0xbffffa54) at jpegtran.c:585
(gdb)

samples (hope github don't postprocess them):

Use more precise chroma subsampling

There are few tricky problems in chroma subsampling:
http://www.glennchan.info/articles/technical/chroma/chroma1.htm

e.g. libjpeg averages chroma of black pixels with chroma of neighboring color pixels, which causes darkened or desaturated edges. Even most naive blending with weighting by saturation gives a noticeable improvement:

https://github.com/pornel/jpeg-compressor#subsampling

Please add LICENSE file.

Please add LICENSE file so that it will be more clear how can one use this project.

Can't be compiled in VS Environment

Thanks for great project. I love it!!

BTW, this source cannot be compiled in VS environment.
(I tested with VS2008/2012 in Win7/x86)
There are 3 points to be fixed.

• CMakeLists.txt / line 488

SOURCES libjpeg-turbo.nsi)

should be modified to

SOURCES libmozjpeg.nsi)

• jcparam.c / line 741 ~

Some parts of jpeg_simple_progression (j_compress_ptr cinfo) are
written in C++ style.

int ncomps... / int nscans... / jpeg_scan_info * scanptr (line 749~751)

These 3 lines should be moved and slightly modified.

• name of generated files are turbojpeg-static.lib (and so on)

No. not mozjpeg-static.lib (and so on)

stdin does not work, stdout produces broken output.

Which version jpegrescan was considered in mozjpeg?

...when moving code into c? the original code is different from the modified. In the new code added only jfifremove or something I missed?

Evaluate DC Tune algorithm

As far as I can tell the DC Tune patent has expired in December.

Blog post link in repo

Always good to put the link to the blog post in the description and/or readme:

https://blog.mozilla.org/research/2014/03/05/introducing-the-mozjpeg-project/

If someone doesn't know about the blog post, they can check it out. Otherwise, a better description in the readme would help.

re-brand from libjpeg-turbo to libmozjpeg

Error when optimizing CMYK JPEG

When I try to optimize the attached CMYK JPEG, the optimizer crashes, and it generates a 0 bytes output file.

cjpeg help indicates that -optimize is not on by default

This option should be on by default and the help should reflect that.

[josh@raspberry mozjpeg] ./cjpeg --help
usage: /Users/josh/src/mozilla/mozjpeg/.libs/cjpeg [switches] [inputfile]
Switches (names may be abbreviated):
-quality N[,...] Compression quality (0..100; 5-95 is useful range)
-grayscale Create monochrome JPEG file
-rgb Create RGB JPEG file
-optimize Optimize Huffman table (smaller file, but slow compression)
...

resolve incompatibility with ipp-samples (Intel Performance Primitives) JPEG decoder

Images generated with mozjpeg's command line tool with mozjpeg defaults (with multiple scans) cannot be decoded by ipp-samples JPEG decoder.

jpgcrush feature breaks grayscale JPEG support

From DRC, author of libjpeg-turbo:

The jpgcrush feature breaks grayscale JPEG support. More specifically, an incorrect JPEG image is generated whenever both the input and JPEG colorspaces are grayscale. Attempting to decompress this incorrect JPEG image produces an error or a segfault. This is most easily reproducible by running tjbench.

Avoid 0xFF00 markers in Huffman data

0xFF bytes in Huffman data need special coding that adds another byte of overhead. If this overhead could be reliably minimized, then approximation of size if scan data described #15 would be more effective.

Here's an algorithm off the top of my head:

1. Find which code happens to overlap the 0xFF bytes most frequently
2. Swap the code with another code of the same length (if there isn't a code to swap with — maybe modify/regenerate the Huffman tree (e.g. flip all bits in all codes?))
3. goto 1 as long as result keeps getting smaller

And/Or:

1. For all symbols in the stream make histogram of pairs (code[symbol[n-1]], code[symbol[n]])
2. Find which pairs have 8+ consecutive 1s.
3. Starting from most frequent pair swap codes with other codes of the same length to break the streaks.

Likely it can be improved.

Implement activity masking

The current trellis quantization implementation does not take block variance into account when estimating error. By weighting by the block variance we should be able to get a decent improvement on SSIM style metrics like Theora did:
http://people.xiph.org/~xiphmont/demo/theora/demo9.html

decide whether or not we care about ABI compat

In mozjpeg 1.0 we broke ABI compatibility with the new configuration options. At the same time we've talked about trying to make it possible for our library to be a drop-in replacement for other libjpeg libraries. These are in conflict. We should either maintain ABI compatibility or give up on the idea that existing users might be able to, or should, use our library as a drop-in replacement.

I'm currently leaning towards the latter because most library users are better off sticking with libjpeg-turbo. This is because most users care about decoding and we aren't planning to invest in the decoder. Furthermore, the deployment context for our software, which is focused on the Web, will rarely involve mozjpeg being used in library form. Maintaining ABI compat is extra work we don't benefit much from, and it might give the wrong impression about replacing existing libjpeg usage with mozjpeg.

mozjpeg jpegtran fails to optimize files with JCS_RGB colorspace

jpegtran outputs 20-bytes file.

Sample created from "cjpeg -rgb -dct float -outfile rgb.jpg Lenna.bmp"

add logging to show which progressive modes are chosen

A request:

It would be interesting to analyze which of the progressive modes are selected. Then, we could preselect some specific progressive mode as a function of image width (for example). Could we add logging to show which of the progressive modes are chosen?

Implement trellis quantization

Trellis quantization is an adaptive quantization technique that select the set of levels in a transform block that minimises a rate-distortion metric. See http://en.wikipedia.org/wiki/Trellis_quantization

An adequate distortion metric should be defined for such optimization. A weighted MSE metric may be considered.

An initial implementation should consider only the baseline entropy coding scheme of JPEG. Later refinements may consider progressive entropy coding modes.

create system for evaluating compression improvements

As we work on this project we'll need a reliable way to measure the progress of compression improvements. Ideally this can be an automated set of tests using various metrics (SSIM and PSNR variants).

See this repository for something that might be a good starting point:

https://github.com/bdaehlie/web_image_formats

Ultimately we'd like something integrated more directly into this project, though.

Remove support for image thumbnails

For efficient, web-based images - we should never write a thumbnail image.
Strip out support from code for thumbnails.

Single QT for small images

For very small images (thumbnails), we should evaluate using single quantisation tables to save some bytes and measure against impact on image quality.

better greyscale sensing

The problem: On the web you frequently find JPEG files with greyscale content that was coded as a full-colour JPEG - with a set of useless colour channels. Although those end up empty or mostly empty, they nevertheless make up a significant percentage of wasted bits in the file.
(In the case of BMP input, cjpeg doesn't even recognise files with only one [greyscale] channel as greyscale input and encodes full-colour JPEG. It works for targa, though...)

If speed is little of an issue, then cjpeg/jpegtran could well read through the whole file and detect if no pixel has colour.
It could even use a threshold of colourfulness and discard colour information below a certain threshold and this way eliminate colour channels that only contain some colour noise residue, if no pixel exceeds the threshold.

Merge successive DQT (FFDB) and DHT (FFC4) markers in a single one

A DQT marker can hold more than a single Quantization Table and a DHT marker can hold more than a single Huffman Table.
Currently mozjpeg only puts one table per marker which is a bit clunky since it introduces a small overhead (extra markers and size fields: 2 bytes each).

For instance compare these two files (dumps produced by JPEGsnoop).

---

What mozjpeg does (2 DQT markers, mushr-current.jpg):

*** Marker: DQT (xFFDB) ***
Define a Quantization Table.
OFFSET: 0x00000014
Table length = 67

Precision=8 bits
Destination ID=0 (Luminance)
DQT, Row ♯0: 1 1 1 1 1 1 1 1
DQT, Row ♯1: 1 1 1 1 1 1 1 1
DQT, Row ♯2: 1 1 1 1 1 1 1 2
DQT, Row ♯3: 1 1 1 1 1 1 2 2
DQT, Row ♯4: 1 1 1 1 1 2 2 3
DQT, Row ♯5: 1 1 1 1 2 2 3 3
DQT, Row ♯6: 1 1 1 2 2 3 3 3
DQT, Row ♯7: 1 1 2 2 3 3 3 3
Approx quality factor = 98.25 (scaling=3.50 variance=4.81)

*** Marker: DQT (xFFDB) ***
Define a Quantization Table.
OFFSET: 0x00000059
Table length = 67

Precision=8 bits
Destination ID=1 (Chrominance)
DQT, Row ♯0: 1 1 1 2 2 3 3 3
DQT, Row ♯1: 1 1 1 2 3 3 3 3
DQT, Row ♯2: 1 1 1 3 3 3 3 3
DQT, Row ♯3: 2 2 3 3 3 3 3 3
DQT, Row ♯4: 2 3 3 3 3 3 3 3
DQT, Row ♯5: 3 3 3 3 3 3 3 3
DQT, Row ♯6: 3 3 3 3 3 3 3 3
DQT, Row ♯7: 3 3 3 3 3 3 3 3
Approx quality factor = 98.39 (scaling=3.23 variance=0.50)

---

What could be done (a single DQT marker, mushr-merged.jpg):

*** Marker: DQT (xFFDB) ***
Define a Quantization Table.
OFFSET: 0x00000014
Table length = 132

Precision=8 bits
Destination ID=0 (Luminance)
DQT, Row ♯0: 1 1 1 1 1 1 1 1
DQT, Row ♯1: 1 1 1 1 1 1 1 1
DQT, Row ♯2: 1 1 1 1 1 1 1 2
DQT, Row ♯3: 1 1 1 1 1 1 2 2
DQT, Row ♯4: 1 1 1 1 1 2 2 3
DQT, Row ♯5: 1 1 1 1 2 2 3 3
DQT, Row ♯6: 1 1 1 2 2 3 3 3
DQT, Row ♯7: 1 1 2 2 3 3 3 3
Approx quality factor = 98.25 (scaling=3.50 variance=4.81)

Precision=8 bits
Destination ID=1 (Chrominance)
DQT, Row ♯0: 1 1 1 2 2 3 3 3
DQT, Row ♯1: 1 1 1 2 3 3 3 3
DQT, Row ♯2: 1 1 1 3 3 3 3 3
DQT, Row ♯3: 2 2 3 3 3 3 3 3
DQT, Row ♯4: 2 3 3 3 3 3 3 3
DQT, Row ♯5: 3 3 3 3 3 3 3 3
DQT, Row ♯6: 3 3 3 3 3 3 3 3
DQT, Row ♯7: 3 3 3 3 3 3 3 3
Approx quality factor = 98.39 (scaling=3.23 variance=0.50)

---

The same goes for the DHT markers.

This saves 16 bytes on this file (one DQT less and three DHT less, 4 bytes of overhead each).

It should be pretty reliable since Adobe Photoshop does it this way.

compile problem with trellis branch on Linux, needs -lm

Use overflow for deringing

See the result: https://github.com/pornel/jpeg-compressor#deringing

Decoders always clamp decoded pixels to the appropriate range, so the encoder is free to cause overflows to its advantage.

In high-contrast where black and white edges are present (e.g. text, line art) if instead of encoding white RGB 255,255,255 as-is you encode it as, say, RGB 272,272,272 then distortions around this edge will fall above 255, and thus the decoder will end up hiding the distortion and sharpening the edge by clamping decoded values.

In my experimental implementation I'm just increasing brightness of white pixels proportionally to DC quantization (using that as a proxy for expected distortion, assuming higher distortion needs bigger overflow to compensate):

https://github.com/pornel/jpeg-compressor/blob/9eeb009fdb8673224de5be978574cb139e4b8bf6/jpge.cpp#L884

Ideally this should be applied in blocks only when it improves compression (the cost is in potentially increased DC delta and more non-0 ACs).

Also instead of just overflowing all white pixels evenly there should be gradual increase from edges to avoid introducing high frequencies (something like blur&darken on values > 255).

Complete quoting for parameters of some CMake commands

Some parameters (like "${CMAKE_SOURCE_DIR}" and "${CMAKE_CURRENT_BINARY_DIR}") are passed to CMake commands in your build scripts without enclosing them by quotation marks. I see that these places will result in build difficulties if the contents of the used variables will contain special characters like spaces.

I would recommend to apply advices from a Wiki article.

Add -copy exif option

I would like to strip all markers except caption, author, etc. Would be nice to include a -copy exif option.

trellis branch merge breaks jpegtran -optimize

jpegtran -v -optimize shows:

Start of Image
JFIF APP0 marker: version 1.02, density 72x72  1
Miscellaneous marker 0xfe, length 36
Define Quantization Table 0  precision 0
Define Quantization Table 1  precision 0
Start Of Frame 0xc0: width=1200, height=1600, components=3
    Component 1: 1hx1v q=0
    Component 2: 1hx1v q=1
    Component 3: 1hx1v q=1
Define Huffman Table 0x00
Define Huffman Table 0x10
Define Huffman Table 0x01
Define Huffman Table 0x11
Start Of Scan: 3 components
    Component 1: dc=0 ac=0
    Component 2: dc=1 ac=1
    Component 3: dc=1 ac=1
  Ss=0, Se=63, Ah=0, Al=0
End Of Image
Bogus buffer control mode

Add DC trellis quantization

We could save a couple of bits here and there by adding support for trellis quantization of the dc coefficients.

I wrote a toy implementation of this kind of thing a long time ago:
https://pastebin.mozilla.org/5476854

implement 'jpgcrush' feature

Belatedly filing feature request for 'jpgcrush' functionality - finding the progressive config that requires the fewest bits. This is just about done, but any further discussion can happen here.

GUI for mozjpeg

Are you going to create a gui for mozjpeg?
I was thinking about something portable and multi-platform. Maybe simple application, supports the batch conversion, created in the shape of WinFF?
http://winff.org/

make trellis quantization work with transcoder

Filing this so we don't forget, I don't think it works right now and we need it to.

Speedup progressive scan optimization

It should be possible to double the speed of progressive scan optimization:
Currently each proposed scan is processed twice, first to collect huffman statistics and then to construct its bitstream. But for the purpose of deciding which scans to keep, we only need to know their sizes, not their contents, and size can be derived from the huffman statistics alone.

Add the capabilities of imgmin for optimizing images.

Imgmin is a lossy optimize but it does it by making the image appear no worse so it's a really good system. Also just optimizing the huffman trees I doubt will do a whole of extra work.

https://github.com/rflynn/imgmin

That is the link to his project where it goes through attempting to optimize the jpegs. If you've not already, I'd try to look at 7zip's deflate encoder and see if their huffman encoder is better than what most everyone else does with it. I know that their system can be beat every-so-slightly in full deflate streams when optimizing the huffman trees but still since this is trying to be the single best system out there I would like for you to look at that system, and also looking at imgmin to encode jpegs at a lossy level(not by default but an option) so that way the files are going to be greatly smaller whilst not losing any visual quality.

P.S. I imagine this is going to be used on the web/for websites so I see no reason to not add the option to do the lossy optimizations from imgmin in the file step/a non-standard option.

Jpeg decompression is quite poor

Jpeg decompression (of compressed with q=100 jpeg in other program) leaves a lot of artifacts.
If you need I can attach examples if you teach me how.
Maybe there is some way to teach decompressor to do it better.

Two buffer overflows

Submitted a bug report to libjpeg-turbo, mozjpeg is downstream from them, so I figured I should notify you.

Original bug report with patch is here:
http://sourceforge.net/p/libjpeg-turbo/bugs/68/

Their diff is here:
http://sourceforge.net/p/libjpeg-turbo/code/1323/tree//trunk/wrjpgcom.c?diff=5072eea21be1ce7be997428d:1322

one part of jcphuff.c is written C++ style

line 500: int sign = temp >> 31;
is written in C++ style.

So, it cannot be compiled with Visual Studio 2012.
If this is not intentional, it should be fixed.

no libmozjpeg files are compiled

When I configure/make/make install the code, I only get libjpeg and libturbojpeg files. They are placed in my /opt/libmozjpeg folder, but there are no mozjpeg files. Am I understanding this wrong?

Right shift bitwise operations

@fbossen From Issue #38 ... and then #39...
Assumes integers are stored as two's-complement 32-bit values and that the right-shift operator does sign extension. Earlier in the code we deal with Portability RIGHT_SHIFT_IS_UNSIGNED:
https://github.com/mozilla/mozjpeg/blob/master/jcphuff.c#L77
Not sure whether it's worth maintaining portability? Otherwise may be worth removing the extra code...
Only other occurrence is wrapped with an ifdef:
https://github.com/mozilla/mozjpeg/blob/master/jdhuff.c#L492

Is compatibility with Adobe Photoshop mandatory?

When I wrote JSK to show the different JPEG scans I quickly discovered that Adobe Photoshop CS5 (and CS6 at least) was unable to decode some JPEGs like the one included here (if I'm right the problem comes from the non-interleaved first scan, it holds only Y).
So is the compatibility target restricted to web browsers or larger i.e. including well-known software like Photoshop (which unfortunately is a bit crappy)?

Custom progressive scan used to produce it:
0: 0-0, 0, 0 ; # DC coefficients Y
0: 1-5, 0, 0 ; # 5 first AC coefficients Y
1,2: 0-0, 0, 0 ; # Interleaved DC scan Cb & Cr
0: 6-27, 0, 0 ; # 22 more Y
1: 1-14, 0, 0 ; # 14 first AC coefficients Cb
2: 1-14, 0, 0 ; # 14 first AC coefficients Cr
0: 28-63, 0, 0 ; # Remaining Y coefficients
1: 15-63, 0, 0 ; # Remaining Cb coefficients
2: 15-63, 0, 0 ; # Remaining Cr coefficients

targeting MS-SSIM produces worse MS-SSIM results than targeting PSNR-HVS-M

Targeting MS-SSIM doesn't work very well. We should tweak that configuration to do better.

-arithmetic is broken

You guys seem to be using an outdated version of libjpeg-turbo. I reported bugs with the mode a while back. The relevant commits are 1079, 1048, and 1031 from the svn repository.

Define supported decoding software

We should always produce a 'standard' and 'compliant' bitstream (although at times this may be poorly defined!).
Since some decoders have broken / incomplete implementations- and this may add a burden to support their legacy, we should define a range of 'minimum' decoders that will be supported and actively tested against (most common use cases). These would include certain browsers (i.e. do we support right back to IE6?) and common libraries.
As well as improving QA and Unit Tests, it will allow new concepts to be tested and fast tracked.
Previously support for editors (Photoshop Issue #29) was considered mandatory. If software has a broken decoder, there should be a discussion whether this should be supported at the cost of performance. Optimizing images for web publishing is normally a final step, and breaking editor compatibility may be acceptable. ...?

Add simple command-line wrapper

In another Issue, there was the decision that this project won't have binary releases, which is fine. However, to try and use the library, it'd be useful to at least have a way to build this down into a simple command-line tool that you could use something like:

mozjpeg < myimage.jpg > newimage.jpg

This way us interested parties can follow along ;) and someone could even add it to homebrew.

Precompiled Binaries?

I'd love to play with this project, but I'm struggling with the BUILDING.txt instructions on my macbook. Are you currently or do you plan to distribute binaries?

mozjpeg is an order of magnitude slower than libjpeg-turbo

Do you have any benchmarks on v1.0 of mozjpeg? I'm working on a project where compression speed is a much bigger concern than compression size. We found that libjpeg-turbo gave us some great speed improvements over libjpeg. We are excited about mozjpeg, but our first run didn't yield promising results. A quick benchmark showed that mozjpeg resulted in roughly a 60% smaller image than libjpeg-turbo, but it took almost 10 times longer to compress the raw image.

Are these the kind of results you guys are seeing or should we look for something we might have done wrong?