bakape/thumbnailer

Go media thumbnailer

thumbnailer

Package thumbnailer provides a more efficient media thumbnailer than available with native Go processing libraries through ffmpeg bindings.

Use

go get -u github.com/bakape/thumbnailer/v2

to install the library in your project.

For a comprehensive list of file formats supported by default see main.go:Process().

Dependencies

• Go >= 1.10
• C11 compiler
• make
• pkg-config
• pthread
• ffmpeg >= 4.1 libraries (libavcodec, libavutil, libavformat, libswscale)

NB:

• ffmpeg should be compiled with all the dependency libraries for formats you want to process. On most Linux distributions you should be fine with the packages in the stock repositories.
• Ubuntu patches to ffmpeg on some Ubuntu versions <19.10 break this library. In this case, please compile from unmodified ffmpeg sources using:

sudo apt build-dep ffmpeg
git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg
git checkout n4.1
./configure
make -j`nproc`
sudo make install

thumbnailer's Issues

Rewrite thumbnailer in Rust

I'll do this, and probably some parts of meguca/hydron too afterwards so we don't have to worry about memory bloat.
This'll take a little while, but I guess give me collaborator permissions on this repo and assign me to this issue.
In the meantime, could you give me some ideas on how thumbnailer could be improved, as well as your thoughts on whether or not we should have different libraries to use?

Compilation error "inconsistent definitions for C.struct_Buffer"

With go version go1.10.2 linux/amd64 on Ubuntu 18.04 i get compilation error from title.
// #include "thumbnailer.h" at cover_art.go fixes the issue

thumbnailer couldn't compile. blur.c no such file or directory

vendor/github.com/bakape/thumbnailer/build_flags_gpl.go:7:11: fatal error: blur.c: No such file or directory // #include "blur.c" ^~~~~~~~ compilation terminated.

Provide alternative pure C API

Test that with static tools and valgrind.

Rotate JPEG image thumbnails according to EXIF

Pass PNGs thumbs through libimagequant for lossy compression

Implement SVG rasterization using ffmpeg >=3.4

https://github.com/memononen/nanosvg

Depend on #30 #36

PDF processing

Ghostscript crashing program

Thumbnailing this PDF crashes the program somehow with
Magick: "gs" "-q" "-dBATCH" "-dSAFER" "-dMaxBitmap=50000000" "-dNOPAUSE" "-sDEVICE=pnmraw" "-dTextAlphaBits=4" "-dGraphicsAlphaBits=4" "-r72x72" "-sOutputFile=/tmp/gmizbT7x" "--" "/tmp/gm7SOzEA" "-c" "quit" (child process quit due to signal 9).
sO_o.pdf

Use libmagick for mime detection

Will enable us to move more code over to the C/C++ side.

Make rule for external dep upgrade

Add make rule for copying dep C and C++ files to project root from git submodules.

Use "thumbnail" filter to extract the most representative frame

https://www.ffmpeg.org/doxygen/trunk/vf__thumbnail_8c_source.html

We can maybe just adapt the linked code instead of calling a filter through libavfilter.

Add support for SVG

SVG format is quite popular, would be nice to have it supported.

Fix UB in C code

recompile with clang and UBSan, that should tell you exactly where the heap corruption happens.

Some webm not detected properly

beatbirb2.zip

ffmpeg: Invalid data found when processing input

Happens with vp9 webm encoded with -threads 16 -row-mt 1

[libvpx-vp9 @ 0x7f784c3d7800] Failed to decode frame: Bitstream not supported by this decoder
[libvpx-vp9 @ 0x7f784c3d7800] Additional information: Unsupported bitstream profile

Probably need to upgrade to ffmpeg 4.

Thumbnailing timeouts to prevent blocking and OOM

See bakape/shamichan#932

Segfault

select_best_frame (frames=frames@entry=0x7fff8dffab80, size=size@entry=1) at thumbnailer.c:43
43                      hists[frame_i][p[i * 3]]++;
```

Cleaner error passing from C

Write any errors to a char** argument. This reduces possible errors in error handling.

Improper file extension detection

Add opt-in perceptual hashes

• Steal IQDB's implementation or stick with the one used in Node.js meguca?
• Add functions for hash XOR comparison

fatal error: unexpected signal during runtime execution

Trying to run this neat library, but keeps on crashing for me.

• Ubuntu 18.10 (GNU/Linux 4.18.0-25-generic x86_64)
• Golang version go1.12 linux/amd64
• ffmpeg version 4.1

I tried with the ffmpeg mentioned in the readme but still the same.

My code:

opts := thumbnailer.Options{
    ThumbDims: thumbnailer.Dims{250, 250},
}

_, thumbImg, err := thumbnailer.Process(f, opts)
if err != nil {
    return nil, err
}

// name of output file, also determines output type
dir := common.GetDir() + "/upload/dam/"
outFilename := fmt.Sprintf("%s%d_%s.%s", dir, size, m["fileID"], "png")

// Save File
if err != thumbnailer.ErrCantThumbnail {
    f, err := os.Create(outFilename)
    if err != nil {
        return nil, err
    }
    defer f.Close()

    err = png.Encode(f, thumbImg)
    if err != nil {
        return nil, err
    }
}

Panic:

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0xcffe97]

runtime stack:
runtime.throw(0x150fdeb, 0x2a)
    /usr/lib/go-1.12/src/runtime/panic.go:617 +0x72
runtime.sigpanic()
    /usr/lib/go-1.12/src/runtime/signal_unix.go:374 +0x4a9

goroutine 45 [syscall]:
runtime.cgocall(0x957430, 0xc0001329b8, 0xc000198d80)
    /usr/lib/go-1.12/src/runtime/cgocall.go:128 +0x5b fp=0xc000132988 sp=0xc000132950 pc=0x4a439b
github.com/bakape/thumbnailer._Cfunc_retrieve_meta(0x7f68500022e0, 0x0, 0x0)
    _cgo_gotypes.go:1222 +0x49 fp=0xc0001329b8 sp=0xc000132988 pc=0x82a7c9
github.com/bakape/thumbnailer.(*FFContext).Meta.func1(0xc00000e940, 0xc0001a83c0, 0x7f68500022e0)
    /srv/www/app.platformdev.com/public_html/golang/src/github.com/bakape/thumbnailer/meta.go:8 +0x5e fp=0xc0001329f8 sp=0xc0001329b8 pc=0x82d27e
github.com/bakape/thumbnailer.(*FFContext).Meta(0xc00000e940, 0x0, 0x0, 0x0, 0x0)
    /srv/www/app.platformdev.com/public_html/golang/src/github.com/bakape/thumbnailer/meta.go:8 +0x49 fp=0xc000132a28 sp=0xc0001329f8 pc=0x82c049
github.com/bakape/thumbnailer.processMedia(0x15b3b00, 0xc0000100c8, 0xc0000648a0, 0x0, 0x0, 0xfa, 0xfa, 0x0, 0x0, 0x0, ...)
    /srv/www/app.platformdev.com/public_html/golang/src/github.com/bakape/thumbnailer/thumbnailer.go:64 +0x110 fp=0xc000132aa0 sp=0xc000132a28 pc=0x82c4c0
github.com/bakape/thumbnailer.Process(0x15b3b00, 0xc0000100c8, 0x0, 0x0, 0xfa, 0xfa, 0x0, 0x5100de, 0xc000132bb0, 0x4cc311, ...)

Accessing member of nil struct cause deadlock when thumbnailer is imported

This is very strange issue and might be caused by tons of reasons and maybe only specific to my system. But I'm just curious, can you reproduce it.

Minimal go program:

package main

import (
	"fmt"
	"github.com/bakape/thumbnailer"
)

type A struct {
	b *B
}

type B struct {
	c int
}

func main() {
	a := A{nil}
	fmt.Println(a.b.c)  // <- should panic here
	fmt.Println(thumbnailer.Dims{})
}

On my system it deadlocks. If I either remove thumbnailer import or .c in print, everything is ok.

Don't malloc a C buffer for image thumbnailing

According to CGo rules, we can point directly to the underlying array. Needs testing.

Port all GM code to FFmpeg and drop GM as a dependancy

GM is buggy shit and we don't need any advanced processing like that for our purposes.

Add support for comicbook archives

Comicbook archives are just zip/rar/7z/tar archives with images inside. Readers usually just take a list of files in archives, sort it in a natural order and take the first image, usually that one is the correct cover image.

This is the list of mimetypes:

application/x-cbz
application/vnd.comicbook+zip

application/x-cbr
application/vnd.comicbook-rar

application/x-cb7
application/x-cbt

I have a wrapper for unarr here https://github.com/gen2brain/go-unarr that can be used (it has a bug with LZMA2 currently) or in native Go https://github.com/mholt/archiver (doesn't have 7z support, but such comic books are rare anyway).

Provide separate API for only retrieving source image properties

Support WebP

Addresses bakape/hydron#49

Corrupt GIF false positives?

sample:

error:
invalid input: Magick: Corrupt image () reported by coders/gif.c:389 (DecodeImage)

Colour corruption of WebM thumbnails with odd dimensions

Doing something wrong with RAW YUV images. Perhaps wrong GraphicsMagick ImageInfo settings?

Add minimal source dimensions counterpart to MaxSourceDims

Avoid extra pass through Go FFI for video thumbnails

We can just change the path to Go -> FFMPEG -> GM -> Go

Rotate JPEG by EXIF

Migrate GM code to libswscale

Less deps, less fragile deps. Need some other way to extract PDF first page. Possibly libpoppler.

Errors when updating to the newest version

Hello, I just updated to the newest version and I get a bunch of error messages when I try to start my go application now. Everything was working perfectly fine before the upgrade.

This is the list of all error messages:

In file included from /usr/local/go/pkg/src/github.com/bakape/thumbnailer/build_flags_gpl.go:10:
/usr/local/go/pkg/src/github.com/bakape/thumbnailer/libimagequant/libimagequant.c:158:30: warning: implicitly declaring library function 'vsnprintf' with type 'int (char *, unsigned long, const char *, struct __va_list_tag *)' [-Wimplicit-function-declaration]
/usr/local/go/pkg/src/github.com/bakape/thumbnailer/libimagequant/libimagequant.c:158:30: note: include the header <stdio.h> or explicitly provide a declaration for 'vsnprintf'
# github.com/bakape/thumbnailer
In file included from init.c:2:
In file included from /usr/local/Cellar/graphicsmagick/1.3.28_1/include/GraphicsMagick/magick/api.h:87:
/usr/local/Cellar/graphicsmagick/1.3.28_1/include/GraphicsMagick/magick/memory.h:17:48: warning: '__alloc_size__' attribute only applies to functions [-Wignored-attributes]
/usr/local/Cellar/graphicsmagick/1.3.28_1/include/GraphicsMagick/magick/common.h:188:70: note: expanded from macro 'MAGICK_FUNC_ALLOC_SIZE_1ARG'
In file included from init.c:2:
In file included from /usr/local/Cellar/graphicsmagick/1.3.28_1/include/GraphicsMagick/magick/api.h:87:
/usr/local/Cellar/graphicsmagick/1.3.28_1/include/GraphicsMagick/magick/memory.h:19:60: warning: '__alloc_size__' attribute only applies to functions [-Wignored-attributes]
/usr/local/Cellar/graphicsmagick/1.3.28_1/include/GraphicsMagick/magick/common.h:188:70: note: expanded from macro 'MAGICK_FUNC_ALLOC_SIZE_1ARG'
# github.com/bakape/thumbnailer
video.c:41:23: warning: address of array 'f->data' will always evaluate to 'true' [-Wpointer-bool-conversion]
video.c:79:59: warning: duplicate 'const' declaration specifier [-Wduplicate-decl-specifier]
video.c:124:39: warning: address of array 'frames[0]->data' will always evaluate to 'true' [-Wpointer-bool-conversion]
# github.com/bakape/thumbnailer
error: invalid value 'c++17' in '-std=c++17'

Can anyone help me with this? I just don't know what to do.

Don't upscale images that fit into the thumbnail

Logo contribution for thumbnailer

Hello!

I want to contribute to open source software by designing logos. If you like, I can design a logo for the restic. What do you think?

I will waiting feedback.

Add VP9 test to assert it works on Debian stable

Getting SIGABRT when processing bad video

Hey @bakape,

I was just wondering how to properly catch a SIGABRT error. I though by using Notify (signals) but that doesn't seem to work properly. any suggestion?

  Assertion desc failed at libswscale/swscale_internal.h:674
  SIGABRT: abort
  PC=0x7f592d86f1f7 m=4 sigcode=0
  signal arrived during cgo execution

Add travis tests

Can reuse the meguca docker image.

Use LTO for C code

unknown thumbnailing error

Thank you for your hard work, I'm trying to use your thumbnail library on OSX but I keep getting an error "unknown thumbnailing error" while trying to generate from a mp4. Any ideas?

Improper file extension detection

Compilation error

When I try to run Meguca's make update_deps (using the latest version of the master branch), I get this error.

Go version: go1.9 linux/amd64
Distro: Debian 9

I have all the dependencies installed.

Error:

github.com/bakape/thumbnailer
# github.com/bakape/thumbnailer
/root/go/src/github.com/bakape/thumbnailer/ffmpeg.go:157: constant 18446744072328293384 overflows C.int
/root/go/src/github.com/bakape/thumbnailer/ffmpeg.go:178: constant 18446744072328293384 overflows C.int
Makefile:74: recipe for target 'update_deps' failed
make: *** [update_deps] Error 123

Add flag for WebP thumbnail output

More relevant, as more software supports it now.

Thumbnail video without loading entire file.

Hey (again),

I was wondering if it would be possible to only load the first x bytes of a video in memory, instead of the whole file. When thumbnailing a large video it's very memory intensive., and might take some time.

Investigate, if image source buffers could be passed from Go without copy

Might require LockOSThread().

ffmpeg: Invalid data found when processing input

Triggered by:

Thank you.

Thank you for publishing this awesome library. You don't know how much you helped me with my personal project.

Please add a donations link, so we can financially contribute.

Possible to add API to extract nth frame from a video ?

Currently the processVideo function returns just the first frame from a video for the thumbnail. I would like to be able to specify the timestamp of the frame I want to return ?

Is this something that can be added ?

Thanks.