snickers/snickers

:chocolate_bar: An open source alternative to the video cloud encoding services.

Snickers is an open source alternative to the existent cloud encoding services. It is a HTTP API that encode videos.

Setting Up

First make sure you have Go and FFmpeg with --enable-shared installed on your machine. If you don't know what this means, look at how the dependencies are being installed on our Dockerfile.

Download the dependencies:

$ make build

You can store presets and jobs on memory or MongoDB. On your config.json file:

• For MongoDB, set DATABASE_DRIVER: "mongo" and MONGODB_HOST: "your.mongo.host"
• For memory, just set DATABASE_DRIVER: "memory" and you're good to go.

Please be aware that in case you use memory, Snickers will persist the data only while the application is running.

Run!

$ make run

Running tests

Make sure you have mediainfo installed and a local instance of MongoDB running.

$ make test

Using the API

Check out the Wiki to learn how to use the API.

Contributing

1. Fork it
2. Create your feature branch: git checkout -b my-awesome-new-feature
3. Commit your changes: git commit -m 'Add some awesome feature'
4. Push to the branch: git push origin my-awesome-new-feature
5. Submit a pull request

License

This code is under Apache 2.0 License.

snickers's Issues

s3 downloader: track download progress

Create another layer between the handlers and db

We don't want logic inside the handlers

ffmpeg encoder: raise and handle the errors

use LOGFILE variable from config.json

we're not using LOGFILE on rest/helpers.go yet

trigger an error when updating an non-existent preset

rest: create json handler to add json headers

support webm outputs

ffmpeg encoder: track encoding progress

db: create a mongodb driver

Return REST errors based on HTTP Statuses

Refs 7f7c629#diff-30f0210fd2bccd32f37891601212d7ddR27

Job should be a "first-class" object/struct

Instead of having helpers to change jobs we should have methods that change the job itself

add http port configuration

encoding: support source's aspect ratio by setting only width or height

examples and docs: lowercase all values of JSON

QuickTime TC sources are not working

the source on http://flv.io/comingsoon.mov breaks the ffmpeg_encoder driver.

General
Complete name                            : comingsoon.mov
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt  
File size                                : 1.63 MiB
Duration                                 : 1s 667ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 8 180 Kbps
Encoded date                             : UTC 2016-06-07 19:02:57
Tagged date                              : UTC 2016-06-07 19:03:03
Writing library                          : Apple QuickTime
©TIM                                     : 00:00:00:00
©TSC                                     : 30
©TSZ                                     : 1

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings, CABAC                   : No
Format settings, ReFrames                : 2 frames
Format settings, GOP                     : N=1
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1s 667ms
Bit rate mode                            : Variable
Bit rate                                 : 8 149 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 30.000 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.131
Stream size                              : 1.62 MiB (100%)
Language                                 : English
Encoded date                             : UTC 2016-06-07 19:02:57
Tagged date                              : UTC 2016-06-07 19:03:02
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Other
ID                                       : 2
Type                                     : Time code
Format                                   : QuickTime TC
Duration                                 : 1s 667ms
Time code of first frame                 : 00:00:00:00
Time code, striped                       : Yes
Language                                 : English
Encoded date                             : UTC 2016-06-07 19:03:02
Tagged date                              : UTC 2016-06-07 19:03:02

Fix golint warnings from Go Report Card

https://goreportcard.com/report/github.com/flavioribeiro/snickers

Retrieve jobs on db

tests: lowercase all JSON values

create a s3 uploader

Store jobs on db

decrease build time on Travis-CI

we're compiling ffmpeg every time we trigger a new job on travis. it would be faster if we had the compiled binaries somewhere and we could just download them.

Remove ClearDatabase() method

It's only being used for tests, we should have a FakeTestDB that implements all the stuff that memory has or figure out another way to do this

cleanup swap directory at the end of each execution

create config module to handle environment variables and configs

Bug when listing presets

Result:

$ curl http://localhost:8080/presets
[{"name":"test","video":{},"audio":{}} {"name":"test2","video":{},"audio":{}}]

Expected:

$ curl http://localhost:8080/presets
[{"name":"test","video":{},"audio":{}}, {"name":"test2","video":{},"audio":{}}]

support e-mail "uploader"

we should support e-mail, saving the output of the job on S3 with download permissions and send the link to the job's owner, the same way encoding.com does

create swagger file for the api endpoints

lib: remove job updates from the drivers and put on lib pipeline

drivers don't need to import db/types of snickers, they should only have one job. The pipeline on lib/lib.go should take care of this.

Create a db selector on db module

We want to create an instance of the db based on some kind of configuration (memory, mongodb, redis..)

support HLS output

refs #40

we should group all HLS presets of a given job in a single output with a master playlist that maps the levels to the presets.

trigger an error when updating an non-existent job

support multiple presets on a job

One job should be able to handle more than one preset. This way we could create a job which outputs both WebM and MP4 renditions, for example.

Define job statuses

rest: use http.Error instead of the HTTPError helper

https://github.com/flavioribeiro/snickers/blob/master/rest/helpers.go#L12-L15

transform downloaders, encoders and uploaders into interfaces

Database methods should be able to raise errors

Transform Database in interface

Fix skipped tests

ffmpeg encoder: use the presets information on the encoding process

create a s3 downloader

move project to github.com/snickers

• fix imports
• fix build & badges

return API documentation on / location

Run same db tests for different drivers

We should run the same test against differente test drivers (memory, mongo, redis?, etc)

s3 uploader: track upload progress

refs aws/aws-sdk-go#279

move profile and profile level attributes from Preset to VideoPreset

Drone.io: Use SVG badge on Readme

create encoding driver using Gstreamer

refs https://github.com/ziutek/gst/network

fix LocalSource and LocalDestination paths

We should create a /[SWAP_DIRECTORY]/[jobID]/src/ and /[SWAP_DIRECTORY]/[jobID]/dst/ folders to drop the files when downloading/encoding/uploading

add logging

right now only http requests are being logged, we need to have more information about what's happening during the job execution

use templates for mediainfo

refs https://sourceforge.net/p/mediainfo/discussion/297610/thread/d89bcaff/