intel/libyami

Yet Another Media Infrastructure. it is core part of media codec with hardware acceleration, it is yummy to your video experience on Linux like platform.

DISCONTINUATION OF PROJECT

This project will no longer be maintained by Intel.
Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.
Intel no longer accepts patches to this project.
If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.

libyami

Yet Another Media Infrastructure.

It is YUMMY to your video experience on Linux like platform.

Yami is core building block for media solution. it parses video stream

and decodes them leverage hardware acceleration.

• Copyright (c) 2010, The WebM Project authors.

• Copyright (C) 2011-2018 Intel Corporation

• Copyright (C) 2015-2016 Alibaba

License

libyami libraries are available under the terms of the

Apache License 2.0

Overview

libyami consists of several libraries:

• codecparsers: it is bit stream parser,
• common: common objects/operation to work with vaapi (hw acceleration interface)
• decoder: video decoder implementation
• encoder: video encoder implementation
• vpp: video post process implementation

Features

• MPEG-2, VC-1, WMV 9 (WMV3), H.264, HEVC (H.265), VP8, VP9, and JPEG ad-hoc decoders
• H.264, HEVC (H.265), VP8, VP9, and JPEG ad-hoc encoders
• Sharpening, Denoise, Deinterlace, Hue, Saturation, Brightness, Contrast, CSC and scaling

Requirements

Hardware requirements

• Intel Sandybridge, Ivybridge, Haswell, Broadwell, Skylake, Kaby Lake (HD Graphics)
• Intel Bay Trail, Braswell, Apollo Lake, Gemini Lake

Sources

Git repository for work-in-progress changes is available at:

https://github.com/01org/libyami

Demos, Examples and Test Applications

The libyami-utils project provides various example, test and demo applications that use libyami. For more details, please refer to

https://github.com/01org/libyami-utils

Simple api demo application

https://github.com/01org/libyami-utils/blob/master/examples/simpleplayer.cpp

FFmpeg integration

You can refer to https://github.com/01org/ffmpeg_libyami for FFmpeg integration.

You can report FFmpeg related issue to https://github.com/01org/ffmpeg_libyami/issues

Build instructions

https://github.com/01org/libyami/wiki/Build

Docs

http://01org.github.io/libyami_doxygen/index.html

Testing

Unit Tests

The gtest framework library https://github.com/google/googletest is required in order to write and compile unit tests in libyami. To make it convenient to use, we add gtest source to subdirectory gtestsrc. The gtest documentation can be found in their source tree under docs (online or in the subdirectory gtestsrc).

To build gtest and enable the unit tests, when configuring libyami you need to specify:

--enable-tests

Contributing

Create pull request at https://github.com/01org/libyami/compare

Code style

https://github.com/01org/libyami/wiki/Coding-Style

Review process

Create pull requests at https://github.com/01org/libyami/compare

We highly recommend that unit tests accompany your contributed patches. See "Testing" section above. However, we do understand that not everything can be tested by a unit test. So use your best judgement to determine if a unit test is appropriate for your contribution. The maintainer(s) reserve the right to refuse submission's without unit tests, when reasonable, or if a submission causes existing unit tests to regress.

Mail list

[email protected]

Reporting Bugs

Bugs can be reported in the github system at:

https://github.com/01org/libyami/issues/new

Reporting a security issue

Please mail to [email protected] directly for security issue

FAQ

https://github.com/01org/libyami/wiki/FAQ

libyami's Issues

v4l2decode can't show frame when use render mode 1/2/3

v4l2decode -i /path/to/h264 -m 1/2/3
There is no frame display when use render mode 1/2/3

omxh264enc need to support more parameter

omxh264enc should add more parameter control, please reference the gstreamer vaapi's parameter
"vaapiencode_h264 rate-control=cqp keyframe-period=30 max-bframes=2 init-qp=28 cabac=true"
gstreamer vaapi parameter information:

1. rate-control : Rate control mode
   flags: readable, writable
   Enum "GstVaapiRateControlH264" Default: 1, "cqp"
   (1): cqp - Constant QP
   (2): cbr - Constant bitrate
   (4): vbr - Variable bitrate
   (5): vbr_constrained - Variable bitrate - Constrained
2. keyframe-period : Maximal distance between two keyframes (0: auto-calculate)
   flags: readable, writable
   Unsigned Integer. Range: 1 - 300 Default: 30
3. max-bframes : Number of B-frames between I and P
   flags: readable, writable
   Unsigned Integer. Range: 0 - 10 Default: 0
4. init-qp : Initial quantizer value
   flags: readable, writable
   Unsigned Integer. Range: 1 - 51 Default: 26
5. cabac : Enable CABAC entropy coding mode
   flags: readable, writable
   Boolean. Default: false

Decoding some VP8 files failed by low ssim value

#1. Testing Env:

libva: (staging)35e70cb9b9c77dfb99fb370e319ed501f0c31b17
intel driver: (staging)fbbe401aa28a0b3859d587ef08f0df15a2f7c8f2
libyami: (master)585059310594e89baa69d057f15c39476122fa5a
omxil-core: (staging-yami)cb613287332584a94ba004dacdaf54f00f89ba3c
omx-components: (staging-yami)51f6bd9c62546baa32a664e6334ca3797b727bd2
gst-omx: (master)d52c613d56a43ba4fd2be03139c9a1a6330a7d75
#2. Testing Steps:

1.run command "gst-launch-1.0 filesrc location=/home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/dotivf/vp80-03-segmentation-1436.ivf '!' ivfparse '!' omxvp8dec '!' vaapisink sync=false"
2.run command "/home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/vpxdec -o vp80-03-segmentation-1436.i420 --i420 /home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/dotivf/vp80-03-segmentation-1436.ivf"
3.judge two YUV file from above command, get the ssim.
e.g.
filename  Y SSIM extreme Y SSIM average U SSIM extreme U SSIM average V SSIM extreme V SSIM average  
'vp80-03-segmentation-1425.ivf 0.047145 0.809429 0.069556 0.813911 0.187299 0.837460'
'vp80-03-segmentation-1436.ivf 0.038759 0.359173 0.508027 0.672018 0.582592 0.721728'

first run pipeline crash after set gstreamer env

when gstreamer envirenment is set. and then run libyami pipeline:"gst-launch-1.0 filesrc location=/root/Videos/Base_Ext_Main_profile/MR2_TANDBERG_E.264 ! h264parse ! omxh264dec ! videoconvert ! ximagesink," it will crash.

Makefile.am doesn't add v4l2 folder when enabling v4l2-glx

v4l2-glx can be enabled without v4l2, but the SUBDIR v4l2 is not added to compilation. by adding v4l2-glx to configure v4l2 sub-dir should be added to compilation.

[BDW][Media][Encode]startAborted when doing unit test for v4l2 encode

1.Testing environment:
libva: (master) ccd93de5a707e92a629cccd595757c8d436fa3cc
intel driver: (master) 24cba20a119c96556ae4dc9a90043896ea70e56
libyami: (master) ddd2a83
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command:
"/home/zliu50x/libyami/omx/libyami/tests/v4l2encode -i /root/media_tools/encoder/encoderbitstreams/stockholm_1280x720_604.yuv -s I420 -W 1280 -H 720 -c AVC -o stockholm_1280x720_604.yuv.enc.264"
(2)Check the dmesg and get the information:
libva info: VA-API version 0.36.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_36
libva info: va_openDriver() returns 0
lt-v4l2encode: v4l2_encode.cpp:119: virtual bool V4l2Encoder::inputPulse(int32_t): Assertion `m_inputFrames[index].bufAvailable' failed.
startAborted

[BDW][Media][Encode]blocking when doing 1080p encoding test

1.Test Env :
libva: (master) c356638098e55453d2fac8bbb92eafc8cc32d6f4
intel driver: (staging)98b28c5b4e7f6c00270dd146974be97ce6f321f4
libyami: (master)1c52de73a6540e766806f626a5a4c72bf3e3ed9e
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 1f671b2cfa6d2709501f324fa7ffd81cb6255554
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a241
2.Testing steps:
(1)run the command
" /home/zliu50x/libyami/omx/libyami/tests/h264encode -i /root/media_tools/encoder/encoderbitstreams/1080p_blue_sky.yuv -s I420 -W 1920 -H 1080 -c AVC -o 1080p_blue_sky.yuv.enc.264"
(2)check the dmesg and get the information :
"libva info: VA-API version 0.36.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_36
libva info: va_openDriver() returns 0"

[BDW][Media][Decode]yamidecode blocks at XGetWindowAttributes() after libva dlopen

1. Test Env:
   Libva: (master)e97ac9e78cd475a13e722c455e34d5d39d0f059d
   Libva_intel_driver: (master)43bd81abdde40b50ac71f6f44eb04e4eaf5af5f6
   libyami: (master) 8cc7e42
   2..Test command:
   /home/wenxin1/libyami/omx/libyami/tests/yamidecode -i /media/ts/H264_176x144_frame320.mp4 -m 1
   input file: /media/ts/H264_176x144_frame320.mp4, renderMode: 1
   libva info: VA-API version 0.37.0
   libva info: va_getDriverName() returns 0
   libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
   libva info: Found init function __vaDriverInit_0_37
   libva info: va_openDriver() returns 0
   X Error of failed request: BadWindow (invalid Window parameter)
   Major opcode of failed request: 3 (X_GetWindowAttributes)
   Resource id in failed request: 0x2a4f740
   Serial number of failed request: 14
   Current serial number in output stream: 15

MR8_BT_B.264 block bug

the pipeline block and log is as follows:
gst-launch-1.0 filesrc location=/root/Videos/Base_Ext_Main_profile/MR8_BT_B.h264 ! h264parse ! omxh264dec ! videoconvert ! ximagesink
Setting pipeline to PAUSED ...
omxil-core info: Added component libOMXVideoDecoderAVC.so to list
omxil-core info: Added component libOMXVideoDecoderVP8.so to list
omxil-core info: module libOMXVideoDecoderAVC.so successfully loaded
omxil-core info: module libOMXVideoDecoderVP8.so successfully loaded
omxil-core info: get handle of component OMX.Intel.VideoDecoder.AVC successfully
Pipeline is PREROLLING ...
omxil-core error: No more enumeration.

libva info: VA-API version 0.34.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/1.0/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_34
libva info: va_openDriver() returns 0
omxil-core error: Get all the buffers from libmix
omxil-core error: FillRenderBuffer() not ready?
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
libyami error: found no long-term reference picture with LongTermPicNum = 1
libyami error: found no long-term reference picture with LongTermPicNum = 3
libyami error: found no long-term reference picture with LongTermPicNum = 1
libyami error: found no long-term reference picture with LongTermPicNum = 3
libyami error: found no long-term reference picture with LongTermPicNum = 0
libyami error: found no long-term reference picture with LongTermPicNum = 3
libyami error: found no long-term reference picture with LongTermPicNum = 1
libyami error: found no long-term reference picture with LongTermPicNum = 2

libyami doesn't build with libva-1.3.1

Please add a check on the configure for the libva version. If version is not supported, configure should say so.

It would be good to support version mentioned libva-1.3.1 onward.

H264 video playback becomes stuck after several minutes

for high resolution h264 stream, video playback becomes stuck (almot freeze) after playback for several minutes.
it is reproduced on either chromeos or ubuntu(with gst-omx).

[BDW][Media][Decode] md5 value is not equal to the value using software tools when doing YamiUnitTest for Vp9 decoding

1.Testing Env:
Libva: (staging)9d128cf241830f7ead49fa5b104248b44c680873
Libva_intel_driver: (hybrid_vp9_rebase2)a343161979984b4307a015233e79380b55ab2d4f
Ffmpeg: (master)037b527d62500cfde89d3bda5d4ac04ce8b8d959
Libyami: (master)8cc7e42ca49ea407b7e49db0e200f794e20fc49c

2.Test command:
(1)use following command to get dump yuv file
/home/wenxin1/libyami/omx/libyami/tests/yamidecode -i /home/wenxin/VP9-Hybrid/vp90-2-05-resize.ivf -m 0 -f I420
(2)use following command to get reference yuv' file
/GFX/Test/Infrastructure/infrastructure/tests/vp8_dec_cfm/vpxdec/ --i420 --md5 -o vp90-2-05-resize.ivf /home/wenxin/VP9-Hybrid/vp90-2-05-resize.ivf
(3)compare the md5 value of two methods
de1c64f80d33db015b1a9da21e59bff0 vp90-2-05-resize.ivf(libyami)
d1c46d62ef713cb7f6fd5024c510f2da vp90-2-05-resize.ivf(vpxdec)

low ssim when decoding vp80-03-segmentation-1436.ivf

low ssim when decoding vp80-03-segmentation-1436.ivf.

regression fail, test command:
gst-launch-1.0 filesrc location=/media/vp8/vp80-03-segmentation-1436.ivf '!' ivfparse '!' omxvp8dec '!' vaapisink sync=false
judge the YUV file decoding by soft decoder to get the ssim.

result:
filename Y SSIM extreme Y SSIM average U SSIM extreme U SSIM average V SSIM extreme V SSIM average
vp80-03-segmentation-1436.ivf 0.118283 0.559141 1.000000 1.000000 1.000000 1.000000 1 2 0

There is an error when play an odd number of resolution about vp8

omxil-core error: seems to be up layer bug, no room for nv12 data copy, need 37697, have 37537

Decoding some h264 files failed by frame loss

#1. Testing Env:

libva: (staging)35e70cb9b9c77dfb99fb370e319ed501f0c31b17
intel driver: (staging)fbbe401aa28a0b3859d587ef08f0df15a2f7c8f2
libyami: (master)585059310594e89baa69d057f15c39476122fa5a
omxil-core: (staging-yami)cb613287332584a94ba004dacdaf54f00f89ba3c
omx-components: (staging-yami)51f6bd9c62546baa32a664e6334ca3797b727bd2
gst-omx: (master)d52c613d56a43ba4fd2be03139c9a1a6330a7d75
#2. Testing Steps:

1.run command "gst-launch-1.0 filesrc location=/root/media_tools/decoder/Base_Ext_Main_profile/CAWP5_TOSHIBA_E.264 '!' h264parse '!' omxh264dec '!' vaapisink sync=false"
2.run command "/root/media_tools/decoder/ldecod.dbg.exe -i CAWP5_TOSHIBA_E.264 -o CAWP5_TOSHIBA_E.264.ref.yuv"
3.judge two YUV file from above command, can find that there are 7 frames loss

[BDW][Media][Decode]the parameter (-w 2) not work when doing Unit test for pure C API interface

1.Testing environment:
libva: (master) ccd93de5a707e92a629cccd595757c8d436fa3cc
intel driver: (master) 24cba20a119c96556ae4dc9a90043896ea70e56
libyami: (master) ddd2a83
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c

2.Testing steps:
(1)run the command
"./decodecapi -i /home/wenxin1/libyami/Base_Ext_Main_profile/CVFI2_Sony_H.jsv -w 2"
(2)Check the dmesg and get the information :
-w wait before quit: 0:no-wait, 1:auto(jpeg wait), 2:wait
the parameter (-w 2) not work when doing Unit test for pure C API interface

vp80-03-segmentation-1425.ivf cannot decode by omxvp8dec

vp80-03-segmentation-1425.ivf cannot decode by omxvp8dec.
test comand:
gst-launch-1.0 filesrc location=/home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/dotivf/vp80-03-segmentation-1425.ivf '!' ivfparse '!' omxvp8dec '!' vaapisink sync=false

log:
[root@x-bdwmedia gst-omx]# gst-launch-1.0 filesrc location=/home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/dotivf/vp80-03-segmentation-1425.ivf '!' ivfparse '!' omxvp8dec '!' vaapisink sync=false
libva info: VA-API version 0.35.2
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_35
libva info: va_openDriver() returns 0
Setting pipeline to PAUSED ...
omxil-core info: Added component libOMXVideoDecoderAVC.so to list
omxil-core info: Added component libOMXVideoDecoderVP8.so to list
omxil-core info: Added component libOMXVideoDecoderMJPEG.so to list
omxil-core info: Added component libOMXVideoEncoderAVC.so to list
omxil-core info: module libOMXVideoDecoderAVC.so successfully loaded
omxil-core info: module libOMXVideoDecoderVP8.so successfully loaded
omxil-core info: module libOMXVideoDecoderMJPEG.so successfully loaded
omxil-core info: module libOMXVideoEncoderAVC.so successfully loaded
omxil-core info: get handle of component OMX.Intel.VideoDecoder.VP8 successfully
Pipeline is PREROLLING ...
omxil-core error: No more enumeration.
libva info: VA-API version 0.35.2
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_35
libva info: va_openDriver() returns 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
omxil-core info: Flushing port# 0x1.

Decoding some h264 files failed by low ssim value

#1. Testing Env:

libva: (staging)35e70cb9b9c77dfb99fb370e319ed501f0c31b17
intel driver: (staging)fbbe401aa28a0b3859d587ef08f0df15a2f7c8f2
libyami: (master)585059310594e89baa69d057f15c39476122fa5a
omxil-core: (staging-yami)cb613287332584a94ba004dacdaf54f00f89ba3c
omx-components: (staging-yami)51f6bd9c62546baa32a664e6334ca3797b727bd2
gst-omx: (master)d52c613d56a43ba4fd2be03139c9a1a6330a7d75
#2. Testing Steps:

1.run command "gst-launch-1.0 filesrc location=/root/media_tools/decoder/Base_Ext_Main_profile/CAWP5_TOSHIBA_E.264 '!' h264parse '!' omxh264dec '!' vaapisink sync=false"
2.run command "/root/media_tools/decoder/ldecod.dbg.exe -i CAWP5_TOSHIBA_E.264 -o CAWP5_TOSHIBA_E.264.ref.yuv"
3.judge two YUV file from above command, get the ssim.
e.g.
filename  Y SSIM extreme Y SSIM average U SSIM extreme U SSIM average V SSIM extreme V SSIM average  
'CAWP5_TOSHIBA_E.264 0.773177 0.997300 0.955969 0.999476 0.961375 0.999540
'CANL4_SVA_B.264     0.892528 0.993678 0.990116 0.999419 0.991019 0.999472

*if don't judge the last 16 frame, most h264 file can git hight ssim(1.000000)

VP9 decode does not work on ChromeOS with Libyami

After running VP9 decode with hybrid driver on ChromeOS, I found some changes were missing in Libyami. I have added the changes to this pull request:
#173

After these changes, VP9 decode works fine on ChromeOS. Please review and merge.

Playing back 4k video is not smooth when using omxh264dec

#1. Testing Env:

libva: (staging)35e70cb9b9c77dfb99fb370e319ed501f0c31b17
intel driver: (staging)fbbe401aa28a0b3859d587ef08f0df15a2f7c8f2
libyami: (master)585059310594e89baa69d057f15c39476122fa5a
omxil-core: (staging-yami)cb613287332584a94ba004dacdaf54f00f89ba3c
omx-components: (staging-yami)51f6bd9c62546baa32a664e6334ca3797b727bd2
gst-omx: (master)d52c613d56a43ba4fd2be03139c9a1a6330a7d75
#2. Testing Steps:

1.run command "gst-launch-1.0 filesrc location=./4K_2013.mp4 ! qtdemux ! h264parse ! omxh264dec ! vaapisink"
2.check the video output, not smooth.
3.use command "gst-launch-1.0 filesrc location=./4K_2013.mp4 ! qtdemux ! vaapidecode ! vaapisink", cannot find this problem.
* use 720p video file to play back, cannot find this problem

There is no frame display when play alphaconformanceG.264

There is no frame display when play alphaconformanceG.264

alphaconformanceG.264 couldn't find associated sequence parameter set with id: 0

On BDW, 4 Of 171 stream fails
CVSE3_Sony_H.jsv
CVSEFDFT3_Sony_E.jsv
CVSE2_Sony_B.jsv
alphaconformanceG.264

The omxh264dec miss some frames at the end of the file.

Use the next commandline to dump yuv data, and use BeyondCompare to compare the yuv of software decode with the yuv of hardward decode.

the file BA1_FT_C.264 miss 7 frames，
BA3_SVA_C.264 miss 17 frames,
BAMQ1_JVC_C.264 miss 17 frames.

gst-launch-1.0 filesrc location=/root/Videos/h264/Base_Ext_Main_profile/BA1_FT_C.264 ! h264parse ! avdec_h264 ! filesink location=av.yuv
gst-launch-1.0 filesrc location=/root/Videos/h264/Base_Ext_Main_profile/BA1_FT_C.264 ! h264parse ! omxh264dec ! videoconvert ! video/x-raw,format=I420 ! filesink location=omx.yuv

[BDW][Media][Encode]garbage data display when doing encoding test with camera -s <fourcc: NV12>

1.Testing environment:
libva: (master) ccd93de5a707e92a629cccd595757c8d436fa3cc
intel driver: (master) 24cba20a119c96556ae4dc9a90043896ea70e56
libyami: (master) ddd2a83
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command
"./yamiencode -i /dev/video0 -s NV12 -W 1280 -H 720 -c AVC -b 41472000 -f 30 -N 50 -o out.h264"
(2)Check the dmesg and get the information :
garbage data display when doing encoding test with camera .

[BDW][Media][Decode]no video output when doing decoding test for the texture rendering mode(-m 4)

1.Testing environment:
libva: (master) ccd93de5a707e92a629cccd595757c8d436fa3cc
intel driver: (master) 24cba20a119c96556ae4dc9a90043896ea70e56
libyami: (master) ddd2a83
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command
"./yamidecode -i /home/wenxin1/h264clips/1080p_blue_sky.yuv.enc.264 -m 4"
(2)Check the dmesg and get the information :
find that no video output when doing decoding test for the texture rendering mode(-m 4).

[SKL][media][Encode] frame numbers in the head of file are not same with real frames

1.Testing environment:
libva: (staging) d9626b77f865ce7bdbc8c78988a86d3e4f55d67b
intel driver: (staging) a8ca5f911c7c19b5ede3f96ac6113606b9dd4d95
libyami: (master) 7730d21
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
.2.Test command
(1)/home/zliu50x/libyami/omx/libyami/tests/yamiencode -i /home/encoderbitstreams/encoderbitstreams/1080p_blue_sky.yuv -s I420 -W 1920 -H 1080 -c VP8 -o 1080p_blue_sky.yuv.enc.vp8
(2)using vega test tools to verify the correctness,found frame numbers in the head of file were not same with real frames

Decoding some VP8 files failed by frame loss

#1. Testing Env:

libva: (staging)35e70cb9b9c77dfb99fb370e319ed501f0c31b17
intel driver: (staging)fbbe401aa28a0b3859d587ef08f0df15a2f7c8f2
libyami: (master)585059310594e89baa69d057f15c39476122fa5a
omxil-core: (staging-yami)cb613287332584a94ba004dacdaf54f00f89ba3c
omx-components: (staging-yami)51f6bd9c62546baa32a664e6334ca3797b727bd2
gst-omx: (master)d52c613d56a43ba4fd2be03139c9a1a6330a7d75
#2. Testing Steps:

1.run command "gst-launch-1.0 filesrc location=/home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/dotivf/vp80-03-segmentation-1425.ivf '!' ivfparse '!' omxvp8dec '!' vaapisink sync=false"
2.run command "/home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/vpxdec -o vp80-03-segmentation-1425.i420 --i420 /home/GFX/build/testsuite/Media_tools/media_tools/vp8_dec_cfm/dotivf/vp80-03-segmentation-1425.ivf"
3.judge two YUV file from above command, can find that there are 20 frames loss

h264: support process for gaps in frame_num

short-term reference can not be finded when play back some h264 clips.the reason is that there are gaps in frame_num.
Clip: MR3_TANDBERG_B.264

[BDW][Media][Decode]libyami can not work when doing v4l2decode for .mp4 format with (-m 0/3/4)

1.Test Env:
Libva: (master)e97ac9e78cd475a13e722c455e34d5d39d0f059d
Libva_intel_driver: (master)43bd81abdde40b50ac71f6f44eb04e4eaf5af5f6
Libyami: (master)8cc7e42ca49ea407b7e49db0e200f794e20fc49c

2.Test command:
(1)use following command to get dump yuv file
/home/wenxin1/libyami/omx/libyami/tests/v4l2decode -i /media/ts/H264_176x144_frame320.mp4 -m 0
the result is that:
libyami can not work

playback FM1_FT_E.264 exit with H264Baseline profile

playback FM1_FT_E.264 1-200 frames with H264ConstrainedBaseline Profile. And then sps and pps updated. Profile of this clip changed to H264Baseline Profile. But the intel-driver can not support H264Baseline profile but support H264ConstrainedBaseline profile.
So the decoder ends.

So this is not issue of yami decoder.

some clips get low ssim value when using omxmjpegdec to decode

low ssim value and frame loss happened when using omxmjpegdec to decode

test comamnd:

1. use following command to get dump yuv file
   gst-launch-1.0 filesrc location=/root/media_tools/encoder/encoderbitstreams/$j '!' videoparse format=i420 width=${WIDTH} height=${HEIGHT} '!' jpegenc '!' omxmjpegdec '!' vaapisink
2. use following command to get reference yuv' file
   gst-launch-1.0 filesrc location=/root/media_tools/encoder/encoderbitstreams/$j '!' videoparse format=i420 width=${WIDTH} height=${HEIGHT} '!' jpegenc '!' jpegdec '!' vaapisink
3. compare yuv and yuv' file to get ssim. and find loss ssim and frame loss

freeze issue with HD clips on BDW

For example this one freezes occasionally when playing:
http://otc-chromeos-media.jf.intel.com/media/video/mp4/sintel-1080p-30fps-20Mbs-mp4II.mp4

.264 file encoding by omxh264enc get low psnr when comparing with windows driver data

1. Testing Env:
   libva: (staging)35e70cb9b9c77dfb99fb370e319ed501f0c31b17
   intel driver: (staging)fbbe401aa28a0b3859d587ef08f0df15a2f7c8f2
   libyami: (master)585059310594e89baa69d057f15c39476122fa5a
   omxil-core: (staging-yami)cb613287332584a94ba004dacdaf54f00f89ba3c
   omx-components: (staging-yami)51f6bd9c62546baa32a664e6334ca3797b727bd2
   gst-omx: (master)d52c613d56a43ba4fd2be03139c9a1a6330a7d75
2. Testing Steps:
   1.run command "gst-launch-1.0 -v filesrc location=/root/media_tools/encoder/encoderbitstreams/CITY_704x576_30_orig_01.yuv '!' videoparse format=i420 width=704 height=576 '!' videoconvert '!' omxh264enc '!' h264parse '!' filesink location=/root/media_tools/encoder/encoderbitstreams/CITY_704x576_30_orig_01.yuv.enc.264"
   to get encoding .264 file
   2.run command "/opt/X11R7/ffmpeg/bin/ffmpeg -i CITY_704x576_30_orig_01.yuv.enc.264 CITY_704x576_30_orig_01.yuv.jm.dump.yuv"
   to get dump YUV file
   3.run command "/root/media_tools/encoder/metrics_calc_lite.ia32e -i1 CITY_704x576_30_orig_01.yuv -i2 CITY_704x576_30_orig_01.yuv.jm.dump.yuv -w 704 -h 576 -nopfm psnr y"
   to get Y psnr=34.67803.
3. with same bitrate, the windows driver date is 37.8722010022

decodecapi can't playback when use render mode 2/3/4/5

decodecapi -i /path/to/h264 -m 2/3/4/5
When use render mode 2/3/4/5, there is an error：
decodehelp.h:358 renderOutputFrames: Assertion ‘0’ failed.

FM1_FT_E.264 playback the nalus before the first slice nalu and the first slice nalu are decoded twice.

[BDW][Media][Encode]YamiUnitTest encoding PSNR is low when comparing with window driver data

1.Test Env:
libva: (master) c356638098e55453d2fac8bbb92eafc8cc32d6f4
intel driver: (staging)98b28c5b4e7f6c00270dd146974be97ce6f321f4
libyami: (master) 1c52de7
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 1f671b2cfa6d2709501f324fa7ffd81cb6255554
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command
"/home/zliu50x/libyami/omx/libyami/tests/h264encode -i /root/media_tools/encoder/encoderbitstreams/720p5994_parkrun_ter.yuv -s I420 -W 1280 -H 720 -c AVC -o 720p5994_parkrun_ter.yuv.enc.264" to get encoding .264 file
(2)run the command
"/opt/X11R7/ffmpeg/bin/ffmpeg -i 720p5994_parkrun_ter.yuv.enc.264 720p5994_parkrun_ter.yuv.jm.dump.yuv" to get dump YUV file
(3)run the command
" /root/media_tools/encoder/metrics_calc_lite.ia32e -i1 /root/media_tools/encoder/encoderbitstreams/720p5994_parkrun_ter.yuv -i2 720p5994_parkrun_ter.yuv.jm.dump.yuv -w 1280 -h 720 -nopfm psnr y"
to get Y psnr=28.38239
3.with same bitrate, the windows driver date is 34.96929

[BDW][Media][Encode]gst-launch-1.0 abort when doing encoding test

1.Test Env:
libva: (master) c356638098e55453d2fac8bbb92eafc8cc32d6f4
intel driver: (staging)98b28c5b4e7f6c00270dd146974be97ce6f321f4
libyami: (master) 1c52de7
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 1f671b2cfa6d2709501f324fa7ffd81cb6255554
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command:
"gst-launch-1.0 -v filesrc location=/root/media_tools/encoder/encoderbitstreams/CITY_704x576_30_orig_01.yuv '!' videoparse format=i420 width=704 height=576 '!' videoconvert '!' omxh264enc '!' filesink location=CITY_704x576_30_orig_01.yuv.enc.264"
(2)Check the dmesg and get the information :
"Setting pipeline to PAUSED ...
omxil-core info: Added component libOMXVideoDecoderAVC.so to list
omxil-core info: Added component libOMXVideoDecoderVP8.so to list
omxil-core info: Added component libOMXVideoDecoderMJPEG.so to list
omxil-core info: Added component libOMXVideoEncoderAVC.so to list
omxil-core info: module libOMXVideoDecoderAVC.so successfully loaded
omxil-core info: module libOMXVideoDecoderVP8.so successfully loaded
omxil-core info: module libOMXVideoDecoderMJPEG.so successfully loaded
omxil-core info: module libOMXVideoEncoderAVC.so successfully loaded
omxil-core info: get handle of component OMX.Intel.VideoEncoder.AVC successfully
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstVideoParse:videoparse0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)704, height=(int)576, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, framerate=(fraction)25/1
omxil-core error: No more enumeration.
omxil-core error: No more enumeration.
omxil-core error: No more enumeration.
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)704, height=(int)576, format=(string)NV12, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, framerate=(fraction)25/1
omxil-core error: No more enumeration.
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:sink: caps = video/x-raw, width=(int)704, height=(int)576, format=(string)NV12, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)704, height=(int)576, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, framerate=(fraction)25/1
omxil-core error: No more enumeration.
omxil-core error: ComponentGetParameter: No handler for index 100663312
libva info: VA-API version 0.36.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_36
libva info: va_openDriver() returns 0
omxil-core error: ComponentGetParameter: No handler for index 100663312
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)704, height=(int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)704, height=(int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1
gst-launch-1.0: vaapiencoder_base.cpp:226: YamiMediaCodec::SurfacePtr YamiMediaCodec::VaapiEncoderBase::createSurface(uint32_t): Assertion `0' failed.
Aborted"

v4l2decode can't playback vp8 and jpg

v4l2decode -i /path/to/vp8 or jpg

[BDW][Media][Decode]1080p Video freeze when doing unit test for v4l2 decode wrapper

1.Testing environment:
libva: (master) ccd93de5a707e92a629cccd595757c8d436fa3cc
intel driver: (master) 24cba20a119c96556ae4dc9a90043896ea70e56
libyami: (master) ddd2a83
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command
"./v4l2decode -i /home/workdir.jpeg_dec/1080p_blue_sky.yuv.jpg -m 3 "
(2)Check the dmesg and get the information :
1080p Video freeze when doing unit test for v4l2 decode wrapper

lack of const

in VideoDecoder class
virtual const VideoRenderBuffer* getOutput(bool draining = false) = 0;
it return "const " value,but
virtual void renderDone(VideoRenderBuffer * renderBuf);
has no "const".

VC-1 (VC1) and WMV 9 (WMV3) support missing

Request SMPTE decoder support for VC-1 (VC1/WVC1) and Windows Media Video 9 (WMV3) decode:

http://en.wikipedia.org/wiki/VC-1

SMPTE 421M, informally known as VC-1, is a video coding format. It was initially developed as a proprietary video format based on WMV 9 (Windows Media Video 9) by Microsoft in 2003 and officially approved as a SMPTE video codec standard on April 3, 2006. VC-1 is today a supported standard found in the Blu-ray Disc specification, Microsoft's Silverlight framework, and the now-discontinued HD-DVD.

[BDW][Media][Decode]segmentation fault when doing yamidecode for .mp4 with no render (-m -1)

1.Test Env:
Libva: (master)e97ac9e78cd475a13e722c455e34d5d39d0f059d
Libva_intel_driver: (master)43bd81abdde40b50ac71f6f44eb04e4eaf5af5f6
Libyami: (master)8cc7e42ca49ea407b7e49db0e200f794e20fc49c

2.Test command:
(1)use following command to get dump yuv file
/home/wenxin1/libyami/omx/libyami/tests/yamidecode -i /media/ts/H264_176x144_frame320.mp4 -m -1
the messege is here:
input file: /media/ts/H264_176x144_frame320.mp4, renderMode: -1
libva info: VA-API version 0.37.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_37
libva info: va_openDriver() returns 0
Segmentation fault

[BDW][Media][Decode]segment fault when doing decoding test

1.Test Env :
libva: (master) c356638098e55453d2fac8bbb92eafc8cc32d6f4
intel driver: (staging)98b28c5b4e7f6c00270dd146974be97ce6f321f4
libyami: (master)1c52de73a6540e766806f626a5a4c72bf3e3ed9e
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 1f671b2cfa6d2709501f324fa7ffd81cb6255554
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command
"gst-launch-1.0 filesrc location= /root/media_tools/decoder/Base_Ext_Main_profile/HPCAMOLQ_BRCM_B.264 ! vaapiparse_h264 ! omxh264dec ! filesink location=HPCAMOLQ_BRCM_B.yuv--gst-debug=*:3"
(2)Check the dmesg and get the information :
Setting pipeline to PAUSED ...
omxil-core info: Added component libOMXVideoDecoderAVC.so to list
omxil-core info: Added component libOMXVideoDecoderVP8.so to list
omxil-core info: Added component libOMXVideoDecoderMJPEG.so to list
omxil-core info: Added component libOMXVideoEncoderAVC.so to list
omxil-core info: module libOMXVideoDecoderAVC.so successfully loaded
omxil-core info: module libOMXVideoDecoderVP8.so successfully loaded
omxil-core info: module libOMXVideoDecoderMJPEG.so successfully loaded
omxil-core info: module libOMXVideoEncoderAVC.so successfully loaded
omxil-core info: get handle of component OMX.Intel.VideoDecoder.AVC successfully
Pipeline is PREROLLING ...
omxil-core error: No more enumeration.
libva info: VA-API version 0.36.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_36
libva info: va_openDriver() returns 0
omxil-core error: Get all the buffers from libmix
omxil-core error: FillRenderBuffer() not ready?
Caught SIGSEGV
#0 0x00007faf18bf0f7d in poll () at ../sysdeps/unix/syscall-template.S:81
#1 0x00007faf193349d4 in g_main_context_poll (priority=2147483647, n_fds=2,
#2 g_main_context_iterate (context=0x115b1f0, block=block@entry=1,
#3 0x00007faf19334d02 in g_main_loop_run (loop=0x1159e30) at gmain.c:3986
#4 0x00007faf19fbdf14 in gst_bus_poll (bus=bus@entry=0x11276c0,
#5 0x00000000004045b8 in event_loop (pipeline=0x1148080,
#6 0x0000000000403611 in main (argc=11, argv=0x7fffa417e368)

Spinning. Please run 'gdb gst-launch-1.0 29983' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

[BDW][Media][Decode]YamiUnitTest JPEG decoding ssim value is low

1.Testing Env:
libva: (master) c356638098e55453d2fac8bbb92eafc8cc32d6f4
intel driver: (staging)98b28c5b4e7f6c00270dd146974be97ce6f321f4
libyami: (master) ea0c742
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c

2.Test command:
(1)use following command to get dump yuv file
/home/zliu50x/libyami/omx/libyami/tests/yamidecode -i /home/workdir.jpeg_dec/1080p_blue_sky.yuv.jpg -m 0 -f I420
(2)use following command to get reference yuv' file
gst-launch-1.0 filesrc location=/home/workdir.jpeg_dec/1080p_blue_sky.yuv.jpg '!' 'jpegparse!' jpegdec '!' videoconvert '!' video/x-raw, format=I420 '!' filesink location=dump.yuv
(3)compare yuv and yuv' file to get ssim. and find ssim value is low
yuv filename y_min_ssim y_ave_ssim u_min_ssim u_ave_ssim v_min_ssim v_ave_ssim
1080p_blue_sky.yuv.jpg 0.997193 0.997702 0.979815 0.986921 0.985011 0.991386
MOBILE_352x288_30_orig_01.yuv.jpg 0.999048 0.999239 0.983513 0.985694 0.983802 0.985874

omxil-core error: programmer bug, uv data copy from 0x33434d49 to 0x13 format hasn't supported yet

#1. Testing Env:

libva: (staging)35e70cb9b9c77dfb99fb370e319ed501f0c31b17
intel driver: (staging)fbbe401aa28a0b3859d587ef08f0df15a2f7c8f2
libyami: (master)585059310594e89baa69d057f15c39476122fa5a
omxil-core: (staging-yami)cb613287332584a94ba004dacdaf54f00f89ba3c
omx-components: (staging-yami)51f6bd9c62546baa32a664e6334ca3797b727bd2
gst-omx: (master)d52c613d56a43ba4fd2be03139c9a1a6330a7d75
#2. Testing Steps:

1.run command "gst-launch-1.0 filesrc location=/root/media_tools/encoder/encoderbitstreams/CITY_704x576_30_orig_01.yuv '!' videoparse format=i420 width=704 height=576 '!' jpegenc '!' omxmjpegdec '!' vaapisink"
2.check the output on console.
3.the console keey printing following message
    "omxil-core error: programmer bug, uv data copy from 0x33434d49 to 0x13 format hasn't supported yet
    omxil-core error: programmer bug, uv data copy from 0x33434d49 to 0x13 format hasn't supported yet
    Pipeline is PREROLLED ...
    Setting pipeline to PLAYING ...
    New clock: GstSystemClock
    omxil-core error: programmer bug, uv data copy from 0x33434d49 to 0x13 format hasn't supported yet
    omxil-core error: programmer bug, uv data copy from 0x33434d49 to 0x13 format hasn't supported yet"

Build error: –enable-tests –enable-tests-gles –enable-capi

build libyami error when use config –enable-tests-gles –disable-x11
build libyami error when use config –enable-tests –enable-tests-gles –enable-capi

[SKL][Media][Encode]fps value was set error when doing vp8 encoding test

1.Testing environment:
libva: (staging) d9626b77f865ce7bdbc8c78988a86d3e4f55d67b
intel driver: (staging) a8ca5f911c7c19b5ede3f96ac6113606b9dd4d95
libyami: (master) 7730d21
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Test command
(1)/home/zliu50x/libyami/omx/libyami/tests/yamiencode -i /home/encoderbitstreams/encoderbitstreams/1080p_blue_sky.yuv -s I420 -W 1920 -H 1080 -c VP8 -o 1080p_blue_sky.yuv.enc.vp8
(2)using vega test tools to verify the correctness,found that carton occurred while playing the file.The fps value was set error when doing vp8 encoding test.

[BDW][Media][Encode]startAborted when doing encoding with camera for pure C API interface

1.Testing environment:
libva: (master) ccd93de5a707e92a629cccd595757c8d436fa3cc
intel driver: (master) 24cba20a119c96556ae4dc9a90043896ea70e56
libyami: (master) ddd2a83
omxil-core: (master) d6f6edef3119d5622f9066521fc4b53004410a27
omx-components: (master) 464135af23ac7d3ae5d3a9240f2fd7a058eb2d7c
gst-omx: (master) f178afe59e507ea7113bbc630affd6d502a2418c
2.Testing steps:
(1)run the command
" ./encodecapi -i /dev/video0 -s YUY2 -W 1920 -H 1080 -c AVC -b 93312000 -f 30 -N 50 -o out.h264"
(2)Check the dmesg and get the information :
libva info: VA-API version 0.36.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_36
libva info: va_openDriver() returns 0
lt-encodecapi: encodeInputCamera.cpp:295: virtual bool EncodeStreamInputCamera::getOneFrameInput(VideoEncRawBuffer&): Assertion `frameIndex>=0 && frameIndex<m_frameBufferCount' failed.
startAborted

v4l2decode can't dump video frame

v4l2decode -i /path/to/h264 -m 0
The yuv file is not created when use render mode 0.

[BDW][media][Decode]segmentation fault when testing for vp9 decoding

1. Testing Env:
   Libva: (staging)9d128cf241830f7ead49fa5b104248b44c680873
   Libva_intel_driver: (hybrid_vp9_rebase2)a343161979984b4307a015233e79380b55ab2d4f
   Ffmpeg: (master)037b527d62500cfde89d3bda5d4ac04ce8b8d959
   Libyami: (master)8cc7e42ca49ea407b7e49db0e200f794e20fc49c
2. Test command:
   (1)use following command to get dump yuv file
   /home/wenxin1/libyami/omx/libyami/tests/yamidecode -i /home/wenxin/VP9-Hybrid/andrei_480p.webm -m 0 -f I420
   input file: /home/wenxin/VP9-Hybrid/andrei_480p.webm, renderMode: 0
   libva info: VA-API version 0.99.0
   libva info: va_getDriverName() returns 0
   libva info: Trying to open /opt/X11R7/lib/dri/i965_drv_video.so
   libva info: Found init function __vaDriverInit_0_34
   libva info: va_openDriver() returns 0
   Segmentation fault