matroska-org/foundation-source

libEBML2, libMatroska2, mkvalidator, mkclean and the specifications

foundation-source's Issues

mkvalidator should return non-zero exit code on an error

So it would be easier to script it.

MKValidator doesn't recognize Secondary Seekhead

There is an example file at http://www.danhinsley.com/downloads/trailer.mkv

Update README

Hello. I'm using Debian 10.

The Readme file's build instructions are outdated. The first two steps have been superseeded by a simple cmake . command.

Also the binaries are created not under <root>/release_ but under the tool's source directory, e.g. <root>/mkclean/mkclean

Finally I would like to thank you for mkclean, it's been a life saver. 🙂

[wish] release mkvalidator 0.5.2 on Brew

There is still 0.5.0.

specdata.xml: license and copyright

I want to clarify what is the license for specdata.xml and, subsequently, its rendition as webpage?

Official website is also not clear about that, listing licenses for LibEBML, LibEBML2, LibMatroska and Core C, but not the specifications.

My aim is to create a specification for .mkv files using Kaitai Struct definition, so ideally I'd like to convert existing spec and reuse the comments / field names.

mkvalidator output options

Hello. Just a suggestion to make mkvalidator output a bit more parsable: a format option flag that results in well-formed XML output, preferably with a valid/invalid element and <error_message> elements. A JSON rendering of the same would also be a good option.

Compiling in FreeBSD fails

While it's similar to OSX, because TARGET_OSX parameter is not defined it tries to find sys/vfs.h instead of sys/mount.h in corec/corec/helpers/file/file_libc.c

Putting "#define TARGET_OSX" in config.h file fixes this but there should be check for freebsd systems I think.

Multiple bugs

the Node_GetData function in corec/corec/node/node.c in mkvalidator 0.5.1 can cause a denial of service(Null pointer dereference and application crash) via a crafted mkv file.

./mkvalidator mkvalidator_0.5.1_null_pointer_dereference.mkv

----debug info:----
Program received signal SIGSEGV, Segmentation fault.
0x0000000000421767 in Node_GetData (p=0x0, Id=256, Type=1)
at ../corec/corec/node/node.c:681
681 for (i=p->Data;i;i=i->Next)
(gdb) bt
#0 0x0000000000421767 in Node_GetData (p=0x0, Id=256, Type=1)
at ../corec/corec/node/node.c:681
#1 0x000000000042cb29 in EBML_ElementIsFiniteSize (Element=0x0)
at ebmlelement.c:98
#2 0x000000000042cf51 in EBML_ElementPositionEnd (Element=0x0)
at ebmlelement.c:195
#3 0x0000000000405917 in main (argc=2, argv=0x7fffffffdf78)
at mkvalidator.c:1036
(gdb) disassemble
Dump of assembler code for function Node_GetData:
0x0000000000421743 <+0>: push %rbp
0x0000000000421744 <+1>: mov %rsp,%rbp
0x0000000000421747 <+4>: mov %rdi,-0x18(%rbp)
0x000000000042174b <+8>: mov %rsi,-0x20(%rbp)
0x000000000042174f <+12>: mov %rdx,-0x28(%rbp)
0x0000000000421753 <+16>: mov -0x20(%rbp),%rax
0x0000000000421757 <+20>: shl $0x8,%rax
0x000000000042175b <+24>: or -0x28(%rbp),%rax
0x000000000042175f <+28>: mov %rax,-0x8(%rbp)
0x0000000000421763 <+32>: mov -0x18(%rbp),%rax
=> 0x0000000000421767 <+36>: mov 0x10(%rax),%rax
0x000000000042176b <+40>: mov %rax,-0x10(%rbp)
0x000000000042176f <+44>: jmp 0x421794 <Node_GetData+81>
0x0000000000421771 <+46>: mov -0x10(%rbp),%rax
0x0000000000421775 <+50>: mov 0x8(%rax),%rax
0x0000000000421779 <+54>: cmp -0x8(%rbp),%rax
0x000000000042177d <+58>: jne 0x421789 <Node_GetData+70>
0x000000000042177f <+60>: mov -0x10(%rbp),%rax
0x0000000000421783 <+64>: add $0x10,%rax
0x0000000000421787 <+68>: jmp 0x4217a0 <Node_GetData+93>
0x0000000000421789 <+70>: mov -0x10(%rbp),%rax
0x000000000042178d <+74>: mov (%rax),%rax
---Type to continue, or q to quit---q
Quit
(gdb) i r
rax 0x0 0
rbx 0x1 1
rcx 0x7ffff7b00810 140737348896784
rdx 0x1 1
rsi 0x100 256
rdi 0x0 0
rbp 0x7fffffffb740 0x7fffffffb740
rsp 0x7fffffffb740 0x7fffffffb740
r8 0x411e10 4267536
r9 0x7fffffffb370 140737488335728
r10 0xfffffffffffffa82 -1406
r11 0x246 582
r12 0x401420 4199456
r13 0x7fffffffdf70 140737488346992
r14 0x0 0
r15 0x0 0
rip 0x421767 0x421767 <Node_GetData+36>
eflags 0x10202 [ IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
---Type to continue, or q to quit---
gs 0x0 0
(gdb)

POC:mkvalidator_0.5.1_null_pointer_dereference.mkv
It has been assigned as CVE-2017-12779

the ReadData function in ebmlstring.c in libebml2(through 2012-08-26) can cause a denial of service(invalid free and application crash) via a crafted mkv file.

./mkvalidator libebml2_invalid_free.mkv

----debug info:----
.*** Error in `/home/a/Downloads/mkvalidator-0.5.1/release/gcc_linux_x64/mkvalidator': free(): invalid next size (fast): 0x000000000066fa40 ***

Program received signal SIGABRT, Aborted.
0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff7a4f0d8 in __GI_abort () at abort.c:89
#2 0x00007ffff7a88394 in __libc_message (do_abort=do_abort@entry=1,
fmt=fmt@entry=0x7ffff7b96b28 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff7a9466e in malloc_printerr (ptr=,
str=0x7ffff7b96cc8 "free(): invalid next size (fast)", action=1)
at malloc.c:4996
#4 _int_free (av=, p=, have_lock=0)
at malloc.c:3840
#5 0x0000000000431c0e in ReadData (Element=0x678c90, Input=0x675c40,
ParserContext=0x7fffffffb750, AllowDummyElt=0, Scope=1, DepthCheckCRC=0)
at ebmlstring.c:102
#6 0x000000000042fc8e in ReadData (Element=0x673b70, Input=0x675c40,
ParserContext=0x7fffffffb880, AllowDummyElt=0, Scope=1, DepthCheckCRC=1)
at ebmlmaster.c:331
#7 0x000000000040549d in main (argc=2, argv=0x7fffffffdf68)
at mkvalidator.c:974
(gdb)

POC:libebml2_invalid_free.mkv
It has been assigned as CVE-2017-12780.

the EBML_BufferToID function in ebmlelement.c in libebml2(through 2012-08-26) can cause a denial of service(Null pointer dereference and application crash) via a crafted mkv file.

./mkvalidator libebml2_null_pointer_dereference_1.mkv

----debug info:----
Program received signal SIGSEGV, Segmentation fault.
0x000000000042d233 in EBML_BufferToID (Buffer=0x0) at ebmlelement.c:261
261 if (Buffer[0] & 0x80)
(gdb) bt
#0 0x000000000042d233 in EBML_BufferToID (Buffer=0x0) at ebmlelement.c:261
#1 0x000000000040987a in MATROSKA_MetaSeekID (MetaSeek=0x6792f0)
at matroskamain.c:336
#2 0x00000000004030e6 in CheckSeekHead (SeekHead=0x678fa0)
at mkvalidator.c:472
#3 0x000000000040707b in main (argc=2, argv=0x7fffffffdf68)
at mkvalidator.c:1333
(gdb) disassemble
Dump of assembler code for function EBML_BufferToID:
0x000000000042d227 <+0>: push %rbp
0x000000000042d228 <+1>: mov %rsp,%rbp
0x000000000042d22b <+4>: mov %rdi,-0x8(%rbp)
0x000000000042d22f <+8>: mov -0x8(%rbp),%rax
=> 0x000000000042d233 <+12>: movzbl (%rax),%eax
0x000000000042d236 <+15>: test %al,%al
0x000000000042d238 <+17>: jns 0x42d249 <EBML_BufferToID+34>
0x000000000042d23a <+19>: mov -0x8(%rbp),%rax
0x000000000042d23e <+23>: movzbl (%rax),%eax
0x000000000042d241 <+26>: movzbl %al,%eax
0x000000000042d244 <+29>: jmpq 0x42d320 <EBML_BufferToID+249>
0x000000000042d249 <+34>: mov -0x8(%rbp),%rax
0x000000000042d24d <+38>: movzbl (%rax),%eax
0x000000000042d250 <+41>: movzbl %al,%eax
0x000000000042d253 <+44>: and $0x40,%eax
0x000000000042d256 <+47>: test %eax,%eax
0x000000000042d258 <+49>: je 0x42d27e <EBML_BufferToID+87>
0x000000000042d25a <+51>: mov -0x8(%rbp),%rax
0x000000000042d25e <+55>: movzbl (%rax),%eax
0x000000000042d261 <+58>: movzbl %al,%eax
0x000000000042d264 <+61>: shl $0x8,%eax
0x000000000042d267 <+64>: mov %eax,%edx
---Type to continue, or q to quit---q
Quit
(gdb) i r
rax 0x0 0
rbx 0x67 103
rcx 0x0 0
rdx 0x643ff0 6569968
rsi 0x644980 6572416
rdi 0x0 0
rbp 0x7fffffffb690 0x7fffffffb690
rsp 0x7fffffffb690 0x7fffffffb690
r8 0x7ffff7dd59d0 140737351866832
r9 0x7fffffffa880 140737488332928
r10 0x42bca4 4373668
r11 0x246 582
r12 0x401420 4199456
r13 0x7fffffffdf60 140737488346976
r14 0x0 0
r15 0x0 0
rip 0x42d233 0x42d233 <EBML_BufferToID+12>
eflags 0x10246 [ PF ZF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
---Type to continue, or q to quit---
gs 0x0 0
(gdb)

POC:libebml2_null_pointer_dereference_1.mkv
It has been assigned as CVE-2017-12781.

the ReadData function in ebmlmaster.c in libebml2(through 2012-08-26) can cause a denial of service(assert fault) via a crafted mkv file.

./mkvalidator libebml2_assert_fault_1.mkv

----debug info:----
..mkvalidator: ebmlmaster.c:427: ReadData: Assertion `SubElement!=((void *)0)' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff7a4f0d8 in __GI_abort () at abort.c:89
#2 0x00007ffff7a44b86 in __assert_fail_base (
fmt=0x7ffff7b95830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x43b93f "SubElement!=((void *)0)",
file=file@entry=0x43b5d0 "ebmlmaster.c", line=line@entry=427,
function=function@entry=0x43ba73 <PRETTY_FUNCTION.4885> "ReadData")
at assert.c:92
#3 0x00007ffff7a44c32 in __GI___assert_fail (
assertion=0x43b93f "SubElement!=((void *)0)",
file=0x43b5d0 "ebmlmaster.c", line=427,
function=0x43ba73 <PRETTY_FUNCTION.4885> "ReadData") at assert.c:101
#4 0x000000000043032e in ReadData (Element=0x678fa0, Input=0x675c40,
ParserContext=0x7fffffffb8a0, AllowDummyElt=1, Scope=1, DepthCheckCRC=2)
at ebmlmaster.c:427
#5 0x0000000000405c5f in main (argc=2, argv=0x7fffffffdf68)
at mkvalidator.c:1074
(gdb)

POC:libebml2_assert_fault_1.mkv
It has been assigned as CVE-2017-12782.

the ReadDataFloat function in ebmlnumber.c in libebml2(through 2012-08-26) can cause a denial of service(assert fault) via a crafted mkv file.

./mkvalidator libebml2_assert_fault_2.mkv

----debug info:----
....mkvalidator: ebmlnumber.c:222: ReadDataFloat: Assertion `Element->Base.DataSize == 8 || Element->Base.DataSize == 4' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff7a4f0d8 in __GI_abort () at abort.c:89
#2 0x00007ffff7a44b86 in __assert_fail_base (
fmt=0x7ffff7b95830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x43bb08 "Element->Base.DataSize == 8 || Element->Base.DataSize == 4", file=file@entry=0x43bab0 "ebmlnumber.c",
line=line@entry=222,
function=function@entry=0x43bcb2 <PRETTY_FUNCTION.4760> "ReadDataFloat") at assert.c:92
#3 0x00007ffff7a44c32 in __GI___assert_fail (
assertion=0x43bb08 "Element->Base.DataSize == 8 || Element->Base.DataSize == 4", file=0x43bab0 "ebmlnumber.c", line=222,
function=0x43bcb2 <PRETTY_FUNCTION.4760> "ReadDataFloat")
at assert.c:101
#4 0x0000000000430e8b in ReadDataFloat (Element=0x67abc0, Input=0x675c40,
ParserContext=0x7fffffffb520, AllowDummyElt=1, Scope=1, DepthCheckCRC=1)
at ebmlnumber.c:222
#5 0x000000000042fc8e in ReadData (Element=0x67aae0, Input=0x675c40,
ParserContext=0x7fffffffb600, AllowDummyElt=1, Scope=1, DepthCheckCRC=2)
at ebmlmaster.c:331
#6 0x000000000042fc8e in ReadData (Element=0x67a400, Input=0x675c40,
ParserContext=0x7fffffffb750, AllowDummyElt=1, Scope=1, DepthCheckCRC=3)
---Type to continue, or q to quit---
at ebmlmaster.c:331
#7 0x000000000040e558 in ReadTrackEntry (Element=0x67a400, Input=0x675c40,
ParserContext=0x7fffffffb750, AllowDummyElt=1, Scope=1, DepthCheckCRC=3)
at matroskamain.c:2257
#8 0x000000000042fc8e in ReadData (Element=0x679980, Input=0x675c40,
ParserContext=0x7fffffffb8a0, AllowDummyElt=1, Scope=1, DepthCheckCRC=4)
at ebmlmaster.c:331
#9 0x0000000000406097 in main (argc=2, argv=0x7fffffffdf68)
at mkvalidator.c:1124
(gdb)

POC:libebml2_assert_fault_2.mkv
It has been assigned as CVE-2017-12783.

the EBML_FindNextElement function in ebmlmain.c in libebml2(through 2012-08-26) can cause a denial of service(Null pointer dereference and application crash) via a crafted mkv file.

./mkclean libebml2_null_pointer_dereference_2.mkv

----debug info:----
Program received signal SIGSEGV, Segmentation fault.
0x0000000000447ee1 in EBML_FindNextElement (Input=0x6caef0, pContext=0x0,
UpperLevels=0x7fffffffabd4, AllowDummyElt=0) at ebmlmain.c:516
516 OrigContext = *pContext;
(gdb) bt
#0 0x0000000000447ee1 in EBML_FindNextElement (Input=0x6caef0, pContext=0x0,
UpperLevels=0x7fffffffabd4, AllowDummyElt=0) at ebmlmain.c:516
#1 0x0000000000446263 in EBML_ElementSkipData (p=0x6c9060, Input=0x6caef0,
Context=0x0, TestReadElt=0x0, AllowDummyElt=0) at ebmlelement.c:122
#2 0x00000000004039cb in CheckMatroskaHead (Head=0x6c9380,
Parser=0x7fffffffb470, Input=0x6caef0) at mkclean.c:673
#3 0x0000000000407c07 in main (argc=2, argv=0x7fffffffdf78) at mkclean.c:1643
(gdb) disassemble
Dump of assembler code for function EBML_FindNextElement:
0x0000000000447deb <+0>: push %rbp
0x0000000000447dec <+1>: mov %rsp,%rbp
0x0000000000447def <+4>: push %rbx
0x0000000000447df0 <+5>: sub $0xd8,%rsp
0x0000000000447df7 <+12>: mov %rdi,-0xb8(%rbp)
0x0000000000447dfe <+19>: mov %rsi,-0xc0(%rbp)
0x0000000000447e05 <+26>: mov %rdx,-0xc8(%rbp)
0x0000000000447e0c <+33>: mov %rcx,-0xd0(%rbp)
0x0000000000447e13 <+40>: mov %fs:0x28,%rax
0x0000000000447e1c <+49>: mov %rax,-0x18(%rbp)
0x0000000000447e20 <+53>: xor %eax,%eax
0x0000000000447e22 <+55>: movb $0x0,-0xac(%rbp)
0x0000000000447e29 <+62>: movb $0x0,-0xaa(%rbp)
0x0000000000447e30 <+69>: movl $0x0,-0xa0(%rbp)
0x0000000000447e3a <+79>: mov -0xc8(%rbp),%rax
0x0000000000447e41 <+86>: mov (%rax),%eax
0x0000000000447e43 <+88>: mov %eax,-0x9c(%rbp)
0x0000000000447e49 <+94>: cmpq $0x0,-0xb8(%rbp)
0x0000000000447e51 <+102>: jne 0x447e72 <EBML_FindNextElement+135>
0x0000000000447e53 <+104>: lea 0x37e56(%rip),%rcx # 0x47fcb0 <PRETTY_FUNCTION.4925>
0x0000000000447e5a <+111>: mov $0x1fc,%edx
---Type to continue, or q to quit---
0x0000000000447e5f <+116>: lea 0x37b8b(%rip),%rsi # 0x47f9f1
0x0000000000447e66 <+123>: lea 0x37d5b(%rip),%rdi # 0x47fbc8
0x0000000000447e6d <+130>: callq 0x401510 __assert_fail@plt
0x0000000000447e72 <+135>: mov -0xb8(%rbp),%rax
0x0000000000447e79 <+142>: mov 0x8(%rax),%rax
0x0000000000447e7d <+146>: mov 0x78(%rax),%rax
0x0000000000447e81 <+150>: mov -0xb8(%rbp),%rcx
0x0000000000447e88 <+157>: mov $0x1,%edx
0x0000000000447e8d <+162>: mov $0x0,%esi
0x0000000000447e92 <+167>: mov %rcx,%rdi
0x0000000000447e95 <+170>: callq *%rax
0x0000000000447e97 <+172>: mov %rax,-0x70(%rbp)
0x0000000000447e9b <+176>: lea -0x50(%rbp),%rax
0x0000000000447e9f <+180>: mov %rax,-0x80(%rbp)
0x0000000000447ea3 <+184>: cmpq $0xffffffffffffffff,-0x70(%rbp)
0x0000000000447ea8 <+189>: jne 0x447eb4 <EBML_FindNextElement+201>
0x0000000000447eaa <+191>: mov $0x0,%eax
0x0000000000447eaf <+196>: jmpq 0x448814 <EBML_FindNextElement+2601>
0x0000000000447eb4 <+201>: cmpq $0x0,-0x80(%rbp)
0x0000000000447eb9 <+206>: jne 0x447eda <EBML_FindNextElement+239>
0x0000000000447ebb <+208>: lea 0x37dee(%rip),%rcx # 0x47fcb0 <PRETTY_FUNCTION.4925>
0x0000000000447ec2 <+215>: mov $0x203,%edx
---Type to continue, or q to quit---
0x0000000000447ec7 <+220>: lea 0x37b23(%rip),%rsi # 0x47f9f1
0x0000000000447ece <+227>: lea 0x37d2b(%rip),%rdi # 0x47fc00
0x0000000000447ed5 <+234>: callq 0x401510 __assert_fail@plt
0x0000000000447eda <+239>: mov -0xc0(%rbp),%rax
=> 0x0000000000447ee1 <+246>: mov (%rax),%rdx
0x0000000000447ee4 <+249>: mov %rdx,-0x50(%rbp)
0x0000000000447ee8 <+253>: mov 0x8(%rax),%rdx
0x0000000000447eec <+257>: mov %rdx,-0x48(%rbp)
0x0000000000447ef0 <+261>: mov 0x10(%rax),%rdx
0x0000000000447ef4 <+265>: mov %rdx,-0x40(%rbp)
0x0000000000447ef8 <+269>: mov 0x18(%rax),%rax
0x0000000000447efc <+273>: mov %rax,-0x38(%rbp)
0x0000000000447f00 <+277>: jmp 0x447f32 <EBML_FindNextElement+327>
0x0000000000447f02 <+279>: mov -0x80(%rbp),%rax
0x0000000000447f06 <+283>: mov 0x8(%rax),%rax
0x0000000000447f0a <+287>: test %rax,%rax
0x0000000000447f0d <+290>: jne 0x447f11 <EBML_FindNextElement+294>
0x0000000000447f0f <+292>: jmp 0x447f55 <EBML_FindNextElement+362>
0x0000000000447f11 <+294>: mov -0x80(%rbp),%rax
0x0000000000447f15 <+298>: mov 0x8(%rax),%rax
0x0000000000447f19 <+302>: mov %rax,-0x80(%rbp)
0x0000000000447f1d <+306>: mov -0xc8(%rbp),%rax
0x0000000000447f24 <+313>: mov (%rax),%eax
---Type to continue, or q to quit---q
Quit
(gdb) i r
rax 0x0 0
rbx 0x7fffffffb4d0 140737488336080
rcx 0x7ffff7b0f4b0 140737348957360
rdx 0x1 1
rsi 0x0 0
rdi 0x3 3
rbp 0x7fffffffab90 0x7fffffffab90
rsp 0x7fffffffaab0 0x7fffffffaab0
r8 0x0 0
r9 0xb 11
r10 0x7fffffffa9a0 140737488333216
r11 0x246 582
r12 0x4017b0 4200368
r13 0x7fffffffdf70 140737488346992
r14 0x0 0
r15 0x0 0
rip 0x447ee1 0x447ee1 <EBML_FindNextElement+246>
eflags 0x10202 [ IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
---Type to continue, or q to quit---
gs 0x0 0
(gdb)

POC:libebml2_null_pointer_dereference_2.mkv
It has been assigned as CVE-2017-12800.

the UpdateDataSize function in ebmlmaster.c in libebml2(through 2012-08-26) can cause a denial of service(assert fault) via a crafted mkv file.

./mkclean libebml2_assert_fault_3.mkv

----debug info:----
mkclean: ebmlmaster.c:244: UpdateDataSize: Assertion `CheckMandatory((ebml_master*)Element, bWithDefault)' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff7a4f0d8 in __GI_abort () at abort.c:89
#2 0x00007ffff7a44b86 in __assert_fail_base (
fmt=0x7ffff7b95830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x47feb8 "CheckMandatory((ebml_master*)Element, bWithDefault)", file=file@entry=0x47fcd0 "ebmlmaster.c", line=line@entry=244,
function=function@entry=0x480334 <PRETTY_FUNCTION.4876> "UpdateDataSize") at assert.c:92
#3 0x00007ffff7a44c32 in __GI___assert_fail (
assertion=0x47feb8 "CheckMandatory((ebml_master*)Element, bWithDefault)",
file=0x47fcd0 "ebmlmaster.c", line=244,
function=0x480334 <PRETTY_FUNCTION.4876> "UpdateDataSize")
at assert.c:101
#4 0x0000000000449223 in UpdateDataSize (Element=0x6d1ec0, bWithDefault=0,
bForceWithoutMandatory=0) at ebmlmaster.c:244
#5 0x0000000000419f3e in UpdateDataSizeTrackEntry (Element=0x6d1ec0,
bWithDefault=0, bForceWithoutMandatory=0) at matroskamain.c:2343
#6 0x00000000004492ee in UpdateDataSize (Element=0x6d1e50, bWithDefault=0,
bForceWithoutMandatory=0) at ebmlmaster.c:256
#7 0x0000000000409672 in main (argc=2, argv=0x7fffffffdf78) at mkclean.c:2012
(gdb)

POC:libebml2_assert_fault_3.mkv
It has been assigned as CVE-2017-12801.

the EBML_IntegerValue function in ebmlnumber.c in libebml2(through 2012-08-26) can cause a denial of service(assert fault) via a crafted mkv file.

./mkclean libebml2_assert_fault_4.mkv

----debug info:----
mkclean: ebmlnumber.c:428: EBML_IntegerValue: Assertion `Node_IsPartOf(Element,(fourcc_t)(((uint8_t)('E') << 0) | ((uint8_t)('B') << 8) | ((uint8_t)('I') << 16) | ((uint8_t)('T')<< 24))) || Node_IsPartOf(Element,(fourcc_t)(((uint8_t)('E') << 0) | ((uint8_t)('B') << 8) | ((uint8_t)('S') << 16) | ((uint8_t)('I')<< 24)))' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff7a4bcc9 in __GI_raise (sig=sig@entry=6)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff7a4f0d8 in __GI_abort () at abort.c:89
#2 0x00007ffff7a44b86 in __assert_fail_base (
fmt=0x7ffff7b95830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x480478 "Node_IsPartOf(Element,(fourcc_t)(((uint8_t)('E') << 0) | ((uint8_t)('B') << 8) | ((uint8_t)('I') << 16) | ((uint8_t)('T')<< 24))) || Node_IsPartOf(Element,(fourcc_t)(((uint8_t)('E') << 0) | ((uint8_t)"..., file=file@entry=0x480390 "ebmlnumber.c", line=line@entry=428,
function=function@entry=0x480650 <PRETTY_FUNCTION.4901> "EBML_IntegerValue") at assert.c:92
#3 0x00007ffff7a44c32 in __GI___assert_fail (
assertion=0x480478 "Node_IsPartOf(Element,(fourcc_t)(((uint8_t)('E') << 0) | ((uint8_t)('B') << 8) | ((uint8_t)('I') << 16) | ((uint8_t)('T')<< 24))) || Node_IsPartOf(Element,(fourcc_t)(((uint8_t)('E') << 0) | ((uint8_t)"...,
file=0x480390 "ebmlnumber.c", line=428,
function=0x480650 <PRETTY_FUNCTION.4901> "EBML_IntegerValue")
at assert.c:101
#4 0x000000000044c07a in EBML_IntegerValue (Element=0x6d1190)
at ebmlnumber.c:428
#5 0x000000000040850a in main (argc=2, argv=0x7fffffffdf78) at mkclean.c:1764
(gdb)

POC:libebml2_assert_fault_4.mkv
It has been assigned as CVE-2017-12802.

the Node_ValidatePtr function in corec/corec/node/node.c in mkclean 0.8.9 can cause a denial of service(assert fault) via a crafted mkv file.

./mkclean mkclean_0.8.9_assert_fault.mkv

----debug info:----
Program received signal SIGSEGV, Segmentation fault.
0x000000000043d3d2 in Node_ValidatePtr (Node=0x0)
at ../corec/corec/node/node.c:155
155 assert(((node*)Node)->Magic==NODE_MAGIC);
(gdb) bt
#0 0x000000000043d3d2 in Node_ValidatePtr (Node=0x0)
at ../corec/corec/node/node.c:155
#1 Node_IsPartOf (Node=0x0, PartOfClass=1414087237)
at ../corec/corec/node/node.c:1534
#2 0x000000000044c040 in EBML_IntegerValue (Element=0x0) at ebmlnumber.c:428
#3 0x0000000000404a87 in CleanTracks (Tracks=0x6d1160, SrcProfile=1,
DstProfile=0x6a0584 , Attachments=0x0,
Alternate3DTracks=0x7fffffffb410) at mkclean.c:962
#4 0x0000000000408812 in main (argc=2, argv=0x7fffffffdf88) at mkclean.c:1811

POC:mkclean_0.8.9_assert_fault.mkv
It has been assigned as CVE-2017-12803.

poc.zip

mkvalidator sometimes shows negative bitrates

See http://thread.gmane.org/gmane.comp.multimedia.matroska.devel/5221

store the information about EBML recurring elements in ebml_semantic

We need to update the spectool and libmatroska at the same time.

mkvalidator 0.6.0 empty archive issue

👋 it looks the mkvalidator 0.6.0 artifact on sourceforge is empty archive. Thanks!

relates to Homebrew/homebrew-core#70111

Mkvalidator roadmap and 0.5.3 ETA ?

Dear @robUx4 ,

Happy New Year 🎉

What's your roadmap for mkvalidator ? Would you have an ETA for the 0.5.3 release ?

On a side note, is the code on the master branch ready for production ?

Best regards,

Samuel, from CINES (Montpellier, France)
https://www.cines.fr/en/

Non-UNIX line endings in configure cause build failure on macOS

Using the 0.5.2 release tarball from https://sourceforge.net/projects/matroska/files/mkvalidator/mkvalidator-0.5.2.tar.bz2/download the build fails on macOS:

bash-3.2$ ./configure
bash: ./configure: /bin/sh^M: bad interpreter: No such file or directory
bash-3.2$ 

Running dos2unix on configure fixes it.

spectools should use the new EBML Schema for Matroska

That shouldn't be too hard for now. Parsing the ABNF path is the tricky part. Later there will be XML includes and it will be a little trickier.

mkvclean "Failed to write a Cluster"

Hi,

I use mkclean in most of my video files since my TV doesn't like (i.e. doesn't play) non-standard mkv. Don't really know what standard it supports but the result of mkclean is usually enough.

Once in a while I get lots of "Failed to write a Cluster at the required position", where the actual positions shown has a difference of one.

The result when this happens is usually unplayable. Some times it plays but can't be seeked (i.e. move forward, or backward, or start at any given position other than the beginning).

Any idea what could be causing this?

Any idea about how to fix it?

I'm willing to build a debug version and run mkclean under gdb.

---

My actual command is:

mkclean --quiet --optimize --keep-cues --unsafe "$NEW_FILE"

I have tried other options, w/o --keep-cues, --remux, with the same result.

mkclean v0.8.10

mkvalidator mkvalidator.c gcd function Floating point exception

Credit: giantbranch of NSFOCUS Security Team

What's the problem?

Floating point exception was found in gcd function of mkvalidator.c in mkvalidator v0.5.2.

ASAN reports:

$ ./mkvalidator ./tests_82.mkv
WRN080: Unknown element [30][57][41] at 59 size 18
WRN080: Unknown element [2A][D7][B1] at 81 size 2
WRN080: Unknown element [44][89] at 87 size 4
ERR202: Unique element 'DisplayHeight' in Video at 177 found more than once at 196
ERR202: Unique element 'DisplayHeight' in Video at 177 found more than once at 196
ERR0E7: Video track #1 at 99 has a null display height
AddressSanitizer:DEADLYSIGNAL
=================================================================
==12833==ERROR: AddressSanitizer: FPE on unknown address 0x0000004ca475 (pc 0x0000004ca475 bp 0x7ffe9d9d8380 sp 0x7ffe9d9d3820 T0)
    #0 0x4ca475 in gcd /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:209:23
    #1 0x4ca475 in CheckVideoTrack /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:265:31
    #2 0x4ca475 in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:1178:19
    #3 0x7f330cf9a83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291
    #4 0x41bf58 in _start (/root/reproduce/mkvalidator+0x41bf58)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: FPE /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:209:23 in gcd
==12833==ABORTING

location: foundation-source/libebml2/ebmlcrc.c:244

How can we reproduce the issue?

Compile command I use:

clang corec/tools/coremake/coremake.c -o coremake && ./coremake gcc_linux_x64

make -C mkvalidator -e CC="clang -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e CXX="clang++ -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e STRIP=""

reproduce the issue

unzip tests_xx.zip
mkvalidator ./tests_xx.mkv

poc:
tests_82.zip

the details about my environment.

• mkvalidator version:

$ ./mkvalidator --version
mkvalidator v0.5.2, Copyright (c) 2010-2015 Matroska Foundation
	file "--version"

• Host Operating System and version: Ubuntu 16.04.3 LTS
• Host CPU architecture: x86_64
• clang: clang version 11.0.0

mkvalidator libebml2/ebmlmaster.c ReadData function null pointer dereference

Credit: giantbranch of NSFOCUS Security Team

What's the problem?

The ReadData function in libebml2/ebmlmaster.c in mkvalidator v0.5.2 can be cause a denial of service (Null pointer dereference and application crash) via a crafted mkv file.

ASAN reports:

$ ./mkvalidator ./tests_73.mkv
..AddressSanitizer:DEADLYSIGNAL
=================================================================
==12818==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000038 (pc 0x000000518f84 bp 0x7ffc70c9fe50 sp 0x7ffc70c9fc20 T0)
==12818==The signal is caused by a READ memory access.
==12818==Hint: address points to the zero page.
    #0 0x518f84 in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:428:9
    #1 0x4c9ab6 in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:1109:17
    #2 0x7f60ce2c183f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291
    #3 0x41bf58 in _start (/root/reproduce/mkvalidator+0x41bf58)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:428:9 in ReadData
==12818==ABORTING

location: foundation-source/libebml2/ebmlmaster.c:428

How can we reproduce the issue?

Compile command I use:

clang corec/tools/coremake/coremake.c -o coremake && ./coremake gcc_linux_x64

make -C mkvalidator -e CC="clang -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e CXX="clang++ -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e STRIP=""

reproduce the issue

unzip tests_xx.zip
mkvalidator ./tests_xx.mkv

poc:
tests_73.zip

the details about my environment.

• mkvalidator version:

$ ./mkvalidator --version
mkvalidator v0.5.2, Copyright (c) 2010-2015 Matroska Foundation
	file "--version"

• Host Operating System and version: Ubuntu 16.04.3 LTS
• Host CPU architecture: x86_64
• clang: clang version 11.0.0

mkclean doesn't support the V_VP9 codec ID for the WebM profile

add a flag in the EBML semantic to mark recurring elements

This is slightly different than the maxOccurs value and should be handled accordingly.

use clz()/ctz() to quicky find the size of an EBML encoded value

https://en.wikipedia.org/wiki/Find_first_set

mkvalidator, webm and CueRelativePosition

Hello,

MKVToolNix by default writes CueRelativePosition elements for every cue entry, even for those relating to video keyframes (which is of course a waste of space given that they are the first blocks in their clusters, but not a significant one). This is also true for webm files and for such files mkvalidator complains (ERR201) that CueRelativePosition is incompatible with webm. But this is not true: Both CueRelativePosition and CueDuration are allowed in webm.

Greetings
Andi

cmake build issue

Hello,

I tried to build the new release of mkclean (0.9.0) with cmake and the build stops trying to link string_test.exe (this is in Windows under a Cygwin environment with all needed development libraries installed -- mkclean-0.8.10 builds with no problem). Linking fails because the iconv symbols are not found, and in fact libiconv is not being linked.

I did not found any reference to iconv in the cmake files included with the code. Is this supposed to work as it currently is?

mkvalidator should print out the filename or other identifier

I'm on Windows and when I do

CMD /k C:\mkvalidator\mkvalidator.exe $path\$filename.ext

with mkvalidator 0.5.0

I get:
mkvalidator 0.5.0: the file appears to be valid
file created with [...]

It does not tell me the filename of the file it has processed. If I want to feed mkvalidator files automatically I wont have an overview which of the files output an error or warning.

data2lib and data2lib2 segfault

Compiling a clean version at revision 15d0741, then running data2spec in the cellar-wg/matroska-specification repo directory ends with a segfault:

[0 mosu@sweet-chili (master) ~/prog/video/matroska-specification] ../matroska-foundation/release/gcc_linux_x64/data2spec
unknown attribute length
unknown attribute recurring
unknown attribute length
unknown attribute length
unknown attribute length
unknown attribute length
unknown attribute recurring
unknown attribute length
unknown attribute length
unknown attribute length
unknown attribute recurring
unknown attribute length
did not parse element 'EBMLMaxIDLength' path \EBML\EBMLMaxIDLength
zsh: segmentation fault (core dumped)  ../matroska-foundation/release/gcc_linux_x64/data2spec

mkclean doesn't support the A_OPUS codec ID for the WebM profile

Chrome's MediaRecorder API records audio in Opus format. Would be awesome to have support for this in mkclean and mkvalidator. Not sure how much work this would be, but I'm down to contribute if you guys could point me to a general area of the codebase.

StreamOpen cannot negotiate class id for on disk file

When trying the examples that came with the code, StreamOpen which calls to NodeEnumClassStr internally has NodeEnumClassStr always return 0 as fourcc for a on-disk file. Stepping into NodeEnumClassFilterRated, it seems that the initialized class nodes that have STRM as fourcc always have FMTM as parent, and then NODE.

What are some suggestions on how to debug this problem? I was using code from tag mkclean-0.8.10, the master branch also seems to have the same behavior. (The code was added DLL defines (those that were defined to __declspec(dllimport) or __declspec(dllexport))prior to all the contexts to use across library boundary, and to use the whole library dynamically, also the zlib implementation was modified to use zlib-ng)

mkvalidator 0.5.0 reports ERR201 for Matroska files created by mkvmerge with option --engage no_cue_duration

Description
What appear to be perfectly valid MKV files produced from mkvmerge with the hack --engage no_cue_duration makes mkvalidator return exit code 255 and report ERR201 to stderr.

examples:
ERR201: Invalid 'CueCodecState' for profile 'matroska v1' in CueTrackPositions at 3985449399
ERR201: Invalid 'CueCodecState' for profile 'matroska v1' in CueTrackPositions at 3985449495
ERR201: Invalid 'CueCodecState' for profile 'matroska v1' in CueTrackPositions at 3985449519

tooling versions:
mkvalidator v0.5.0, Copyright (c) 2010-2015 Matroska Foundation
mkvmerge v7.8.0 ('River Man') 64bit built on Mar 27 2015 20:38:03

Operating System:
Ubuntu 14.04 (Linux N4030 3.13.0-51-generic #84-Ubuntu SMP Wed Apr 15 12:08:34 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux)

Expected result:
mkvalidator to return exit code 0 and not report ERR201 to stderr

Actual result:
mkvalidator returns exit code 255 and reports many ERR201 to stderr

generate mkclean/mkvalidator artifact with version name on tagging

If possible these artifacts should be kept forever. Or maybe rerun the build every 90 days ?

• The ReadMe in the front page should point to that specially tagged artifact.

Add Github actions to test building works

And maybe generate .exe artifacts for Windows.

mkvalidator libebml2/ebmlmain.c EBML_ReadCodedSizeValue function heap buffer overflow read

Credit: giantbranch of NSFOCUS Security Team

What's the problem?

A heap buffer overflow read in libebml2/ebmlmain.c EBML_ReadCodedSizeValue function in mkvalidator v0.5.2.

ASAN reports:

$ ./mkvalidator ./tests_84.mkv
WRN00C: Unknown element in TrackEntry [63][8C] at 4367 (size 38 total 41)
ERR201: Invalid 'FlagEnabled' for profile 'matroska v1' in TrackEntry at 4305
ERR201: Invalid 'CodecDecodeAll' for profile 'matroska v1' in TrackEntry at 4305
ERR201: Invalid 'FlagInterlaced' for profile 'matroska v1' in Video at 4436
ERR201: Invalid 'FlagEnabled' for profile 'matroska v1' in TrackEntry at 4459
ERR201: Invalid 'CodecDecodeAll' for profile 'matroska v1' in TrackEntry at 4459
ERR200: Missing element 'FileUID' in AttachedFile at 6198
WRN080: Unknown element [46][AE] at 45257 size 4
..=================================================================
==12845==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000001408 at pc 0x000000513121 bp 0x7fffc6d498f0 sp 0x7fffc6d498e8
READ of size 1 at 0x603000001408 thread T0
    #0 0x513120 in EBML_ReadCodedSizeValue /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmain.c:139:7
    #1 0x513120 in EBML_ReadCodedSizeSignedValue /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmain.c:172:21
    #2 0x4da9a2 in ReadBlockData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:1470:27
    #3 0x5186eb in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:331:21
    #4 0x5186eb in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:331:21
    #5 0x4c980c in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:1063:17
    #6 0x7f034c30a83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291
    #7 0x41bf58 in _start (/root/reproduce/mkvalidator+0x41bf58)

0x603000001408 is located 0 bytes to the right of 24-byte region [0x6030000013f0,0x603000001408)
allocated by thread T0 here:
    #0 0x495dcd in malloc (/root/reproduce/mkvalidator+0x495dcd)
    #1 0x4da87a in ReadBlockData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:1458:23
    #2 0x5186eb in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:331:21
    #3 0x5186eb in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:331:21
    #4 0x4c980c in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:1063:17
    #5 0x7f034c30a83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291

SUMMARY: AddressSanitizer: heap-buffer-overflow /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmain.c:139:7 in EBML_ReadCodedSizeValue
Shadow bytes around the buggy address:
  0x0c067fff8230: 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00 00 00
  0x0c067fff8240: fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00
  0x0c067fff8250: 00 00 fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa
  0x0c067fff8260: 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00 00 00
  0x0c067fff8270: fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00
=>0x0c067fff8280: 00[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8290: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff82a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff82b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff82c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff82d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==12845==ABORTING

location: foundation-source/libebml2/ebmlmain.c:139

How can we reproduce the issue?

Compile command I use:

clang corec/tools/coremake/coremake.c -o coremake && ./coremake gcc_linux_x64

make -C mkvalidator -e CC="clang -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e CXX="clang++ -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e STRIP=""

reproduce the issue

unzip tests_xx.zip
mkvalidator ./tests_xx.mkv

poc:
tests_84.zip

the details about my environment.

• mkvalidator version:

$ ./mkvalidator --version
mkvalidator v0.5.2, Copyright (c) 2010-2015 Matroska Foundation
	file "--version"

• Host Operating System and version: Ubuntu 16.04.3 LTS
• Host CPU architecture: x86_64
• clang: clang version 11.0.0

mkvalidator libmatroska2/matroskamain.c MATROSKA_LinkBlockWithReadTracks function null pointer dereference

Credit: giantbranch of NSFOCUS Security Team

What's the problem?

The MATROSKA_LinkBlockWithReadTracks function in libmatroska2/matroskamain.c in mkvalidator v0.5.2 can be cause a denial of service (Null pointer dereference and application crash) via a crafted mkv file.

ASAN reports:

$ ./mkvalidator ./tests_71.mkv
ERR007: The EBML max size length is not supported: -2130427149
ERR200: Missing element 'WritingApp' in Info at 45
WRN080: Unknown element [73][C5] at 112 size 1
WRN080: Unknown element [83] at 116 size 1
WRN080: Unknown element [9C] at 119 size 1
WRN080: Unknown element [86] at 122 size 15
WRN080: Unknown element [63][A2] at 139 size 43
WRN080: Unknown element [E0] at 185 size 22
WRN801: The segment has no SeekHead section
.AddressSanitizer:DEADLYSIGNAL
=================================================================
==12631==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000028 (pc 0x0000004d0f93 bp 0x7ffda33b1c10 sp 0x7ffda33b1b40 T0)
==12631==The signal is caused by a READ memory access.
==12631==Hint: address points to the zero page.
    #0 0x4d0f93 in MATROSKA_LinkBlockWithReadTracks /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c
    #1 0x4d5608 in MATROSKA_LinkClusterBlocks /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:852:8
    #2 0x4cb41e in LinkClusterBlocks /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:585:3
    #3 0x4cb41e in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:1352:3
    #4 0x7f628613c83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291
    #5 0x41bf58 in _start (/root/reproduce/mkvalidator+0x41bf58)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c in MATROSKA_LinkBlockWithReadTracks
==12631==ABORTING

location：foundation-source/libmatroska2/matroskamain.c:220

RTrackInfo is initialized to NULL, because it is referenced in the MATROSKA_LinkBlockWithReadTracks function of matroskamain.c without assigning a value to RTrackInfo.

How can we reproduce the issue?

Compile command I use:

clang corec/tools/coremake/coremake.c -o coremake && ./coremake gcc_linux_x64

make -C mkvalidator -e CC="clang -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e CXX="clang++ -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e STRIP=""

reproduce the issue

unzip tests_xx.zip
mkvalidator ./tests_xx.mkv

poc:
tests_71.zip

the details about my environment.

• mkvalidator version:

$ ./mkvalidator --version
mkvalidator v0.5.2, Copyright (c) 2010-2015 Matroska Foundation
	file "--version"

• Host Operating System and version: Ubuntu 16.04.3 LTS
• Host CPU architecture: x86_64
• clang: clang version 11.0.0

mkvalidator libebml2/ebmlcrc.c EBML_CRCMatches function heap buffer overflow read

Credit: giantbranch of NSFOCUS Security Team

What's the problem?

A heap buffer overflow read in libebml2/ebmlcrc.c EBML_CRCMatches function in mkvalidator v0.5.2.

ASAN reports:

$ ./mkvalidator ./tests_76.mkv
WRN00C: Unknown element in Info [90] at 48 (size 33 total 35)
WRN00C: Unknown element in Info [31][57][41] at 84 (size 43 total 47)
WRN00C: Unknown element in Info [4A][89] at 150 (size 4 total 7)
ERR203: Invalid checksum for element 'Info' at 36
ERR200: Missing element 'MuxingApp' in Info at 36
ERR200: Missing element 'WritingApp' in Info at 36
WRN080: Unknown element [11][4D][9B][8A] at 168 size 801
.=================================================================
==12863==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60f000000869 at pc 0x00000050f66d bp 0x7fff61470a60 sp 0x7fff61470a58
READ of size 4 at 0x60f000000869 thread T0
    #0 0x50f66c in EBML_CRCMatches /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlcrc.c:244:14
    #1 0x518e92 in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:416:35
    #2 0x4e0788 in ReadTrackEntry /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:2261:20
    #3 0x5186eb in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:331:21
    #4 0x4c9c74 in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:1136:17
    #5 0x7f70c848e83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291
    #6 0x41bf58 in _start (/root/reproduce/mkvalidator+0x41bf58)

0x60f000000869 is located 1 bytes to the right of 168-byte region [0x60f0000007c0,0x60f000000868)
allocated by thread T0 here:
    #0 0x495dcd in malloc (/root/reproduce/mkvalidator+0x495dcd)
    #1 0x51fbb8 in Data_ReAlloc /root/debug-fuzz-reslut/mkvalidator/foundation-source/corec/corec/array/array.c:87:24

SUMMARY: AddressSanitizer: heap-buffer-overflow /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlcrc.c:244:14 in EBML_CRCMatches
Shadow bytes around the buggy address:
  0x0c1e7fff80b0: 00 00 00 fa fa fa fa fa fa fa fa fa 00 00 00 00
  0x0c1e7fff80c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c1e7fff80d0: 00 fa fa fa fa fa fa fa fa fa fd fd fd fd fd fd
  0x0c1e7fff80e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
  0x0c1e7fff80f0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
=>0x0c1e7fff8100: 00 00 00 00 00 00 00 00 00 00 00 00 00[fa]fa fa
  0x0c1e7fff8110: fa fa fa fa fa fa 00 00 00 00 00 00 00 00 00 00
  0x0c1e7fff8120: 00 00 00 00 00 00 00 00 00 00 00 fa fa fa fa fa
  0x0c1e7fff8130: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c1e7fff8140: 00 00 00 00 00 00 00 00 00 fa fa fa fa fa fa fa
  0x0c1e7fff8150: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==12863==ABORTING

location: foundation-source/libebml2/ebmlcrc.c:244

How can we reproduce the issue?

Compile command I use:

clang corec/tools/coremake/coremake.c -o coremake && ./coremake gcc_linux_x64

make -C mkvalidator -e CC="clang -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e CXX="clang++ -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e STRIP=""

reproduce the issue

unzip tests_xx.zip
mkvalidator ./tests_xx.mkv

poc:
tests_76.zip

the details about my environment.

• mkvalidator version:

$ ./mkvalidator --version
mkvalidator v0.5.2, Copyright (c) 2010-2015 Matroska Foundation
	file "--version"

• Host Operating System and version: Ubuntu 16.04.3 LTS
• Host CPU architecture: x86_64
• clang: clang version 11.0.0

mkvalidator should be able to exit on first error or warning

I'd like to be able to run mkvalidator with a flag that will exit the application after logging the first error or warning depending on how strict to make it.

Use case is for batching file validations and speeding up runtimes by exiting asap.

mkvalidator doesn't detect overlapping timestamps

If you remux this file with mkvmerge, it will create BlockGroup of 1536ms, but the next BlockGroup will start at 256ms (rightfully).

This kind of incorrect behaviour should be detected by mkvalidator.

mkvalidator doesn't detect wrong CueRelativePosition, CueDuration and CueClusterPosition

Hello,

mkvmerge started writing CueRelativePosition and CueDuration elements beginning in version 5.9; but there was a bug in the early implementation: Up until version 6.3 CueRelativePosition pointed to the inner block of a blockgroup (if a subtitle packet is stored in a block and not a simpleblock). But mkvalidator doesn't seem to check whether CueRelativePosition (and probably CueDuration, too) is wrong and says that such files appear to be valid.
[Edit]: After noticing mkvalidator not complaining about a file with a wrong CueClusterPosition entry I looked at the code a bit and found out that CueClusterPosition is not only not checked, but that the check for the validity of the cues is highly inefficient: Currently all blocks that precede (in storage order) the block with the right timestamp (if it exists) are checked, although there is no need to check all these blocks: It is only needed to check the cluster that is actually referenced via CueClusterPosition whether it contains an entry with the right timestamp and TrackID. If there isn't a cluster at the right place or it doesn't contain (simple)block with the right timestamp, then we already know that an error is appropriate and the file is invalid; there is no point in checking whether a different cluster contains a block with the right TrackID and timestamp.
[Edit2]: And judging from the code, only the first CueTrackPositions is actually checked (i.e. if there is no (Simple)Block with the TrackID given in the CueTrack in any subsequent CueTrackPositions, then no error is raised).

Greetings
Andi

use zlib-ng instead of zlib

As soon as there is a stable release.

The code is more modern and builds better.

Not sure if the zlib compatibility should be used or just switch the API entirely.

Ref. #27

mkvalidator 0.5.0 and non-ASCII characters

Hello,

although mkvalidator can handle non-latin command-line characters on windows since version 0.4.1 (according to the changelog), it can't even handle simple German umlauts for me.
I also tested version 4.0 (after all, Latin-1 and CP 1252 contains umlauts, so maybe it worked in version 0.4.0 and maybe the commit broke it), but this didn't work, either.

Thanks in advance
Andi

do not include the branch tool in the source packages

mkclean uses only default zlib compression

Hello,

it seems to me that mkclean uses only the default zlib compression level (equivalent to 6) and not nine (as mkvmerge does); as a result, files can be larger than the ones produced by mkvmerge even when one uses header removal compression and no crc-elements. Could this be changed (level 9 shouldn't pose any problem for modern computers)?

Greetings and thanks
Andi

mkvalidator libmatroska2/matroskamain.c CheckCompression function null pointer dereference

Credit: giantbranch of NSFOCUS Security Team

What's the problem?

The CheckCompression function in libmatroska2/matroskamain.c in mkvalidator v0.5.2 can be cause a denial of service (Null pointer dereference and application crash) via a crafted mkv file.

ASAN reports:

$ ./mkvalidator ./tests_72.mkv
WRN00C: Unknown element in TrackEntry [43][63] at 4353 (size 34 total 37)
WRN00C: Unknown element in TrackEntry [72][6E] at 4410 (size 10 total 13)
WRN00C: Unknown element in ContentEncodings [62][F1] at 4450 (size 11 total 14)
WRN00C: Unknown element in Tracks [D0] at 4475 (size 3 total 5)
WRN00C: Unknown element in Tracks [86] at 4482 (size 10 total 12)
WRN00C: Unknown element in Tracks [22][B5][9C] at 4494 (size 3 total 7)
WRN00C: Unknown element in Tracks [6D][80] at 4501 (size 6 total 9)
ERR202: Unique element 'TrackNumber' in TrackEntry at 4327 found more than once at 4466
ERR202: Unique element 'TrackNumber' in TrackEntry at 4327 found more than once at 4466
ERR200: Missing element 'CodecID' in TrackEntry at 4327
ERR200: Missing element 'ContentEncoding' in ContentEncodings at 4447
WRN080: Unknown element [FF] at 4536 size 101
WRN080: Unknown element [FF] at 5052 size 105
ERR201: Invalid 'EncryptedBlock' for profile 'matroska v2' in Cluster at 5186
ERR201: Invalid 'EncryptedBlock' for profile 'matroska v2' in Cluster at 5186
ERR200: Missing element 'Block' in BlockGroup at 5857
WRN080: Unknown element [FF] at 7426 size 5426
ERR063: The SeekPoint at 57 references a SegmentInfo at wrong position 4151 (real 4170)
ERR065: The SeekPoint at 72 references a TrackInfo at wrong position 4287 (real 4321)
ERR066: The SeekPoint at 87 references an unknown Cues at 23625
.AddressSanitizer:DEADLYSIGNAL
=================================================================
==12700==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000028 (pc 0x0000004d12c3 bp 0x7fff3a3d6ad0 sp 0x7fff3a3d69a0 T0)
==12700==The signal is caused by a READ memory access.
==12700==Hint: address points to the zero page.
    #0 0x4d12c3 in CheckCompression /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:171:13
    #1 0x4d119d in MATROSKA_LinkBlockWithReadTracks /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:232:20
    #2 0x4d5608 in MATROSKA_LinkClusterBlocks /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:852:8
    #3 0x4cb41e in LinkClusterBlocks /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:585:3
    #4 0x4cb41e in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:1352:3
    #5 0x7f3f2cfce83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291
    #6 0x41bf58 in _start (/root/reproduce/mkvalidator+0x41bf58)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /root/debug-fuzz-reslut/mkvalidator/foundation-source/libmatroska2/matroskamain.c:171:13 in CheckCompression
==12700==ABORTING

location: foundation-source/libmatroska2/matroskamain.c:171

How can we reproduce the issue?

Compile command I use:

clang corec/tools/coremake/coremake.c -o coremake && ./coremake gcc_linux_x64

make -C mkvalidator -e CC="clang -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e CXX="clang++ -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e STRIP=""

reproduce the issue

unzip tests_xx.zip
mkvalidator ./tests_xx.mkv

poc:
tests_72.zip

the details about my environment.

• mkvalidator version:

$ ./mkvalidator --version
mkvalidator v0.5.2, Copyright (c) 2010-2015 Matroska Foundation
	file "--version"

• Host Operating System and version: Ubuntu 16.04.3 LTS
• Host CPU architecture: x86_64
• clang: clang version 11.0.0

Add the possibility to in libebml2 to tell if an element can have an infinite size

Only a few elements in Matroska are allowed to have that. That should avoid reading bogus sizes.

UAF in mkvalidator libebml2/ebmlcrc.c EBML_CRCMatches function

Credit: giantbranch of NSFOCUS Security Team

What's the problem?

A Use After Free in libebml2/ebmlcrc.c EBML_CRCMatches function in mkvalidator v0.5.2.

ASAN reports:

$ ./mkvalidator ./tests_74.mkv
.=================================================================
==12870==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000000888 at pc 0x00000050f66d bp 0x7ffdaad596c0 sp 0x7ffdaad596b8
READ of size 4 at 0x603000000888 thread T0
    #0 0x50f66c in EBML_CRCMatches /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlcrc.c:244:14
    #1 0x518e92 in ReadData /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlmaster.c:416:35
    #2 0x4c8d4e in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:981:6
    #3 0x7feed484f83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291
    #4 0x41bf58 in _start (/root/reproduce/mkvalidator+0x41bf58)

0x603000000888 is located 8 bytes inside of 24-byte region [0x603000000880,0x603000000898)
freed by thread T0 here:
    #0 0x495b4d in free (/root/reproduce/mkvalidator+0x495b4d)
    #1 0x51ff88 in ArrayClear /root/debug-fuzz-reslut/mkvalidator/foundation-source/corec/corec/array/array.c:157:2
    #2 0x4c8d4e in main /root/debug-fuzz-reslut/mkvalidator/foundation-source/mkvalidator/mkvalidator.c:981:6
    #3 0x7feed484f83f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291

previously allocated by thread T0 here:
    #0 0x495dcd in malloc (/root/reproduce/mkvalidator+0x495dcd)
    #1 0x51fbb8 in Data_ReAlloc /root/debug-fuzz-reslut/mkvalidator/foundation-source/corec/corec/array/array.c:87:24

SUMMARY: AddressSanitizer: heap-use-after-free /root/debug-fuzz-reslut/mkvalidator/foundation-source/libebml2/ebmlcrc.c:244:14 inEBML_CRCMatches
Shadow bytes around the buggy address:
  0x0c067fff80c0: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
  0x0c067fff80d0: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
  0x0c067fff80e0: fd fd fd fa fa fa 00 00 00 fa fa fa fd fd fd fa
  0x0c067fff80f0: fa fa fd fd fd fa fa fa fd fd fd fa fa fa 00 00
  0x0c067fff8100: 00 04 fa fa 00 00 00 01 fa fa 00 00 06 fa fa fa
=>0x0c067fff8110: fd[fd]fd fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8120: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8130: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8140: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8150: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8160: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==12870==ABORTING

location: foundation-source/libebml2/ebmlcrc.c:244

add a breakpoint in the EBML_CRCMatches function

look at $rsi : 0x67f788
and we look at the free chunk, so 0x67f788 is in the free chunk

How can we reproduce the issue?

Compile command I use:

clang corec/tools/coremake/coremake.c -o coremake && ./coremake gcc_linux_x64

make -C mkvalidator -e CC="clang -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e CXX="clang++ -g -O2 -fsanitize=address -fsanitize-address-use-after-scope" -e STRIP=""

reproduce the issue

unzip tests_xx.zip
mkvalidator ./tests_xx.mkv

poc:
tests_74.zip

the details about my environment.

• mkvalidator version:

$ ./mkvalidator --version
mkvalidator v0.5.2, Copyright (c) 2010-2015 Matroska Foundation
	file "--version"

• Host Operating System and version: Ubuntu 16.04.3 LTS
• Host CPU architecture: x86_64
• clang: clang version 11.0.0

Add Github action to verify the generated semantic files match the ones in libmatroska2

generate mkclean/mkvalidator tarballs on tagging

The tag name should tell whether it's for mkclean or mkvalidator.

The generated pkg.sh could be ported to Gitlab workflows.

Bad link at "Chapter Specifications"

Bad link (http://www.dvd-replica.com/DVD/) at Chapter Specifications https://matroska.org/technical/specs/chapters/index.html

More information on the DVD commands and format on DVD-replica, where we got most of the info about it. You can also get information on DVD from the DVDinfo project.

Is mkclean safe to use on newer audio formats?

I've heard it might not be "safe", can we get a official answer on this issue?

Is there any issue using mkclean with mkv with newer audio formats such as TrueHD, DTS?

Also why it seems that mediainfo information is lost?

Left is file after mkclean and right before.

Handle installation of libraries and tools via cmake

CMake has some tools to install the built packages. We have to make sure each library (static and dynamic) and program install in the proper place on UNIX systems (on Windows either static builds should be used or DLLs and programs have to be install in a single location).

Feat.req. Option to remove "mkclean 0.9.0 o from Lavf59.17.101" string from file header

subj

mkclean drops new elements

I'd like to clean a Matroska file but it seems to drop many elements in the process. For instance:

Create sample:
ffmpeg -f lavfi -i testsrc=d=1 -color_primaries smpte170m -color_trc bt709 -colorspace smpte170m -color_range mpeg -vf setfield=bff,setsar=40/27,setdar=4/3 ffmpeg.mkv

Then clean:

mkclean ffmpeg.mkv 
Progress 1/3: 100%
The TransferCharacteristics element at 464 is not part of profile 'matroska v2', skipping
The Primaries element at 472 is not part of profile 'matroska v2', skipping
The Range element at 476 is not part of profile 'matroska v2', skipping
The Colour element at 461 is not part of profile 'matroska v2', skipping
The FieldOrder element at -1 is not part of profile 'matroska v2', skipping
The SeekPreRoll element at -1 is not part of profile 'matroska v2', skipping
Progress 3/3: 100%
Finished cleaning & optimizing "clean.ffmpeg.mkv"

In the help doc, there is no option to use a matroska v4 profile:

mkclean
mkclean v0.8.10, Copyright (c) 2010-2011 Matroska Foundation
Usage: mkclean [options] <matroska_src> [matroska_dst]
Options:
  --keep-cues   keep the original Cues content and move it to the front
  --remux       redo the Clusters layout
  --doctype <v> force the doctype version
    1: 'matroska' v1
    2: 'matroska' v2
    3: 'matroska' v3
    4: 'webm'
    5: 'matroska' v1 with DivX extensions

Is it possible to clean without dropping the new elements?

replace the GPL lzo library with a port of lzo.net in C

This library should be ported to C
https://github.com/zivillian/lzo.net/tree/master/src

And verified it is valid.