Giter Site home page Giter Site logo

grafov / m3u8 Goto Github PK

View Code? Open in Web Editor NEW
1.2K 40.0 302.0 749 KB

Parser and generator of M3U8-playlists for Apple HLS. Library for Go language. :cinema:

Home Page: http://tools.ietf.org/html/draft-pantos-http-live-streaming

License: BSD 3-Clause "New" or "Revised" License

Go 100.00%
m3u8-playlist hls manifest parsing library golang lib m3u8 playlist-parser playlist-generator

m3u8's Introduction

M3U8

This is the most complete opensource library for parsing and generating of M3U8 playlists used in HTTP Live Streaming (Apple HLS) for internet video translations.

M3U8 is simple text format and parsing library for it must be simple too. It does not offer ways to play HLS or handle playlists over HTTP. So library features are:

  • Support HLS specs up to version 5 of the protocol.
  • Parsing and generation of master-playlists and media-playlists.
  • Autodetect input streams as master or media playlists.
  • Offer structures for keeping playlists metadata.
  • Encryption keys support for use with DRM systems like Verimatrix etc.
  • Support for non standard Google Widevine tags.

The library covered by BSD 3-clause license. See LICENSE for the full text. Versions 0.8 and below was covered by GPL v3. License was changed from the version 0.9 and upper.

See the list of the library authors at AUTHORS file.

Project status

I moved away from videostreaming years ago and directly not used this code in my projects now. Hence the project mostly abandoned. Anyway I interested in keeping the code as useful as possible. I'll keep the eye on the issues when I have the time. Time is the biggest issue :|

  1. Any patches are welcome especially bugfixes.
  2. If you want to maintain the project open the issue or directly contact me.
  3. If you have alternatives (including the forks of this project) that you prefer to maintain by self, drop a link for including to this readme.

Install

go get github.com/grafov/m3u8

or get releases from https://github.com/grafov/m3u8/releases

Documentation GoDoc

Package online documentation (examples included) available at:

Supported by the HLS protocol tags and their library support explained in M3U8 cheatsheet.

Examples

Parse playlist:

	f, err := os.Open("playlist.m3u8")
	if err != nil {
		panic(err)
	}
	p, listType, err := m3u8.DecodeFrom(bufio.NewReader(f), true)
	if err != nil {
		panic(err)
	}
	switch listType {
	case m3u8.MEDIA:
		mediapl := p.(*m3u8.MediaPlaylist)
		fmt.Printf("%+v\n", mediapl)
	case m3u8.MASTER:
		masterpl := p.(*m3u8.MasterPlaylist)
		fmt.Printf("%+v\n", masterpl)
	}

Then you get filled with parsed data structures. For master playlists you get Master struct with slice consists of pointers to Variant structures (which represent playlists to each bitrate). For media playlist parser returns MediaPlaylist structure with slice of Segments. Each segment is of MediaSegment type. See structure.go or full documentation (link below).

You may use API methods to fill structures or create them manually to generate playlists. Example of media playlist generation:

	p, e := m3u8.NewMediaPlaylist(3, 10) // with window of size 3 and capacity 10
	if e != nil {
		panic(fmt.Sprintf("Creating of media playlist failed: %s", e))
	}
	for i := 0; i < 5; i++ {
		e = p.Append(fmt.Sprintf("test%d.ts", i), 6.0, "")
		if e != nil {
			panic(fmt.Sprintf("Add segment #%d to a media playlist failed: %s", i, e))
		}
	}
	fmt.Println(p.Encode().String())

Custom Tags

M3U8 supports parsing and writing of custom tags. You must implement both the CustomTag and CustomDecoder interface for each custom tag that may be encountered in the playlist. Look at the template files in example/template/ for examples on parsing custom playlist and segment tags.

Library structure

Library has compact code and bundled in three files:

  • structure.go — declares all structures related to playlists and their properties
  • reader.go — playlist parser methods
  • writer.go — playlist generator methods

Each file has own test suite placed in *_test.go accordingly.

Related links

Library usage

This library was successfully used in streaming software developed for company where I worked several years ago. It was tested then in generating of VOD and Live streams and parsing of Widevine Live streams. Also the library used in opensource software so you may look at these apps for usage examples:

Project status Go Report Card

Build Status Build Status Coverage Status

DeepSource

Code coverage: https://gocover.io/github.com/grafov/m3u8

Project maintainers

Thank to all people who contrubuted to the code and maintain it. Especially thank to the maintainers who involved in the project actively in the past and helped to keep code actual:

  • Lei Gao @leikao
  • Bradley Falzon @bradleyfalzon

New maitainers are welcome.

Alternatives

On the project start in 2013 there was no any other libs in Go for m3u8. Later the alternatives came. Some of them may be more fit current standards.

Drop a link in issue if you know other projects.

FYI M3U8 parsing/generation in other languages

m3u8's People

Contributors

aleksi avatar andir avatar andyborn avatar bcasinclair avatar bradleyfalzon avatar dennwc avatar dlsniper avatar eric avatar grafov avatar iivarih avatar imjma avatar itsjamie avatar konradreiche avatar kz26 avatar leikao avatar mbowbc avatar md2k avatar mistobaan avatar mlafeldt avatar rogerpales avatar shawnps avatar sinkers avatar soldiermoth avatar waffle-iron avatar yyoshiki41 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

m3u8's Issues

#EXT-X-MEDIA parsing was broken

It seems like parsing of #EXT-X-MEDIA in decodeLineOfMasterPlaylist() did nothing because in not assigned results to a resulting playlist.

#EXT-X-PLAYLIST-TYPE parsing broken

Consider the following playlist:

#EXTM3U
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-TARGETDURATION:15
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:5,
0.ts
#EXTINF:5,
1.ts
#EXTINF:5,
2.ts
#EXTINF:12,
3.ts
#EXT-X-ENDLIST

Calling m3u8.DecodeFrom() results in the error "bad verb %s for integer".

SeqId is not set when parsing media playlist

Upon decoding a media playlist, I expected SeqId of each segment to contain the sequence id of each segment. For example, if SeqNo of the playlist were 123 and it had three segments, I would expect the SeqId of the three segments to be 123, 124, and 125. However, they are all 0, regardless of SeqNo.

My current workaround it to populate the sequence ids after the fact myself:

pls, listType, err := m3u8.DecodeFrom(reader, true)

switch listType {
    case m3u8.MEDIA:
        // Cast to media playlist type
        mediapls := pls.(*m3u8.MediaPlaylist)

        // Populate segment sequence id from playlist sequence number
        for i, segment := range mediapls.Segments {
            if segment == nil {
                break
            }
            mediapls.Segments[i].SeqId = mediapls.SeqNo + uint64(i)
        }
        ...

Add support for multiple SCTE-35 tag syntax

There appear to be various non-standard implementations of SCTE-35 integration with HLS.

The recent draft of HLS now has an official method to support SCTE-35 tags as well as the cue points. See: https://www.ietf.org/rfcdiff?url1=draft-pantos-http-live-streaming-18&url2=draft-pantos-http-live-streaming-19

I'd like this library to support the new standard, but #40 and #41 introduced support for a different standard. There's also at least two other formats I've seen.

I think we need a method to read multiple standards, store and manipulate them consistently and write multiple standards. With the focus being on the HLS RFC's method, and the existing format already supported by this library - but this may require some breaking changes.

This issue is open to discuss this.

Other support that could be added later: http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/primetime/pdfs/PrimetimeDigitalProgramInsertionSignalingSpecification1_1.pdf

Change license to a more permissive

This library uses the GPL license which makes it actually impossible for us to use as it would require us to use the GPL license as well as disclose our source code. Working on a commercial product it would mean that we need to move away from this library and roll our own implementation.

How deliberate is the choice of GPL. Would other, more permissive licenses, like MIT or Apache be a feasible alternative?

Issues related to iframe playlist and bytes ranges

In Reader.Go:
in the following line:
case !state.tagIframeStreamInf && strings.HasPrefix(line, "#EXT-X-I-FRAME-STREAM-INF:"):
you use only the first iframe manifest found and not all.

also, the playlist iframe segments has no offset and linit due to lin "} else if state.tagRange { " in:

case !strings.HasPrefix(line, "#"):
if state.tagInf {
p.Append(line, state.duration, title)
state.tagInf = false
} else if state.tagRange {

because i have state.tagInf and state.tagRange so "else if " isnt good enough and should be:

if state.tagInf {
p.Append(line, state.duration, title)
state.tagInf = false
}
if state.tagRange {

SeqNo bug, increment needed only when chunk removed

m3u8/writer.go at master · grafov/m3u8

Here is necessary code.

It cause following bug,

  • first
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:12
#EXTINF:10.000,
0ts
  • next
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:12
#EXTINF:10.000,
0ts
#EXTINF:10.000,
1ts
  • correct
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:12
#EXTINF:10.000,
0ts
#EXTINF:10.000,
1ts

#EXT-X-ALLOW-CACHE:NO

When you encode a media playlist, this tag is always added.

I'm curious what the reasoning for doing this is when you are also are able to set the media type to VOD, which in most of my experiences should either not specify if caching should be enabled or not (leaving it up to the player) or use a value of yes.

Export winsize

When decoding a playlist with a window size bigger than 8 then an encode will result in a truncated playlist. This due to this line here.

media, err = NewMediaPlaylist(8, 1024) // TODO make it autoextendable

Would it be possible to update the winsize afterwards through Count() internally?

io.ReadCloser interface support in decodeFrom

Hello!

Is there a way to download an m3u8 file in memory and parse it at once without writing it down to a file?

I have problems with this code not really decoding the entire playlist:

res, err := http.Get(url)
defer res.Body.Close()

p := m3u8.NewMasterPlaylist()
err = p.DecodeFrom(res.Body, false)

Where the playlist looks like this:

#EXTM3U

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,URI="audio/stereo/en/128kbit.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo",LANGUAGE="dubbing",NAME="Dubbing",DEFAULT=NO,AUTOSELECT=YES,URI="audio/stereo/none/128kbit.m3u8"

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="surround",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,URI="audio/surround/en/320kbit.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="surround",LANGUAGE="dubbing",NAME="Dubbing",DEFAULT=NO,AUTOSELECT=YES,URI="audio/stereo/none/128kbit.m3u8"

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Deutsch",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="de",URI="subtitles_de.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",URI="subtitles_en.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Espanol",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="es",URI="subtitles_es.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fr",URI="subtitles_fr.m3u8"

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=258157,CODECS="avc1.4d400d,mp4a.40.2",AUDIO="stereo",RESOLUTION=422x180,SUBTITLES="subs"
video/250kbit.m3u8

I only get

#EXTM3U
#EXT-X-VERSION:3

when printing out p. res.Body is a io.ReadCloser type.
Am I doing something wrong?

m3u8.DecodeFrom create playlistMedia with 1024 Segments

https://github.com/grafov/m3u8/blob/master/reader.go#L151

I have mediaPlaylistFile with 1203 segments (c_00000000.ts - c_00001202.ts).
But when I do: m3u8.DecodeFrom I got len(playlistMedia.Segments) == 1024!:

playlistFile, _ := os.Open(pathToMediaPlaylistFile)
defer playlistFile.Close()
playlist, listType, _ = m3u8.DecodeFrom(bufio.NewReader(playlistFile), true)
switch listType {
case m3u8.MEDIA:
    playlistMedia := playlist.(*m3u8.MediaPlaylist)
    fmt.Printf("%d\n", len(playlistMedia.Segments))
}

How can I iterate over more than 1024 segments?
How can I get total duration?
How can I get full URI slice?

Decoding MediaPlaylist, winsize = 0, Encode has no segments.

When decoding a MediaPlaylist, and then reencoding it with a window size of zero. No segments are in the resulting file.

I traced it down to a the for loop inside Encode that loops the Segments, it currently checks i < p.winsize. Which works for a sliding playlist, but where VOD is supposed to set winsize to zero, it results in no segments being printed.

I was able to get rid of the issue by changing the for loop condition to be i <= p.winsize.. but I'm not sure if that effects live playlists in a poor way.

Under load encoder does not perform correctly

I've created a PR introducing some tests that simulates "high traffic" and validates the correctness of the playlist.

I've found that under load rendering the playlist as: MediaPlaylist.String() returns incomplete playlists causing the video to stop playing.

I'll be working on this but would like to know if you run against this issue and any help on fixing it it's also appreciated 💪

Some struct fields are not go-idiomatic

Hi there,

Some field names in the structs in structure.go are not go-idiomatic, for example:

ProgramId (ProgramID)
GroupId (GroupID)
SeqId (SeqID)
Iframe (IFrame)

Happy to provide a PR if you want to go in this direction.

Cheers,

Decoding media playlist with title

Double check if this works, as I don't think it does, looks like it parses the title in decodeLineOfMediaPlaylist, but assigns it to a local variable which isn't added the the decodingState.

Feature Request: m3u8.MediaSegment implements fmt.Stringer

hello dev.
Thanks you for your great work. the library is very useful :-)

I'd like to implement (m MediaSegment) String()string for testing like below.
When failing test, I'd like to report as m3u8 text.

package something

import (
	"github.com/grafov/m3u8"
	"testing"
)

func TestXXX(t *testing.T) {
	var expected, actual m3u8.MediaSegment

        actual = do_something()
	if expected.SeqId != actual.SeqId {
		t.Fatalf("expected:%s,  but actual:%s\n", expected.String(), actual.String())
	}
}

there are already (p *MediaPlaylist) String() string and it converts m3u8.MeidaSegment to string.
We can use the procedure to implement function I request.

If you agree to the issue but don't have enough time, I'd like to send PR.

Некорректный парсинг #EXT-X-DISCONTINUITY

Привет!!

По-моему у тебя некорректно парсится #EXT-X-DISCONTINUITY: в reader.go

    case !state.tagInf && strings.HasPrefix(line, "#EXTINF:"):
        state.tagInf = true
        state.listType = MEDIA
        params := strings.SplitN(line[8:], ",", 2)
        if state.duration, err = strconv.ParseFloat(params[0], 64); strict && err != nil {
            return errors.New(fmt.Sprintf("Duration parsing error: %s", err))
        }
        title = params[1]
    case !state.tagDiscontinuity && strings.HasPrefix(line, "#EXT-X-DISCONTINUITY"):
        state.tagDiscontinuity = true
        state.listType = MEDIA
    case !strings.HasPrefix(line, "#"):
        if state.tagInf {
            p.Append(line, state.duration, title)
            state.tagInf = false
        } else if state.tagRange {
            if err = p.SetRange(state.limit, state.offset); strict && err != nil {
                return err
            }
            state.tagRange = false
        } else if state.tagDiscontinuity {
            state.tagDiscontinuity = false
            if err = p.SetDiscontinuity(); strict && err != nil {
                return err
            }
        } else if state.tagProgramDateTime {
            state.tagProgramDateTime = false
            if err = p.SetProgramDateTime(state.programDateTime); strict && err != nil {
                return err
            }
        }

В блоке case !strings.HasPrefix(line, "#"), если до этого выставлено в true два флага - tagInf и tagDiscontinuity, Discontinuity не выставится, т.к. ты используешь if else вместо if.

Я это обошел, проставив перевод строки после чанка с Discontinuity, чтобы зайти в вышеназванную секцию два раза!

Может это по стандарту надо ставить перевод строки?

missing new line after #EXT-X-DISCONTINUITY-SEQUENCE

Hey! I found that there is a missing new line after the #EXT-X-DISCONTINUITY-SEQUENCE, so when rendering a mediaplaylist with a DiscontinuitySeq different than 0 a new line is missing, so is being displayed as:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-MEDIA-SEQUENCE:15150186350000
#EXT-X-TARGETDURATION:2
#EXT-X-DISCONTINUITY-SEQUENCE:1#EXTINF:2.000,

instead of:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-MEDIA-SEQUENCE:15150186350000
#EXT-X-TARGETDURATION:2
#EXT-X-DISCONTINUITY-SEQUENCE:1
#EXTINF:2.000,

I've created a PR fixing this.

Question:`AVERAGE-BANDWIDTH` and `FRAME-RATE` in `EXT-X-STREAM-INF`

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-STREAM-INF:BANDWIDTH=2340800,AVERAGE-BANDWIDTH=2340800,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=25.000
demo1080.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1790800,AVERAGE-BANDWIDTH=1790800,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=25.000
demo720.m3u8

https://github.com/grafov/m3u8/blob/master/reader.go#L272-L302

How to get AVERAGE-BANDWIDTH and FRAME-RATE, and others if we have customised params?

Controlling the media sequence values of individual segments

I have an application that records HLS streams and I often need to replicate the media sequence values of some source stream. One thing I encounter fairly frequently is that the media sequence is restarted or it could be that some media segments are dropped. In any case, I need more precise control of the media sequence values of individual segments. It is not simply a matter of incrementing the sequence by one.

I already have this working correctly. This is how it works.

I call MediaPlaylist.Remove() if I have to, then I create a segment, set its SeqId and call MediaPlaylist.AppendSegment().

Now I find the first segment in the playlist and set MediaPlaylist.SeqNo = firstSegment.SeqId.

I don't like how I find the first segment, and I was going to ask if you could expose a function that would return the head of the list.

I was also hoping that the changes in #116 would be helpful for my use case. In theory, the playlist could respect the sequence IDs that I set on the segments, and automatically set the media sequence of the playlist to the media sequence of the first segment (although this would break backwards compatibility).

But unfortunately, PR #124 breaks what I have right now. The SeqId values that I set on the segments before I append them to the playlist are lost.

So now I have two problems:

  1. I need the SeqId values that I set on the segments to be preserved.

  2. I would like a way of accessing the first segment in the playlist.

The second thing should be easy, but I'm worried about the first one.

Do you think you could support such a use case?

m3u8.Alternative allows invalid combinations

You are able to add a m3u8.Alternative with Default true and autoselect set to no, but this will break playback on Apple devices and not pass the mediastreamvalidator test.

Not sure if this is clearly in any spec but should not be allowed.

Question: #EXT-X-DISCONTINUITY-SEQUENCE support

I noticed that there is #EXT-X-DISCONTINUITY support as seen in m3u8.md

| EXT-X-DISCONTINUITY | MED | 1 | 0.2 |

Is there future support for the #EXT-X-DISCONTINUITY-SEQUENCE (though it's part of protocol version 6)

| EXT-X-DISCONTINUITY-SEQUENCE | MED | 6 | |

Based on my analysis, the EXT-X-DISCONTINUITY-SEQUENCE would be part of MediaPlaylist in structure.go and the reader.go can add a case like so:

case strings.HasPrefix(line, "#EXT-X-DISCONTINUITY-SEQUENCE:"):

and eventually the writer.go can inject it while Encoding.

Add more sample playlists

I limited with access to streaming servers with old HLS versions with playlists featured v3 protocol. I need more sample playlists from real production systems featured tags from version 4 or higher of the protocol (playlists with alternative media, multiple langs, closed captions, i-frames etc.). Please send me such playlists (master and media) for including in library samples and use in unit tests. Please provide your samples with name and version of streaming software. Better to fork repo and make pull request with your sample files placed in /sample-playlists directory of the repo. But you may just send me link to your streams with permission to copy playlists to the repo samples. Links to legal public streams would be appreciated too. Thank you.

Looking for a new project maintainer

The stats show that the library still in use by many people and the code still actual. Thanks a lot to all the contributors! Of course, this lib is the small piece of code but it added some value to the applicability of Go for writing the video casting applications. Unfortunately, working with video is out of the scope of my current interests. I coding mostly on Go since 2012 and think this language was the right choice for me. But the areas where I apply my programming skills shifted a lot from the time when I started the M3U8 library. So in future, I could add minor changes like code style and readme updates but I have no resources to review the major changes that require the actual knowledge in modern M3U8 standard and in the areas of the video casting.

Now the project has two maintainers — me and @bradleyfalzon. Bradley helped a lot with reviewing and contributing. But it seems we need more people who currently working in the area of video casting and interested in support of this lib. There are a lot of issues that could be reviewed and merged.

I could still keep my rights of the project owner for the project safety. And keep the main purpose of the library — be simple and allow basic features for parsing and generating M3U8 playlists.

Drop a comment if you would ready to become the maintainer of the project.

EXT-X-STREAM-INF & EXT-X-I-FRAME-STREAM-INF grouping

Typically we need to slice a playlist, and I'm no HLS expert, but I noticed from other examples, it seems important that the EXT-X-STREAM-INF and EXT-X-I-FRAME-STREAM-INF are together, for e.g.

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=11301000,AVERAGE-BANDWIDTH=4248000,CODECS="mp4a.40.2,avc1.42C028",RESOLUTION=1920x822
1080p/rendition.m3u8
#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1599000,AVERAGE-BANDWIDTH=531073,CODECS="avc1.42C028",RESOLUTION=1920x822,URI="1080p/rendition-iframe-index.m3u8"`

How do people typically do this? Sort on Resolution? https://godoc.org/github.com/grafov/m3u8#VariantParams

Decode got error while `#EXT-X-PROGRAM-DATE-TIME' exists in media playlist

When I use m3u8 to decode a media playlist, it got error:

playlist is empty

The m3u8 file just like this:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PROGRAM-DATE-TIME:2018-12-31T09:47:22+08:00
#EXT-X-TARGETDURATION:15

#EXTINF:14.666000,
20181231/0555e0c371ea801726b92512c331399d_00000000.ts
#EXTINF:13.698000,
20181231/0555e0c371ea801726b92512c331399d_00000001.ts
#EXTINF:14.668000,
20181231/0555e0c371ea801726b92512c331399d_00000002.ts
#EXTINF:13.200000,
20181231/0555e0c371ea801726b92512c331399d_00000003.ts
#EXT-X-ENDLIST

Possibility of changing 'ceiling' to 'round' in EXT-X-TARGETDURATION algorithm

Hello,

I would like to propose changing the algorithm that determines the value of EXT-X-TARGETDURATION in HLS playlists from a ceiling round to a regular round, the relevant code is here:

m3u8/writer.go

Lines 307 to 309 in 6ab8f28

if p.TargetDuration < seg.Duration {
p.TargetDuration = math.Ceil(seg.Duration)
}

The reason is that Apple changed their recommendation for EXT-X-TARGETDURATION in revision 12 of the HLS spec to be greater than or equal EXTINF, when rounded to the nearest integer, see https://tools.ietf.org/html/draft-pantos-http-live-streaming-12#section-3.4.2

You can read the previous spec here: https://tools.ietf.org/html/draft-pantos-http-live-streaming-11#section-3.4.2

The current algorithm, which does a ceiling, does create playlists that conform to the spec.

BUT it can also arbitrarily create playlists with 11s target durations when the user requests 10s (when it would be perfectly valid to have a 10s target duration). Generating playlists that exactly match the spec recommendation is preferable for us, and can also help with scrubbing accuracy on a wider range of players.

I've opened this issue as more of a discussion because I understand that there might be some pushback given that the current behavior technically creates spec-compliant playlists. If there is any chance of this being activated as a feature flag this would also be acceptable for us.

I will submit a preliminary PR shortly that implements this.

Question: Custom TAGs

Hi @grafov , i not sure maybe i overlooked, but do we have support of custom TAGs, for example it can be handy for implementation of custom DRM like system.

Thanks in advance.

Question: #EXT-X-MEDIA:TYPE=AUDIO

Looks like i can get information about alternatives only from first element from variants array.
Playlist example:

#EXTM3U
#EXT-X-VERSION:5
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="Audio 0",DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="ger",URI="audio_3/playlist.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="Audio 1",DEFAULT=NO,AUTOSELECT=YES,LANGUAGE="ger",URI="audio_4/playlist.m3u8"
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=997142,CODECS="avc1.4d0029",AUDIO="audio"
track_0_800/playlist.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=1880000,CODECS="avc1.4d0029",AUDIO="audio"
track_1_1600/playlist.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=0,BANDWIDTH=2736607,CODECS="avc1.4d0029",AUDIO="audio"
track_2_2500/playlist.m3u8

Dumped Variants from masterpl := p.(*m3u8.MasterPlaylist)

{
    "Variants": [
        {
            "URI": "track_0_800/playlist.m3u8",
            "Chunklist": null,
            "ProgramId": 0,
            "Bandwidth": 997142,
            "Codecs": "avc1.4d0029",
            "Resolution": "",
            "Audio": "audio",
            "Video": "",
            "Subtitles": "",
            "Captions": "",
            "Name": "",
            "Iframe": false,
            "Alternatives": [
                {
                    "GroupId": "audio",
                    "URI": "audio_3/playlist.m3u8",
                    "Type": "AUDIO",
                    "Language": "ger",
                    "Name": "Audio 0",
                    "Default": true,
                    "Autoselect": "YES",
                    "Forced": "",
                    "Characteristics": "",
                    "Subtitles": ""
                },
                {
                    "GroupId": "audio",
                    "URI": "audio_4/playlist.m3u8",
                    "Type": "AUDIO",
                    "Language": "ger",
                    "Name": "Audio 1",
                    "Default": false,
                    "Autoselect": "YES",
                    "Forced": "",
                    "Characteristics": "",
                    "Subtitles": ""
                }
            ]
        },
        {
            "URI": "track_1_1600/playlist.m3u8",
            "Chunklist": null,
            "ProgramId": 0,
            "Bandwidth": 1880000,
            "Codecs": "avc1.4d0029",
            "Resolution": "",
            "Audio": "audio",
            "Video": "",
            "Subtitles": "",
            "Captions": "",
            "Name": "",
            "Iframe": false,
            "Alternatives": null
        },
        {
            "URI": "track_2_2500/playlist.m3u8",
            "Chunklist": null,
            "ProgramId": 0,
            "Bandwidth": 2736607,
            "Codecs": "avc1.4d0029",
            "Resolution": "",
            "Audio": "audio",
            "Video": "",
            "Subtitles": "",
            "Captions": "",
            "Name": "",
            "Iframe": false,
            "Alternatives": null
        }
    ],
    "Args": "",
    "CypherVersion": ""
}

This is correct behaviour, shouldn't we have information about alternatives for each variant?

Add values to VariantParams

Sorry, I am a Golang newbie and my use case is to introduce a new key into the variant called 'rendition-id' to help clients create a rendition switcher.

package main

import (
	"fmt"

	"github.com/grafov/m3u8"
)

// https://godoc.org/github.com/grafov/m3u8#VariantParams

type myVariantParams struct {
	*m3u8.VariantParams
	RenditionId string
}

func main() {

	masterPlaylist := m3u8.NewMasterPlaylist()

	pp, _ := m3u8.NewMediaPlaylist(3, 5)
	for i := 0; i < 5; i++ {
		pp.Append(fmt.Sprintf("test%d.ts", i), 5.0, "")
	}

	// WORKS
	masterPlaylist.Append("chunklist2.m3u8", pp, m3u8.VariantParams{ProgramId: 123, Bandwidth: 1500000, Resolution: "576x480"})

	// Trying to extend VariantParams doesn't !
	// masterPlaylist.Append("chunklist2.m3u8", pp, myVariantParams{VariantParams: &m3u8.VariantParams{ProgramId: 123, Bandwidth: 1500000, Resolution: "576x480"}, RenditionId: "foo"})

	fmt.Println(len(masterPlaylist.Variants))
	masterPlaylist.ResetCache()

	fmt.Println(masterPlaylist.Encode().String())

}

Perhaps this is the same as #82. Anyway, just thought I'd convey my use case. My tact now is to fork and add the missing values.

Incorrect versions modules.

$ go list -m -versions github.com/grafov/m3u8
github.com/grafov/m3u8 v0.2.1 v0.2.2 v0.3.1 v0.3.2 v0.3.3 v0.4.1 v0.4.2 v0.4.3 v0.5.1 v0.5.2 v0.6.1
Release 'v0.10' is incorrect for go modules and https://semver.org/

Releases must be like 'v0.10.0', NOT 'v0.10'
Last correct release is v0.6.1
Please rename all incorrect tags to correct working with go modules. Thx

Why is MediaPlaylist.Segments a ring buffer instead of a normal slice?

I've been trying to reason out why the usage of MediaPlaylist.Segments is why it is and I have trouble coming up with good reasoning.

It seems that head and tail are used internally but not exposed anywhere publicly, so it is impossible to get a MediaPlaylist.Segments that matches the internal state.

It doesn't make sense to me why we can't just have MediaPlaylist.Segments be a strait forward slice where the capacity always maps to what is really defined.

Does anyone have any insight for why it is implemented this way?

Decoding a media playlist which has #EXT-X-KEY without any segments causes panic

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x3efcb]

goroutine 1 [running]:
github.com/grafov/m3u8.decodeLineOfMediaPlaylist(0x20821e7e0, 0x20827a000, 0x208256000, 0x0, 0x0, 0x1, 0x0, 0x0)
    /Users/iivari/go/src/github.com/grafov/m3u8/reader.go:478 +0x2b0b

how to reproduce:

main.go:

package main

import (
    "github.com/grafov/m3u8"
    "os"
    "bufio"
)

func main() {
    m3u8.DecodeFrom(bufio.NewReader(os.Stdin), true)
}

test.m3u8:

#EXTM3U
#EXT-X-VERSION:2
#EXT-X-TARGETDURATION:9
#EXT-X-KEY:METHOD=AES-128,URI="http://localhost:20001/key?ecm=AAAAAQAAOpgCAAHFYAaVFH6QrFv2wYU1lEaO2L3fGQB1%2FR3oaD9auWtXNAmcVLxgRTvRlHpqHgXX1YY00%2FpdUiOlgONVbViqou2%2FItyDOWc%3D",IV=0X00000000000000000000000000000000

$cat test.m3u8 | go run main.go

After debbugging this for a while I came up with the following fix:

diff --git a/reader.go b/reader.go
index 770d97f..4b58054 100644
--- a/reader.go
+++ b/reader.go
@@ -163,6 +163,11 @@ func decode(buf *bytes.Buffer, strict bool) (Playlist, ListType, error) {
                        break
                }

+               // possible fix...
+               if len(line) < 1 {
+                       break
+               }
+
                err = decodeLineOfMasterPlaylist(master, state, line, strict)
                if strict && err != nil {
                        return master, state.listType, err

Support for (non standard) Name attribute in #EXT-X-STREAM-INF tag

Hi,

I've begun using this extension for some manifest file manipulation, but come across a small issue where I'm using a non standard extension within the HLS spec that's been implemented by Wowza and JWPlayer.

The functionality adds an extra parameter to #EXT-X-STREAM-INF called Name which is added by Wowza and used by JWPlayer to show a more useful name for the variant in a client's quality selector. See http://support.jwplayer.com/customer/portal/articles/1430240-hls-adaptive-streaming

Note this shouldn't be confused with Name in the #EXT-X-MEDIA.

Would you accept an PR to add this functionality? It would add a Name field added to VariantParams struct (similar to the same tag in Alternative struct), additional support in decodeLineOfMasterPlaylist and (p *MasterPlaylist) Encode, along with corresponding tests, and sample-playlist.

Sample playlist:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1828000,NAME="4 high",RESOLUTION=896x504
chunklist_b1828000_t64NCBoaWdo.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1378000,NAME="3 med",RESOLUTION=768x432
chunklist_b1378000_t64MyBtZWQ=.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=678000,NAME="2 med",RESOLUTION=512x288
chunklist_b678000_t64MiBtZWQ=.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=438000,NAME="1 low",RESOLUTION=384x216
chunklist_b438000_t64MSBsb3c=.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=128000,NAME="0 audio"
chunklist_b128000_t64MCBhdWRpbw==.m3u8

Media Playlist Set* methods modifying wrong segments

I believe there maybe an issue with the Set* methods when the media playlist capacity is not a power of 2. It appears that once the playlist is full, a Set* (such as SetDiscontinuity() or SetKey()) sets the correct value on the wrong segment.

The following should demonstrate this, ignore the fact I'm ignoring errors, even when checked the problem still occurs. The SetKey() method best shows the issue as I can set the key URI (and IV) to be the same as the segment URI to better illustrate the problem, but the same occurs with SetDiscontinuity() (and likely other methods that use the p.Segments[(p.tail-1)%p.capacity] calculation).

package main

import (
    "fmt"

    "github.com/grafov/m3u8"
)

func main() {
    // OK when size is 1,2,4,8,16....
    // Not OK when size is 3,5,6,7,9,10...
    size := uint(5)
    pls, _ := m3u8.NewMediaPlaylist(size, size)

    for i := uint(0); i < size; i++ {
        uri := fmt.Sprintf("uri-%d", i)
        _ = pls.Append(uri+".ts", 4, "")
        _ = pls.SetKey("AES-128", uri+".key", fmt.Sprintf("%d", i), "", "")
    }

    fmt.Print(pls)
}

Output

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-TARGETDURATION:4
#EXT-X-KEY:METHOD=AES-128,URI="uri-4.key",IV=4 <- the 5th iteration overwrote the 1st's
#EXTINF:4.000,
uri-0.ts
#EXT-X-KEY:METHOD=AES-128,URI="uri-1.key",IV=1 <- 2nd iteration correct
#EXTINF:4.000,
uri-1.ts
#EXT-X-KEY:METHOD=AES-128,URI="uri-2.key",IV=2 <- 3rd iteration correct
#EXTINF:4.000,
uri-2.ts
#EXT-X-KEY:METHOD=AES-128,URI="uri-3.key",IV=3 <- 4th iteration correct
#EXTINF:4.000,
uri-3.ts
#EXTINF:4.000,
uri-4.ts  <- 5th iteration is missing the EXT-X-KEY

Generally, it looks like the problem is that the .Append() method sets playlist tail property (p.tail) to be modulo capacity and stores it back in p.tail (p.tail is always between 0 and p.capacity). Whereas the Set* methods assume this is a incrementing integer (based on they're calculation of (p.tail-1)%p.capacity). Essentially, because p.tail wraps to zero and because it is a uint, when p.tail is 0 the result of p.tail-1 is 18446744073709551615. Changing this to an int doesn't quite fix things either, at least not without still requiring further changes within the Set* methods (but this may also be the preferred solution) and more casting.

I believe a simple change maybe:

@@ -275,8 +276,8 @@ func (p *MediaPlaylist) Append(uri string, duration float64, title string) error
        seg.URI = uri
        seg.Duration = duration
        seg.Title = title
-       p.Segments[p.tail] = seg
-       p.tail = (p.tail + 1) % p.capacity
+       p.Segments[p.tail%p.capacity] = seg
+       p.tail++
        p.count++
        if p.TargetDuration < duration {
                p.TargetDuration = math.Ceil(duration)

There's other methods to fix it in each Set* method, via (something similar to, but perhaps using a private tail() method to obtain the correct tail):

+               tail := p.tail - 1
+               if p.tail == 0 {
+                       tail = p.capacity - 1
+               }
+               // OR: tail := int(math.Min(float64(p.tail-1), float64(p.capacity-1)))
-               p.Segments[(p.tail-1)%p.capacity].Key = &Key{method, uri, iv, keyformat, keyformatversions}
+               p.Segments[tail].Key = &Key{method, uri, iv, keyformat, keyformatversions}

I wanted to get hear others' thoughts on this before I go too much further, as this maybe a misunderstanding, maybe another solution, or this proposed change may cause other unintended consequences. This isn't a straight forward issue as there's multiple causes (depending on your opinion) and multiple solutions.

I'll be happy to send in a PR + tests and also address the Remove() methods (as well as the corresponding head variable), just focusing on the Append() and Set* methods first.

Note, I can't reproduce the issue when there capacity is a power of 2, e.g.: 1,2,4,8,16 - this may explain why the issue hasn't been discussed before?

I think there's two ways to solve this, move tail/head to ints and handle negatives in Set* methods, or use the above method (tail to always be incrementing). I'll continue to have a bit of a think about this, but I want to hear others' opinions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.