Bug? "starttime" and "lengthSec" time values not converting to human readable times

[mkrufky]

Bug? "starttime" and "lengthSec" time values not converting to human readable times

[mkrufky]

here's a patched win32 build:
dvbtee.exe.zip

[mkrufky]

@andrewfblake , looking at http://adgile.blob.core.windows.net/general/ABC_Network.txt , I see that the dvbtee command prints a Just Jazz program event from 21:45 - 00:00:

store EIT-255: v15 | ts_id 577 | network_id 4112 service_id 583 | table id: 0x4e, last_table id: 0x4e
  20:15 - 21:45 : Jazztrack
  21:45 - 00:00 : Just Jazz

Is this information correct? The timestamps on some of the other programs look a bit fishy, so I'm not sure. Also, Just Jazz airs again in another EIT block:

store EIT-255: v1 | ts_id 577 | network_id 4112 service_id 583 | table id: 0x51, last_table id: 0x52
  19:50 - 21:00 : Just Jazz
  18:00 - 19:50 : Jazztrack
  21:00 - 22:50 : Jazztrack
  00:00 - 18:00 : Just Jazz
  22:50 - 00:00 : Just Jazz
  19:50 - 21:00 : Just Jazz
  18:00 - 19:50 : Jazztrack
  21:00 - 22:50 : Jazztrack
  00:00 - 18:00 : Just Jazz
  22:50 - 00:00 : Just Jazz
  18:00 - 20:00 : Jessica Nicholas
  20:00 - 21:00 : Just Jazz
  21:00 - 22:00 : Thursday Night Live
  00:00 - 18:00 : Just Jazz
  22:00 - 00:00 : Just Jazz
  18:00 - 19:00 : Talking Jazz
  19:00 - 21:00 : Just Jazz
  21:00 - 22:00 : Home Cooked
  22:00 - 00:00 : Just Jazz
  00:00 - 14:00 : Just Jazz

These timestamps definitely look fishy. I think there indeed is a bug here.

Meanwhile, the descriptor JSON block is simply a raw output of the timestamp parsed from the stream. Once we fix the issue of the possibly incorrect printed timestamps, we can add new timestamp elements to the descriptor objects in a datetime format.

[andrewfblake]

Ah ABC and that channel in particular is probably a little misleading @mkrufky as its a radio channel and its Just Jazz allllllllll day long... the Seven, Nine and TEN Networks are our main ones and the timestamps look ok in the tables at a quick glance but i'll compare them with my TV tomorrow and double check them! :-)

[mkrufky]

If you can confirm that the timestamps are parsed correctly (or help me to confirm any needed fixes) then we can add some parsed timestamp fields to the JSON object for easier parsing.

[andrewfblake]

OK, Seems like the time is simply offset by 4 hours in the tables. The duplicate listings of the shows seems to appear here on TV too so that must just be how they are broadcast.

Example 1:
Chanel 7
http://adgile.blob.core.windows.net/general/2016-06-03-7epg.txt
epg line: 62: "14:00 - 16:00 : The Morning Show - Weekend"
http://adgile.blob.core.windows.net/general/2016-06-03-7.jpg
Showing "The Morning Show" starting at 10:00am - 12pm on the TV, but 2pm - 4pm in the Output..

Example 2:
Channel 7mate:
epg line: 71: " 14:00 - 15:30 : AFL Game Day - Live"
http://adgile.blob.core.windows.net/general/2016-06-03-7mate.jpg
AFL Gme day showing to start at 10am and 2pm on the output table.

I also thought that there may be dupes coming through like the below, but it seems it shows that too on the TV EPG Output so must just be how EPG works! :-)

http://adgile.blob.core.windows.net/general/2016-06-03-7mate.jpg
epg line: 306:
18:30 - 19:00 : 2016 AFL Premiership Season - Live
19:00 - 22:04 : 2016 AFL Premiership Season - Live
22:04 - 23:05 : Mythbusters

I checked the TEN Network and it also had the same offset of 4 hours. (we're on Australian Eastern Standard time [no daylight savings] and +10 from GMT)... So wondering if the time is being broadcast in localtime here and perhaps in NYC its UTC so there is an added +4 hours offset?! Just a guess... :-)

Original file the above EPG was extraced from:
http://adgile.blob.core.windows.net/general/2016-06-03-7epg.ts

[mkrufky]

@andrewfblake , In NYC, television is broadcast using ATSC, a drastically different protocol standard for program service information, while DVB-T is used in AU.

It's possible that there is no bug at all. Perhaps the right time conversion functions are missing on your system and the build system's compat fallback didnt do the right thing... Can you post your config.h (from the top level of the libdvbtee tree after running ./configure) so I can see what your build configuration looks like? (and / or full output of the ./configure script)

[mkrufky]

...to add some perspective, when I analyze the same capture on my system with the following time* config:

/* Define to 1 if you have the `localtime_r' function. */
#define HAVE_LOCALTIME_R 1
/* Define to 1 if you have the <sys/time.h> header file. */
#define HAVE_SYS_TIME_H 1
/* Define to 1 if you have the `timegm' function. */
#define HAVE_TIMEGM 1
/* Define to 1 if you have the `_mkgmtime' function. */
/* #undef HAVE__MKGMTIME */

With your first example, I get:

  20:00 - 22:00 : The Morning Show - Weekend

...so I seem to be seeing +10 hours, while you are seeing +4 hours. Timezone could be a factor, but first I'll need to see your configuration

[rdp]

top level config.h
config.h.txt

[mkrufky]

...just as I thought. Your system uses a different codepath for the time parsing. I'll do another windows cross-build and see if I can solve this here (might take a few days)

[mkrufky]

My local windows cross-build config.h has the following time* config:

/* Define to 1 if you have the `localtime_r' function. */
/* #undef HAVE_LOCALTIME_R */
/* Define to 1 if you have the <sys/time.h> header file. */
#define HAVE_SYS_TIME_H 1
/* Define to 1 if you have the `timegm' function. */
/* #undef HAVE_TIMEGM */
/* Define to 1 if you have the `_mkgmtime' function. */
/* #undef HAVE__MKGMTIME */

and running the windows dvbtee.exe (using wine) with the same example here in NYC, I get:

  21:00 - 23:00 : The Morning Show - Weekend

Next I'll have to actually run it in windows and see what happens...

[rdp]

seems the same as the uploaded...

[mkrufky]

Running that same binary (previously tested under Linux using WINE) in Windows (virtualbox), I get:

  18:00 - 20:00 : The Morning Show - Weekend

EDIT - my windows VM was set up for the wrong timezone. I corrected that back to Eastern -0500, now I see:

  21:00 - 23:00 : The Morning Show - Weekend

this matched what I saw using WINE under Linux.

There may be something weird with the timezone offset handling going on in the windows cross build, but I'm not sure. I don't have the bandwidth to solve that myself right now, but patches are welcome :-)

In the meanwhile, I pushed a patch for testing that will add decoded unixTimeBegin and unixTimeEnd fields to the JSON object. This should hopefully solve your immediate issue for now, @andrewfblake

ecaa252

You can find this in the tabl43-decode-utc#17 branch. ( @rdp - please do not add this to your default build - this branch is only temporary )

To apply one of my branches for testing against rdp's cross build scripts, something like this should work:

Assuming you have already run the cross build, go back to it and issue the following commands. If this is a fresh build, run the default build once before proceeding.

(for win32)

cd sandbox/win32/libdvbtee
git remote update
git pull . origin/tabl43-decode-utc#17
cd ../../..

(for win64)

cd sandbox/x86_64/libdvbtee
git remote update
git pull . origin/tabl43-decode-utc#17
cd ../../..

re-run your cross build, the new dvbtee.exe will contain the latest changes from the desired branch.

[mkrufky]

...if I change my Windows timezone to Canberra, Melbourne, Sydney (UTC +10:00) I get:

  14:00 - 16:00 : The Morning Show - Weekend

Is this your timezone, @andrewfblake ?

Maybe you're right... perhaps it's being broadcast in local time there rather than UTC.

[andrewfblake]

Hey @mkrufky wow thank you for the help so far! I was using @rdp 's build (I have nooo idea how to build on a windows box and looking into it, though its waay over my head at the moment!)

Yup, I'm in Brisbane (AEST // UTC +10:00) which is currently the same as Canberra, Melbourne, Sydney (they have the addition of daylight savings)

Maybe it is?! The Morning Show aired at 10am locally so its still a few hours out oddly enough - but its an odd offset of 4 hours?! Though no biggie and easy to offset back when its coming from the json!! :-)

[andrewfblake]

amazing, much appreciated! thank you!!! Unixtimestamps coming through wonderfully!!