mq4l30ns/android-webcast

Android app to extract video (file/stream) URLs from websites and watch them elsewhere (internal/external video player, Google Chromecast, ExoAirPlayer).

WebCast

Android app to extract video (file/stream) URLs from websites and watch them elsewhere (internal/external video player, Google Chromecast, ExoAirPlayer).

Screenshots..

---

Tour

• BrowserActivity is shown when the app is started, and includes:
  • a very basic web browser
    • address bar
    • WebView
  • actionbar icons:
    • bookmark toggle
      • add/remove current website URL to/from list of persistently saved Bookmarks
  • actionbar menu items:
    • Bookmarks
      • open drawer on left: Bookmarks
    • Videos
      • open drawer on right: Videos
    • Settings
      • open SettingsActivity
    • Exit
      • close all UI and exit the app
  • drawer on left: Bookmarks
    • contains a persistent list of:
      • website URLs that have been saved via the bookmark toggle icon
      • video URLs that have been saved via the Videos drawer
    • click a list item to:
      • open website URL in WebView
      • watch video URL
        • SettingsActivity determines the particular action to be performed
    • long click a list item to:
      • rename
      • delete
  • drawer on right: Videos
    • contains a transient list of video URLs that have been found on the web page that is currently loaded in the WebView
      • this list is cleared each time the WebView navigates to a new web page
    • click a list item to:
      • add video URL to list of persistently saved Bookmarks
      • watch video URL
        • SettingsActivity determines the particular action to be performed
    • long click a list item to:
      • delete
• SettingsActivity is started from the actionbar menu in BrowserActivity, and includes:
  • Video Player to select whether to watch videos using..
    • internal w/ Chromecast sender
      • start VideoActivity
    • external
      • start Activity chooser w/ an implicit Intent
        • action
          • android.intent.action.VIEW
        • data
          • video URL
        • type
          • mime-type for format of video
        • extras
          • referUrl
            • (String) referer URL
            • used by ExoAirPlayer
    • ExoAirPlayer sender
      • start ExoAirPlayerSenderActivity
    • HLS-Proxy configuration
      • start Activity chooser w/ an implicit Intent
        • action
          • android.intent.action.VIEW
        • data
          • http://webcast-reloaded.surge.sh/proxy.html#/watch/${base64_video}/referer/${base64_referer}
        • type
          • "text/html"
      • start HlsProxyConfigurationActivity
        • only when there is no Activity having a matching Intent filter
          • which should never happen, since any standard web browser should offer to handle this Intent
• VideoActivity is started when a video URL is watched using the internal video player, and includes:
  • ExoPlayer
    • displays an icon in lower right corner of video controls toolbar to toggle fullscreen mode on/off
  • Chromecast sender
    • displays an actionbar cast icon when at least one Google Chromecast is detected on LAN
    • when connected to a Chromecast
      • video URLs are communicated to the receiver app running in the Chromecast
      • the Chromecast receiver app loads the video URL in an embedded HTML5 video player
        • transfer of video data occurs directly between the Chromecast and the server that hosts the video URL
        • transfer would not be effected by any of the following events:
          • VideoActivity stopped
          • BrowserActivity stopped
          • WebCast app exited
          • Android device powered off
  • list of video URLs
    • click a list item to:
      • play video URL
        • if connected to a Chromecast:
          • on Chromecast
        • otherwise:
          • on Android, in ExoPlayer
            • all HTTP requests include the referer url
• ExoAirPlayerSenderActivity is started when a video URL is watched using the ExoAirPlayer sender, and includes:
  • WebView that loads a single web page
    • URL of the web page depends on version of Android
      • Android 5.0 and newer
        • version using ES6+ modern javascript
      • Android 4.x and older
        • version using ES5 compliant javascript
    • URL hash contains:
      • #/watch/${base64_video}/referer/${base64_referer}
  • web page reads data from URL hash and pre-populates fields:
    • video url
    • referer url
  • web page reads data from cookies and pre-populates fields:
    • host
    • port
    • https
  • provides a basic UI to control any ExoAirPlayer receiver app that is reachable through the network
• HlsProxyConfigurationActivity is started when a video URL is watched using HLS-Proxy configuration on a device without any available web browser, and includes:
  • WebView that loads an ES5 compliant web page
    • URL hash contains:
      • #/watch/${base64_video}/referer/${base64_referer}
  • web page reads data from URL hash and pre-populates fields:
    • video url
    • referer url
  • web page reads data from cookies and pre-populates fields:
    • host
    • port
    • https
  • clicking the Load Player button performs the following tasks:
    • configures a new video URL that redirects the HLS manifest through HLS-Proxy
    • redirects WebView to another ES5 compliant web page that provides optional endpoint destinations for the proxied HLS manifest
      • ES6 web page for ExoAirPlayer sender
      • ES5 web page for Chromecast sender
        • in WebView on Android 4.x and older:
          • video player functionality does not work
        • in WebView on Android 5.0 and newer:
          • video player functionality works
          • Chromecast sender functionality does not work

---

Important Caveats

• some video URLs may play in WebCast and ExoAirPlayer, but cannot play on Chromecast or other external video players

  • this can occur when a video URL is hosted by a server that uses the Referer HTTP request header to restrict access, which is a common strategy
    • WebCast and ExoAirPlayer have the functionality to configure the value of this header for each unique video URL
    • Chromecast receiver apps cannot change the value of this header because they are restrained by standard browser security policies
      • the specs for XHR and fetch forbid changing certain HTTP request headers, including Referer
      • the WebCast Chromecast receiver app
        • attempts to change the value of this header
        • reveals in the remote debugger console that this attempt raises the warning:
          • Refused to set unsafe header "referer"
    • other external video players would need to:
      • read the referUrl extra in the starting Intent
      • configure its HTTP client library to change the value of this header
    • HLS-Proxy provides a convenient general-purpose workaround
      • setup for integration with WebCast:
        1. install HLS-Proxy
           • npm install --global "@warren-bank/hls-proxy"
        2. install WebMonkey
           • app to open HLS-Proxy configuration
        3. in WebMonkey
           • install WebCast-Reloaded userscript
             • adds enhanced functionality to the ES5 compliant web page that provides HLS-Proxy configuration
               • enables a button labeled: Start App
               • when clicked:
                 • start Activity chooser w/ an implicit Intent
                   • action
                     • android.intent.action.VIEW
                   • data
                     • proxied HLS manifest URL
                   • type
                     • "application/x-mpegurl"
        4. in WebCast
           • Settings > Video Player > HLS-Proxy configuration
      • usage:
        1. run HLS-Proxy
           • hlsd --port 8080 --req-insecure --useragent "Chrome/90"
        2. in WebCast
           • navigate internal WebView to a page having the desired HLS video stream
           • open the Videos drawer (on right)
           • click on the URL for the desired HLS video stream
           • click: Watch
             • an Activity chooser will start
             • WebMonkey will be included in the list of apps that contain a matching Activity
             • click: WebMonkey
        3. in WebMonkey
           • configure the location of your running instance of HLS-Proxy:
             • host
             • port
             • https
           • click: Start App
             • an Activity chooser will start
             • WebCast will be included in the list of apps that contain a matching Activity
             • click: WebCast Video Player
        4. in WebCast
           • click the Chromecast sender icon to cast the proxied HLS video stream to a Chromecast device

• the Android System WebView component is wholly responsible for the web browser experience

  • this component has a complicated history
  • without going into detail:
    • on versions of Android < 5.0
      • the WebView component is baked into the firmware
        • cannot be updated
        • does a poor job loading modern webpages, as the javascript language (ES6+) and html spec (HTML5) have changed significantly
    • on versions of Android >= 5.0
      • the WebView component is a standalone application
        • can be updated
    • on versions of Android >= 7.0
      • the WebView component is superseded by a component of the Google Chrome web browser, when it is installed and enabled
        • can be updated

---

Organization of Git Repo

• stale branches
  • 01-foundation
    • collection of small apps to develop and test experimental features
  • 02-webcast-httpclient
    • used an external HTTP client library: Apache HttpClient
    • used the Content-Type response header to detect video files
  • 03-webcast-okhttp
    • used an external HTTP client library: okhttp
    • used the Content-Type response header to detect video files
• active branches
  • 04-webcast-filename
    • uses WebView to download all HTTP requests
    • uses regular expressions to detect file extensions associated with video formats in URL requests
  • 05-chromecast-receiver-app
    • WebCast Chromecast receiver app
  • gh-pages
    • WebCast Chromecast receiver app
      • the HEAD of this branch should always reference/alias the HEAD of the 05-chromecast-receiver-app branch
    • hosted by GitHub Pages

Highlights of Source Code

• identification of video URLs in outbound HTTP requests
  • BrowserWebViewClient
    • regex to detect video files
• same methodology as implemented by a web browser extension
  • "WebCast-Reloaded" Chromium extension
    • regex to detect video files

---

Legal

• copyright: Warren Bank
• license: GPL-2.0