googlechromelabs/pwa-wp

WordPress feature plugin to bring Progressive Web Apps (PWA) to Core

PWA

WordPress feature plugin to bring Progressive Web App (PWA) capabilities to Core

Contributors: google, xwp, rtcamp, westonruter, albertomedina
Tags: pwa, progressive web apps, service workers, web app manifest
Requires at least: 6.4
Tested up to: 6.5
Stable tag: 0.8.1
License: GPLv2 or later
Requires PHP: 7.0

Description

Progressive Web Apps are user experiences that have the reach of the web, and are:

• Reliable - Load instantly and never show the downasaur, even in uncertain network conditions.
• Fast - Respond quickly to user interactions with silky smooth animations and no janky scrolling.
• Engaging - Feel like a natural app on the device, with an immersive user experience.

This new level of quality allows Progressive Web Apps to earn a place on the user's home screen.

Continue reading more about Progressive Web Apps (PWA) from Google.

In general a PWA depends on the following technologies to be available:

• Service Workers
• Web App Manifest
• HTTPS

This plugin serves as a place to implement support for these in WordPress with the intention of being proposed for core merge, piece by piece. Case in point, the HTTPS functionality from this plugin has largely been made part of Core: Improved HTTPS detection and migration in WordPress 5.7.

This feature plugin is not intended to obsolete the other plugins and themes which turn WordPress sites into PWAs. Rather, this plugin is intended to provide the PWA building blocks and coordination mechanism for these themes and plugins to not reinvent the wheel and also to not conflict with each other. For example, a theme that implements the app shell model should be able to extend the core service worker while a plugin that provides push notifications should be able to do the same. Themes and plugins no longer should have to each create a service worker on their own, something which is inherently problematic because only one service worker can be active at a time: only one service worker can win. If you are developing a plugin or theme that includes a service worker, consider relying on this PWA plugin, or at least only use the built-in implementation as a fallback for when the PWA plugin is not available.

In versions prior to 0.6, no caching strategies were added by default. The only service worker behavior was to serve an offline template when the client's connection is down or the site is down, and also to serve an error page when the server returns with 500 Internal Server Error. As of 0.6, there is a new “Offline browsing” toggle on the Reading Settings screen in the admin. It is disabled by default, but when enabled a network-first caching strategy is registered for navigations so that the offline page won't be shown when accessing previously-accessed pages. The network-first strategy is also used for assets from themes, plugins, and WordPress core. In addition, uploaded images get served with a stale-while-revalidate strategy. For all the details on these changes, see the pull request.

Documentation for the plugin can be found on the GitHub project Wiki.

Development of this plugin is done on GitHub. Pull requests welcome. Please see issues reported there before going to the plugin forum.

Frequently Asked Questions

Please see the frequently asked questions on the GitHub project wiki. Don't see an answer to your question? Please search the support forum to see if someone has asked your question. Otherwise, please open a new support topic.

Changelog

For the plugin’s changelog, please see the Releases page on GitHub.

pwa-wp's Issues

Evaluate and document existing service worker implementations in WordPress

As a developer in the WordPress ecosystem and as a contributor to this project, I should have an understanding of the complex usage of service workers used today before we propose a solution.

• AC1: Record what plugins, themes and headless projects we can find that are using service workers, sharing any quirks and oddities and lessons learned along the way.
• AC2: Commonalities to pain points should recorded and brought to product definition.

Document of plugins to review that use service workers: https://docs.google.com/document/d/1bTgRGyQiGUHm-0yG8LWkKAJ_heLxeLS7BKcfI2SON84/edit#

Allow service worker scripts to be just registered without inclusion

At the moment when you register a service worker script it will get included in a service worker: either the frontend, wp-admin, or both. However, in core you can register a script without enqueueing it. It seems we should perhaps have a similar capability for service worker scripts so that a script could be registered but only included if it is marked as a dependency or explicitly enqueued. For example, in #5 we may bundle Workbox with the plugin. But it may or may not be used always, but it should always be available. So a plugin/theme should be able to mark workbox as a dependency and have it included in the service worker automatically only when it is needed.

As part of this, we may want to revisit the scope arg and reuse the group that is already being used in scripts. So a service worker script could be registered and then "enqueued" to either/both service workers (front and admin).

Convert block validation script to a module, and bundle it with block toggle module

Idea: Add support for custom theme template file to be used for offline page

Although you can use a regular page as the offline page and choose a page template it might be desirable to let theme authors create a template file specifically for offline use as it could be quite different, eg. with no menus etc.

There are a couple of suitable options for this:

• 503.php - service unavailable, also send the same status code? Not sure this is technically accurate to use, as it could be either the site or user who is offline
• offline.php - not status code specific and self explanatory

Standardize a WP approach to app manifest

As a developer of a PWA, I should use an app manifest to describe what my application does - knowing that there can only be one. I should be respectful of other app approaches, and I should have some way to play nicely if there are several app manifests.

AC1: Based on discovery of WP approaches (core, plugins, themes) from #1, document how app manifests are used.
AC2: Provide a basic hierarchy (or some core definition) on a perhaps per-theme basis to establish direction.

Scenario:
A user installs Super Awesome Plugin and Cool PWA theme. They both use PWA features; whose app manifest rules them both? We shouldn't go overkill (i.e. theme template overrides, which works well but is quite complex) but we should find a straightforward approach here.

Plugin updater notification engine for PWAs

As a developer, I should have an approach to leverage PWA notifications in my app.

• AC1: Research and define approach for support of PWA notifications based on the findings from #1.
• AC2: Prototype an example of what notifications could look like based on the shared approach from #7.

Write workbox-sw.js directly into service worker instead of using importScript()

We can avoid an additional HTTP request for each installation of the service worker by inlining the workbox-sw.js file into the service worker response instead of requesting it via importScript():

https://github.com/xwp/pwa-wp/blob/c3b06fe6f7cfc78538c2b256448a41e4a050602d/wp-includes/components/class-wp-service-worker-configuration-component.php#L57-L60

So this could instead do:

$script .= file_get_contents( PWA_PLUGIN_DIR . $workbox_dir . 'workbox-sw.js' );

h/t @kristoferbaxter

Work with a (or several) popular SW-enabled PWA plugin or theme, fork it, add our own code

As a developer, I should have a common approach to service workers to develop a collaborative approach for PWA features.

AC1: Identify a list of WordPress plugins (from #2) of a good fit for this project.
AC2: Use our prototype approach (#7) in place of their service worker code. Determine what level of effort it took, perhaps providing a pull request back to the master plugin branch if the plugin is managed in a open repo.

Investigate solutions for HTTPS detection and upgrading

As XWP proposing solutions for ease of PWA adoption, we need to understand what challenges exist in HTTPS compatibility, whether the solutions are good enough for ease of use and decide if and where to extend the current solutions and approaches to make it easier to migrate a WordPress website from HTTP to HTTPS/HSTS.

• AC1: Investigate existing solutions for HTTPS detection, including insecure content and requests
• AC2: Investigate existing UX for HTTPS upgrades
• AC3: Look at the upgrade-insecure-requests directive as mentioned by @westonruter
• AC4: Look at which solutions are widely used and supported by WordPress, understand their utility and see if there any action steps by our project.

Currently blocks #17.

Developer Documention for 0.2

As a developer using the PWA plugin for WordPress, I should have documentation on how to use the plugin, including declarations for Workbox, and why I should use it.

• AC1: Include drafts in the board's Wiki for how to:
  • Update the existing documentation according to changes
  • Using custom offline / error templates
  • Actions and filters

...Including examples where applicable.

Facilitate notifying user of updated service worker

An updated service worker defers activating until all existing client windows have gone away (e.g. via reloading browser). There should perhaps be a way to notify the user that updates are available without requiring them to have DevTools open. Such an update prompt is common on web apps, and having a standardized notice in WordPress would probably be a good idea.

In the admin the notice could be shown as an admin notice. On the frontend maybe there is no standard way to show the notice, other than to display something in the admin bar. But this isn't shown to logged-out users.

Read more about the lifecycle at https://developers.google.com/web/fundamentals/primers/service-workers/lifecycle#waiting

Add ability to select a default offline page to be served when the user or site is offline

##User story: As a WordPress administrator, I want to be able to select a page that displays when visitors are offline.

• AC1 - As @mehigh suggested, create a UI to select the page that is displayed when offline.
• [ ] AC2 - This page should be served when offline. This AC has moved to ticket #45.

This could be similar to the Settings > Privacy Settings page:

The offline page UI could look something like:

This is based on @westonruter's suggestion in #9 (comment).

Facilitate offline editing in Gutenberg via caching and background sync for REST API responses

See WordPress/gutenberg#6322 (comment)

This ticket goes along with the re-introduction of Turbo mode (#34). Beyond caching of assets and admin screens, we also need to allow REST API calls to be made offline and to use background sync to push changes to REST API resources once re-connected.

Implement precaching

As a site admin, I should have a way to precache important assets to give my users a faster perceived experience of using my WordPress site.

AC1: Propose a user interface to select which assets should be pre-cached.
AC2: As site admins might need to declare precached asset support in different ways, also propose a theme function and/or wp-cli script to select which assets should be precached.

Note: additional ACs likely to be added.

---

As noted by @jeffposnick in #5 (comment):

Something I've not seen mentioned yet has to do with precaching—here's an explainer about how Workbox handles precaching. Precaching gives you the advantage of ensuring that, whenever your service worker is installed, a core set of URLs will be cached and kept up to date, even if they correspond to assets that haven't been used yet. (In other words, you get to cache important things in advance.)

This differs from using routing + caching strategies to implement runtime caching, in which assets and HTML is only cached after a user visits a page that uses them for the first time.

Precaching offers a number of advantages, both in terms of "priming" the cache, as well as efficient updates (a network request is only required when something in the precache manifest actually changes). The downside of precaching is that it requires some integration with a build process, as it Workbox needs to create hash "fingerprints" of local assets on the filesystem in order to create its precache manifest. That might not be viable in a generic Wordpress solution, but I wanted to throw it out there if there are hooks into a local build process that could be taken advantage of.

Precaching is indeed something we haven't talked about yet, but I think that WordPress has some good tooling we can use here. For example, external scripts and stylesheets get registered with dependency system where each URL get a stable handle that does not change. Each dependency gets a version as well which is used to cache-bust the URL, but in the case of precaching we could strip the version (ver) parameter when generating the response since the service worker could handle the cache busting itself.

Rely on `WP_Scripts` for all plugin functionality and introduce components

@westonruter and I chatted for a bit about the code structure of the plugin and how to reduce complexity while making the functionality more straightforward and flexible at the same time.

Here's my suggestion as a takeaway from this discussion:

• Let's split WP_Service_Workers into two different classes:
  • WP_Service_Worker_Scripts will contain the script registry part and will extend WP_Scripts.
  • WP_Service_Workers will contain the serve() method and handle everything related to actually serving the service worker for frontend or admin.
• The wp_service_worker action should be removed. In terms of timing, it is problematic to have it run independently of the wp_front_service_worker / wp_admin_service_worker action respectively. A plugin wanting to register scripts for both scopes could simply hook the same callback into both actions.
• The instance of WP_Service_Worker_Scripts should be passed to the wp_front_service_worker / wp_admin_service_worker actions.
• An interface WP_Service_Worker_Component should be introduced. Components take care of core service worker functionality, for example outputting configuration via the base script (currently part of WP_Service_Workers::get_base_script(), would then move into a WP_Service_Worker_Configuration_Component) or outputting the script for handling precaching URLs (would move into a WP_Service_Worker_Precaching_Routes_Component). The interface should have a method serve( WP_Service_Worker_Scripts $scripts ) and a method get_priority(). The WP_Service_Workers class will be aware of all available components and hook each serve() method into both the frontend and admin service worker actions, at the priority returned by get_priority().
• An interface WP_Service_Worker_Registry should be introduced. This is a general interface defining a unified behavior for any registerable content. It should be implemented by WP_Service_Worker_Scripts, as well as all registries that belong to a specific component, for example the precache component allows other code to register precache URLs (see more below). The interface should have a method register( $handle, $args = array() ) and a method get_all().
• An interface WP_Service_Worker_Registry_Aware should be introduced, with a single get_registry() method. This should be implemented by components that allow other code to integrate with themselves. For example, WP_Service_Worker_Precaching_Routes_Component would implement this interface in combination with its accompanying WP_Service_Worker_Precaching_Routes (which implements WP_Service_Worker_Registry, as already hinted above).
• The WP_Service_Worker_Scripts class would be instantiated from the WP_Service_Workers class, and it would (in addition to its own methods for adding scripts) provide access to all component registries, i.e. all registries returned by the get_registry() method of a component (which can easily be detected via the WP_Service_Worker_Registry_Aware interface that they would implement. This would allow plugins and other integrations to easily register everything from a centralized point, as the WP_Service_Worker_Scripts instance is passed to the service worker actions. A plugin could then for example call $scripts->precaching_routes()->register( ... ).
• The integrations (see WP_Service_Worker_Integration) as they exist now would almost remain the same. The only thing they would receive in addition would be a get_priority() method, in order to determine the priority they are hooked into the actions. The code would also be adjusted to account for the removal of the wp_service_worker action (mentioned earlier).

--
These changes will introduce the following clear responsibilities:

• WP_Service_Workers as main entry point to the service worker functionality.
• WP_Service_Worker_Scripts as the main registry that plugin developers would interact with, via the wp_front_service_worker / wp_admin_service_worker actions.
• Components handle crucial core functionality for service workers. These are hard-coded and it is not possible for plugins to register additional components. Those are purely for solid code organization and to keep the WP_Service_Workers and WP_Service_Worker_Scripts clean and preserve their single responsibility. Anything that happens in a component could theoretically be added as a script directly, components just make their respective functionality much easier to use than if one had to manually register a custom script for it.
• Component registries allow to modify components that support registration of their own data.
• Integrations are an alternative object-oriented way of integrating with the plugin's service worker(s). It is simply a layer around hooking a callback into the wp_front_service_worker / wp_admin_service_worker actions directly. All default integrations (such as for the Custom Logo or Custom Header core features) should use the integration approach, while plugin developers can use whatever of the two approaches they prefer.

Let me know if you have any suggestions or concerns about this. I'll move forward with a PR for these changes tomorrow. With that in place, it might also be easier to see how all of the above comes together.

Adapt Surma's CDS'17 Progressive WP Theme leveraging Workbox and PWA Plugin

As a WordPress developer, I should have the theme developed by Surma for his CDS 2017 talk work well with the PWA Feature Plugin for WordPress so I can activate the new features from the PWA feature plugin on my website.

AC 1: The updated Theme will use proper the WordPress templating approach and extract functionality from theme provided by the PWA Feature Plugin: https://wordpress.org/plugins/pwa/
AC 2: The theme will work properly after adapting it to work with the PWA feature plugin
AC 3: Hook the existing progressive features into the theme codebase

Background:
His talk:
https://www.youtube.com/watch?v=Di7RvMlk9io
Repo for the theme:
https://github.com/GoogleChromeLabs/ProgressiveWordPress
Demo site (as of today):
https://pwp.dassur.ma/
Example of Google offline search
https://www.theverge.com/2017/1/18/14308830/android-google-offline-searches-mobile

Serve static offline/500 template as opposed to using specially-designated page

We had quite a bit of back-and-forth in #9 about whether to serve a static offline.php template or a serve a designated offline page, but went ahead with going down the route of the latter: letting the user to select the offline page. We merged the ability to select the offline page (#23 and #45) and to be able to serve this page when offline (#48).

Nevertheless, through the implementation of this ability we've discovered some problems:

• The page.php template could be including assets that would not be precached, and would thus be broken.
• A user could supply content (e.g. a video) that would not be precached, and thus be broken.
• Reliably precaching the header image which would be served on the offline page can be a challenge, especially if the random header option is selected.
• The designated offline page could be an existing page, and in that case should it be removed from nav menus (#46)? Should the selected offline page be excluded from search results and being added to nav menus in the first place? What if they want to use the contact page as the offline page?

These complications have circled things back to what @nico-martin recommended in #9 (comment) as part of the offline page:

I think the offline.php template would be the better solution than asking the user to pick an offline-page.

Basically this debate comes down to two separate precedences in core for serving a template: do we follow the precedence of the privacy page, or do we follow the precedence of the 404 template? It should be noted that while case could be made for allowing a custom page to be designated to be served on the 404 template, core does not provide that ability (though it could be provided by a plugin). So this feature plugin providing a way to select a custom offline page now seems strange in this way.

If going the static template route, we have another challenge if we want to preserve the theme's design as much as possible. If we look at an example 404.php template, we can see it is not as simple as just:

<?php get_header(); ?>
<p>Not found!</p>
<?php get_footer(); ?>

No, there are multiple levels of wrapper elements that are required by the theme in order to get the hard-coded “Not Found” content to be laid out and styled properly. So we wouldn't be able to introduce an offline.php that could be faithful to the theme's required markup structure. But even if we did, there would be a chance that content in the header or footer that would fail to precache anyway.

One of the arguments I had about going with a user-selected page is that it would allow us to fully use the template hierarchy, where a theme could still fully be served with a single index.php. In such a theme, the 404 case can be built in by having a standard if ( ! have_posts() ) condition. If a user-selected page were used, then it would be included in The Loop™ like any other content, and there wouldn't be the need for an additional hard-coded condition like elseif ( is_offline() ).

Nevertheless, a theme that only contains an index.php template is not exclusively using this template on a site. Enter the theme-compat directory, and in particular embed.php. When calling locate_template() it will first check if a given template exists in the child theme and if not it checks the parent theme. If it doesn't exist in either of those locations, it then checks the theme-compat directory. And this is where the post embed template is served from in most WordPress themes, since embed.php is normally not implemented. And this is a very similar situation to where themes will be once an offline page is introduced.

So the best solution for proposed core merge is to design a good-looking generic offline.php (and 500.php) be located in the theme-compat directory to be served when the user/site is offline. If a theme defines a offline.php then they will have to do additional work to make sure that any assets used on this page are included among those that are precached. Additionally, a plugin should be able to override which URL is used for the offline page so that they can provide the ability to select a custom page.

So in the end a theme would not have to define an offline.php to see the site-specific offline page, but they should define one if they want to have the offline page styled according to their theme, instead of a generic one.

Add reliable method for detecting HTTPS support

The is_ssl() function is woefully inadequate to determine whether WordPress is being served via HTTPS. It is not reliable because HTTPS termination may be happening at a proxy and not at the server running WordPress. So it is not reliable to look at the server environment to determine if HTTPS is being served or is able to be served.

I believe the best way to detect HTTPS support is to actually perform a loopback request onto the home URL of the site with HTTPS and check that it responds properly with sslverify enabled. For added resilience, the request should include a token which the response then hashes with a secret which can then be compared to confirm that in fact it is the same WordPress install itself that is responding, and not some other WordPress site that is configured on HTTPS.

Once the HTTPS check is performed, the result should be stored in an option, and the option can then be checked instead of is_ssl().

This would allow for a site to make use of HTTPS exclusively without manually adding a FORCE_SSL_ADMIN constant.

CDS 18: Extend `twentyseventeen` to a showcase of PWA + AMP

As a site owner building a WordPress site, I should have a twentyseventeen child theme that not only looks polished and modern, but also includes PWA and AMP supports to showcase the modern web technologies of the PWA and AMP plugins.

Replaces #88. To be based on (and work to be conducted in) @westonruter's child theme, westonson.

• AC1: This child theme needs to be Native AMP by default, and surface no errors in the AMP compatibility tool on activation. (Work on this AC will be done in twentyseventeen-westonson repo Issue 3)
• AC2: This child theme should use streaming for loading "heavy" content and work without errors or performance issues. (Work on this AC will be done in twentyseventeen-westonson repo Issue 6)
• AC3: Offline commenting, leveraging <amp-live-list> from the AMP Plugin (and the efforts in #87), should not only be enabled by default but also report no errors in the AMP compatibility tool. (Work on this AC will be done in both #87 Offline Commenting and in twentyseventeen-westonson repo Issue 3)
• AC4: Optimized offline page - come up with animations that work well for a 500 page. Inspiration should be taken from something like Github. Other inspirations could come from 404 pages as well. (A custom 404 page wouldn't be a bad idea, too, but is not a requirement.) (Work on this AC will be done in twentyseventeen-westonson repo Issue 4)

Demo site

• AC5: This child theme should extend the existing designs of twentyseventeen and enhance the user experience though a clean, elegant and playful design. While some efforts would be in the westonson theme itself, some of this would be shown on a demo site with these customizations.
• AC6: This demo site should, of course, showcase AC1-AC4 above.
• AC7: List out what features showcase PWA, AMP and other bells and whistles that make this work interesting. (Related: #68.) This could also be in the form of a blog post describing how we built what we built.

AC2, AC4, AC5 and AC6 will likely be broken into new tickets for design, FE and BE efforts.

Consider is_ssl() and the detection if HTTPS is currently being served

In #6 we added a means for detecting whether HTTPS is available for a given site. However, this does not actually improve the ability to detect whether HTTPS is being served for the current request. WordPress does have an existing function for this: is_ssl(). This is used in redirect_canonical() and should result in most HTTP requests being redirected to HTTPS, assuming your home URL uses HTTPS.

However, as noted in a comment:

With load balancers

It won’t work for websites behind some load balancers, especially Network Solutions hosted websites. To bodgy up a fix, save this gist into the plugins folder and enable it. For details, read “WordPress is_ssl() doesn’t work behind some load balancers.”

Websites behind load balancers or reverse proxies that support HTTP_X_FORWARDED_PROTO can be fixed by adding the following code to the wp-config.php file, above the require_once call:

if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https')
   $_SERVER['HTTPS'] = 'on';

The X-Forwarded-Proto header is not an actual standard, however, and there are alternative headers. On Pantheon, for example, they use an User-Agent-HTTPS header. They advise using this to redirect from HTTP to HTTPS.

The key point to all this is that we need to prevent requiring a user to hack their wp-config.php to add some special handling of HTTPS. I see three options here:

1. 🚫 Update core's is_ssl() function to detect more headers such as X-Forwarded-Proto. However, this has been proposed before and rejected for good reasons.
2. Ask hosts to just set $_SERVER['HTTPS'] = '1' when HTTPS is being served, as recommended in #31288.

So I think the second option is the only thing we can do here.

Facilitate registering scope-specific service worker scripts

In working on the offline page, there was a point where both the frontend and admin service workers were precaching the assets for the offline page. I fixed this in 5b8a0b6:

https://github.com/xwp/pwa-wp/blob/9a10dc7e0a613964427c97a426c77f5842f2c95e/wp-includes/class-wp-service-workers.php#L119-L121

But the use of a conditional like this is not ideal. We'd also need something similar for precaching admin assets in turbo mode (#34). In addition to there being a wp_default_service_workers action, we should also have a wp_front_service_workers and wp_admin_service_workers actions. This is related to #33.

We should also only run these actions when we're actually going to be serving the service worker.

Add ability to serve the default offline page when the user or site is offline

As an administrator site owner, I expect the selected default offline page to be served when either the user or site is offline.

• AC1 Serve the default offline page when the requested page is not available offline.
• AC2 Add is_offline to the query to denote. See Weston's comment.

Issue #23 added the ability to select the default offline page. Issue 23 includes conversations on how to serve the default offline page:

See Weston's comment:

Something important to note here I just realized: this offline page could be served not only when someone's internet is offline but also when the server itself experiences an internal server error (a 500 response). We should try to design the offline page so that it could be used for both use cases. There would need to be some client-side logic involved here, however, since the client would know if the page fails to be fetched because of a network error or a server error.

Important related issues in Issue #9 including this comment:

We could then serve it only if requesting with the query var as you suggested. If no offline page exists, then the response could be treated as is_404() to serve the not found message, which I think would be better then no offline page at all. (Alternatively, I wonder also if the template hierarchy should be extended so that is_offline() would be a special case under is_404().)

There are thousands of themes out there and it would take a long time for them to add an offline.php template. I don't think that should be a requirement before serving the offline page.

Requiring a special offline template would also not align with the template hierarchy expectation either. A minimal theme is only required to include an index.php template which gets used then to serve responses to all queries, including is_404(). A theme should also be able to use this generic index template to serve an offline response as well (is_offline()?). That would seen the most logical from a theme developer's perspective.

And Weston commented on PR 28:

The serving of the offline page should probably incorporate Workbox (#5) and also it will probably involve precaching (#25) since we'll need to make sure the offline page and its required assets get added to the cache.

Allow registering Service Worker without `https` when using localhost

Currently https is always enforced when registering the Service Worker (see here), however, http is allowed on localhost according to this page:

During development you'll be able to use service worker through localhost, but to deploy it on a site you'll need to have HTTPS setup on your server.

When developing on localhost without using https the Service Worker is not registered and an error shown in console instead, for example:

Perhaps we should consider registering the Service Worker without enforcing the https in case of localhost to make it easier for developers using http locally on localhost.

Rewrite HTTP URLs to use HTTPS

Any self-referential URL that is served from the site—including URLs for links, images, videos, scripts, styles, etc.— should get re-written to use HTTP instead of HTTPS when the user has opted-in. See core ticket wp#28521:

• Force https connections (pretty much covered by #27954)
• Force local URLs within content to https
• Force local enqueued scripts and styles to https
• Force non-local enqueued scripts and styles to https
• Set the secure flag on all cookies

What we won't do:

• Force non-local URLs within content to https
• Force the https version of oEmbeds just yet - see #28507
• Send an HSTS header - see #28520

Additionally, pages should be served with the upgrade-insecure-requests content security policy so that if there are any URLs that leak through which didn't get rewritten that supporting browsers (~80%) will then do it instead.

Ensure Turbo mode works seamlessly with force_uncompressed_tinymce in WP 5.0

See phpDoc for force_uncompressed_tinymce in https://core.trac.wordpress.org/changeset/43738:

* Force uncompressed TinyMCE when a custom theme has been defined.
*
* The compressed TinyMCE file cannot deal with custom themes, so this makes
* sure that we use the uncompressed TinyMCE file if a theme is defined.
* Even if we are on a production environment.

With Turbo mode (#42) in the PWA plugin (where admin assets are precached) then this should be irrelevant. In other words, custom themes should not have a negative performance impact. This should be confirmed.

AMP Hooks for ServiceWorker

@westonruter I've seen you are using wp_footer to install the ServiceWorker on AMP (https://github.com/xwp/pwa-wp/blob/master/amp/class-amp-service-worker.php#L172)
Is there a reason you are not using amp_post_template_footer?

it seems like neither https://wordpress.org/plugins/accelerated-mobile-pages/ nor https://wordpress.org/plugins/amp/ support wp_footer.
Here's how I implemented SW for AMP: https://github.com/SayHelloGmbH/progressive-wordpress/blob/master/Classes/class-amp.php

Reconsider use of JavaScript to inject 500 error details

A 500.php template currently needs to include the following to display any 500 error that was returned from the server:

<details id="error-details" hidden>
	<summary><?php esc_html_e( 'More details', 'twentyseventeen-westonson' ); ?></summary>
	<iframe srcdoc=""></iframe>
	<script>
	function renderErrorDetails( data ) {
		if ( data.bodyText.trim().length ) {
			const details = document.getElementById( 'error-details' );
			details.querySelector( 'iframe' ).srcdoc = data.bodyText;
			details.hidden = false;
		}
	}
	</script>
	<?php wp_print_service_worker_error_details_script( 'renderErrorDetails' ); ?>
</details>

However, this currently depends on BroadcastChannel to send the message from the service worker to the client after the page has loaded:

https://github.com/xwp/pwa-wp/blob/57b295c646bfae63f78c3d735fbc37762709fca0/wp-includes/js/service-worker-error-response-handling.js#L11-L31

It would seem more straightforward to instead inject the error messages directly into the details element when the 500 error is being constructed. This could be done done with streams or more simply it could be done just by reading the body of the 500 error response and injecting it straight away.

Do client-side HTTPS request to confirm valid certificate

At the moment we rely on the WP Cron job to do a loopback request to HTTPS URL for the site to make sure that HTTPS is available. However, it could be that the server doesn't have certificates configured properly to successfully verify, or it may verify when they shouldn't be. Additionally, on the general settings screen it could be that the cronjob is delayed and so we can't get up-to-date information to display with the HTTPS checkbox.

So I think for the general settings screen we should do a client-side request with JavaScript to an HTTPS endpoint to make sure that the request succeeds. This could be simply an admin-ajax handler that sends the CORS response headers. Or we could do a fetch to the HTTPS URL for the web manifest, since the REST API sends CORS by default via rest_send_cors_headers(). But since that can be disabled, perhaps best to go with admin-ajax.

So whenever accessing the general settings screen where the HTTPS checkbox is displayed, the Ajax request can be made to check for HTTPS and the checkbox (with notice) can be updated depending on whether the request succeeds.

Implement Offline Commenting

As as WordPress developer, I would like to be able to allow site visitors and admin users add comments to posts even when offline

• AC 1: Build comment offline sync functionality into PWA plugin. When viewing a page or post with commenting enabled, a user will be able to make a comment and have that comment saved and posted once the user is back online

• AC 2: The user interface for offline commenting will be easy to use and include appropriate messaging. By default the plugin should show an offline page where submitting a comment when offline, with a message like “you are offline, but we will post your comment will be posted when your device comes back online.” This should integrate with the offline template to show custom messages similar to how the 500 page can show custom messages. (See related issue in the twentyseventeen-weston repo, Issue 4 Optimize the design of the Offline/500 page to be more engaging for the user)

• AC 3: A theme must be able to override the offline template and messaging for commenting failures.

• AC 4: Site on Using our AMP and PWA plugins, enable this offline commenting support via a theme declaration.

• AC 5: Intercept the amp-form action-xhr request (to submit the form) and when failed return a custom success message to be shown below the form

Offline Routing and messaging

• AC 6: If AMP is enabled, The amp-live-list should continue to update in real time, showing the new comment when background sync happens or if the comment was submitted successfully right away. The error message should probably appear in the context of the posted comment.
• AC 7: If AMP is not enabled, route the user to the site's offline page and allow a theme developer to serve the offline message from AC2/AC3. Make sure the the ability for service worker to populate content of the offline template is added.

For reference, here was the original PR from the AMP plugin:
ampproject/amp-wp#1028

Unregister unused ServiceWorker scopes

By design, once you register a ServiceWorker it stays registered untill you unregister it.
Now since we have the ServiceWorker API we update an active ServiceWorker based on the Scope on each pageload.
Code that is not longer in use will be removed once the ServiceWorker updates (so on every pageload). Thats good.

But what if a plugin uses a special scope? Once the plugin is disabled, the registration/update method won't be included anymore and the ServiceWorker will still be active in browsers that visited the page when the plugin was active.

Should we add an unregister method for all ServiceWorker/Scopes that have ever been registered?

Add WP_HTTPS constant to force whether HTTPS is enabled or disabled

As mentioned in WordPress Slack:

Similarly to how there is a WP_HOME and WP_SITEURL constants which force the home/siteurl values regardless of what there is in the DB. We could also have a WP_HTTPS constant that does the same for the HTTPS checkbox that appears on the general settings screen. If that constant is defined then the checkbox would not be shown. So if they want to turn off HTTPS after accidentally turning it on, they could add define( 'WP_HTTPS', false ); to their wp-config.php.

This would allow sites to prevent users from turning off HTTPS to great peril, and it would also allow users to turn off HTTPS if they locked themselves out of the admin due to a bad certificate (and they can't figure out how to force their browser to accept it).

Handle user authentication for offline/500 pages

At the moment the offline and 500 error pages can be precached with the user's authenticated state. This means that they can include the admin bar. The admin bar should not be shown in the offline template. But should an authenticated response be served to begin with? It can be inconsistent at present whether the authenticated offline page or the unauthenticated offline page ends up getting precached. Perhaps we should vary the revision or the offline (and 500) pages to include the current logged-in user. This would ensure that the page gets re-downloaded whenever the user logs in or out, or switches to another user entirely.

• Prevent admin bar from ever showing on offline pages.
• Include current user ID in the revision for the offline and 500 page URLs.

Create a UI to force HTTPS if it's supported

Please see the "Revised ACs" comment for the ACs, instead of this comment.

Hi @westonruter and @postphotos,
What do you think about these ACs? Feel free to edit this.

User Story: As a WordPress administrator, I want a UI to ensure that my site uses HTTPS if it's supported.

• AC1. Use the HTTPS detection logic from Issue #6
• AC2. On the General Settings page, add a <tr> for HTTPS support (use JavaScript to move it after the home/site URLs)
• AC3. If HTTPS is supported, but the WordPress Address (URL) and Site Address (URL) don't use HTTPS, display this notice:

• AC4. On clicking "Click here," the WordPress Address (URL) and the Site Address (URL) change to HTTPS, and there's a redirect from HTTP to HTTPS
• AC5. If HTTPS is not supported, display:

• AC6. Clicking "Recheck" triggers the loopback request from #6. Maybe there should be a notice at the top, like "HTTPS is still not supported" or "HTTP is supported."

• AC7. If HTTPS is supported, and the WordPress Address (URL) and Site Address (URL) use HTTPS, display:

• AC8. If HTTPS is enabled, show the user a toggle for HSTS (see issue #18).

• AC9. Propose a UI and match the designs provided to meet WP core standards and look-and-feel, all to load on the general settings page.

Prevent serving 500 error template when WordPress is serving response via wp_die()

When wp_die() is called by default it sends a 500 status. This is done (probably incorrectly, as 400 is more appropriate) when trying to re-trash an already-trashed post. Clearly this is not the right use of the 500.php template:

We need to skip serving the 500.php template in this case.

publish on packagist.org

It is easier to install and test, if it's published on packagist.org

Investigate and document Workbox

As a developer in the WordPress ecosystem, I should have a better way to share assets in complex usage of progressive web apps.

• AC1: Record level of effort to set up a working Workbox project, sharing any quirks and oddities and lessons learned along the way.

https://developers.google.com/web/tools/workbox/

Use alternative to BroadcastChannel for sending server error to client 500 page

When a 500 error occurs, a 500 error template is served by the service worker. If there was an error message displayed on the response, the HTML is sent to the served 500 error page via BroadcastChannel:

https://github.com/xwp/pwa-wp/blob/2359ae5a890e199a00b12172297dac2a6af6c7f4/wp-includes/js/service-worker-error-response-handling.js#L11-L31

However, this is not supported by browsers other than Chrome and Firefox: https://caniuse.com/#feat=broadcastchannel

We'll need to find an alternative. (Also the approach taken should be validated.)

Aside: We also need to prevent serving the 500 template when WordPress is actually rendering a response with a 500 status code, per #74.

Prototype App Shell in WordPress using AMP content

• To be validated by research from #12 before proceeding: implementing an app shell site in WordPress introduces several of functional challenges.

AC1: Prototype a usage of app shells (built on #7), using a shelf for a menu or another dynamic feature of WordPress.
AC2: Make use of AMP Sanitization to increase stability in the app shell prototype

Many of the above challenges can be solved/removed if the site is sanitized, similar to what the AMP plugin does. Of course, sanitizing a site without replacing what’s cut out would just break the site, so reliance on the AMP sanitization process is pretty ideal for preparing a WordPress site for app shell implementation, since it preserves content integrity while also removing app shell blockers. In fact, one of the intended benefits of AMP is that it makes content more embeddable for just such cases.

This issue could benefit from using key components from Surma’s PWA.

Navigation preload is causing every navigation request to be doubled

When browsing the WP Admin, I found that links for action links with nonces that trash/restore posts could fail with messages saying that the post was already trashed or restored. Similarly, I found that when I had a “flash” message in a transient that was deleted as soon as it was accessed, that this message would never show up.

The reason for these issues is that every page navigation in the admin is currently causing two requests to be made, one with Service-Worker-Navigation-Preload request header, and another without. This is very problematic for the admin when clicking action links like restore/trash because they are not idempotent. That is why trashing was failing because the browser was showing you the response to the second request, when the item was already trashed.

We need to figure out a solution to this.

Set up repo for PWA feature plugin

As XWP, I need the base framework to build the PWA feature plugin.

• AC1: Create repo for PWA feature plugin
• AC2: Create base plugin (based on foo-bar?) for PWA

@westonruter - not sure how much we're happy with what we've done so far as the repo is currently bare, but feel free to add items here as we establish what we're trying to do.

Use Service Workers (as grandchild of Gears) to bring back Turbo mode in admin

Prior to WordPress 3.0 there was a "Turbo mode" in the WordPress admin which was used to cache scripts and stylesheets in the admin to speed up the experience. This Turbo mode depended on Google Gears. When Gears was discontinued the Turbo mode was also discontinued, and core switched to improving performance by concatenating scripts via load-scripts.php and load-styles.php. This has not been without problems, however. If there are too many scripts included then the URL could become too long in some browsers (though these browsers surely don't support service workers). Also this concatenation breaks the ability to reuse the cache for individual scripts. Google Gears was discontinued in favor of AppCache, and then AppCache has been deprecated in favor of Service Workers. The service worker is now capable of picking up the mantle of its grandfather Google Gears to bring back Turbo mode, and in a standardized way that does not require any browser plugins.

For history of Turbo mode, see:

• #6965: Google Gears support (WordPress 2.6)
• #11301: Remove Gears (WordPress 3.0)
• #20981: Remove gears-manifest.php and manifest.php (WordPress 3.5)
• WordPress.com post on Gears

For more, see WordPress Trac tickets mentioning “Gears” or “Turbo”.

Handle when default offline page is in a menu

The Problem : The default offline page should not be in a menu. But it is possible that when selecting an existing page as the default offline page, that existing page might be included as a menu item or parent menu item within one or more existing menus.

For example:

Currently, no error messaging appears.

In this issue, we are exploring how best to handle this edge case. The discussion began here in #23.

We need to determine whether to:

• Prevent it from happening by disallowing an existing page to be selected as the default offline page.
• Alert - notify the admin and direct him/her to each of the menus in Appearance > Menus
• Automatically resolve and Alert for action when needed (e.g. when the page is a parent menu item).

Create travis integration (with autodeployment) to demo site

Related #3. We want to make sure all of the repo management is established to allow development to work in a focused fashion.

Research a method for caching and using data offline

As a user, I should have a way to experience a website in poor internet conditions. There should be a way for me to easily save the content that the site has already loaded in cache and not lose it when I click on the next link. In this offline state, I should have meaningful content available to me.

AC1: Continue to extract the learnings from #1 and document approaches in this ticket.
AC2: Provide a prototype or a design approach in this ticket outlining how this should work.

AC3: As a developer I want to know how this works and what I'd need to do to turn this on. I'd also want to know what not to do so that I don't hurt my user's experience or excessively store content, nor accidentally permanently cache dynamic data.

Document the current state of app shells

As a WordPress developer, I should be able to leverage the PWA app shell feature.

AC1: Document and describe, if used, app shells from the discovery in #2 .
(AC2 moved to its own issue: #22)

Add support for HTTP Strict Transport Security (HSTS)

As a site admin, I should have a simple way to enable HSTS on my site.

• AC1: Assuming HTTPS checks all clearly show I probably can enable the HSTS header, I should be presented with this option in the HTTPS dialogue in #17.
• AC2: If HSTS activation fails during a loopback request, warn the user and prevent them from activating this setting.

---

A best practice for HTTPS is to also use HTTP Strict Transport Security (HSTS):

If a website accepts a connection through HTTP and redirects to HTTPS, visitors may initially communicate with the non-encrypted version of the site before being redirected, if, for example, the visitor types http://www.foo.com/ or even just foo.com. This creates an opportunity for a man-in-the-middle attack. The redirect could be exploited to direct visitors to a malicious site instead of the secure version of the original site.

The HTTP Strict Transport Security header informs the browser that it should never load a site using HTTP and should automatically convert all attempts to access the site using HTTP to HTTPS requests instead.

If we detect HTTPS support (#6) and the user opts-in to use it (#17) then in addition to redirecting from HTTP to HTTPS then we should also serve the HSTS header.

See also core ticket #28520.

Propose a Workbox solution for WordPress

As a contributor to this project, after I've understood and and documented how the Workbox APIs work, I should propose an architectural plan using Workbox that will allow plugins, themes and site owners alike to share PWA component features in a single service worker implementation without conflict.

Related #1. #2.

Display notice if HTTPS is not enabled, use 301 code in redirect

As a follow-up to #17:

• Display a notice if the siteurl and home (WordPress Address and Site Address) are HTTP, and HTTPS is not enabled (like if there's no certificate)

• Change the code in the redirect from 302 to 301:

https://github.com/xwp/pwa-wp/blob/e1431eae92f75a09742c7cdd3dc438011f53fb9d/wp-includes/class-wp-https-ui.php#L358

Give names to front/admin service workers

In DevTools, the service workers are listed without any names:

I recall seeing there be a way to give a name to a service worker when registering. If possible, we should do that.

Prototype an API to register service workers

As a developer, I should have a discrete way to register service workers to allow my plugin (or theme) to leverage a common approach that allows for collaborative integration features.

AC1: Create documentation based on what we think the API should look like, anticipating its evolution through the project.
AC2: Based on the work we're doing in #1, demo a service worker integration that allow for other tools to also register in our common framework.