justinribeiro/pwa-firebase-functions-botrender

Rendering PWA metadata to link bots with a Firebase Function and Chrome headless

Progressive Web App + Firebase Function + Chrome Headless on App Engine = Happy linkbots

You want the Firebase hosting speed for your PWA, but you really need some way to server render that sweet JavaScript so linkbots stop making your page look horrible on Facetweetlinkedspider card timeline gizmo. Let's piece together a solution.

But how?

This consists of a few pieces to provide our example:

1. Polymer Starter Kit with PRPL that changes page metadata with JavaScript. Could be any PWA, but this is what I've used for the sake of speed of this example.
2. A Firebase function that checks UserAgent against a known list of linkbots that might render your content without JavaScript.
3. Rendertron, a dockerized, headless Chrome rendering solution designed to render & serialise web pages on the fly, acts as our go between.

Setup

1. Deploy and use Chrome teams' Rendertron, a dockerized, headless Chrome rendering solution designed to render & serialise web pages on the fly.
2. Create a project on Firebase.
3. Clone this repo.
4. Run bower install in the project directory
5. Run npm install in the functions directory.
6. Have that URL handy from step 1? Great! Set the firebase config variable firebase functions:config:set rendertron.server="https://YOUR_PROJECT_HERE.appspot.com"
7. Cool. Now tell firebase where this domain is: firebase functions:config:set site.domain="YOUR_SUB_DOMAIN_HERE.firebaseapp.com"
8. Build the sample with the polymer-cli: polymer build
9. Deploy with firebase deploy. Have a victory dance.

What do I do now?

You could give one of the views a spin in say the Twitter Card Validator. In the screenshot below, you can see the expected result: Firebase function kicked, the bot was found, it was sent to our render server, and the the DOM returned with proper JavaScript switched metadata, all of which Twitter reads correctly.

Gotchas

1. The PWA shell is dupped as pwashell.js. I suspect there is a way around this, I just haven't read the docs fully enough.
2. The index.html page (your PWA shell), basically will always fire as a direct match because of "Hosting Priorities" (see https://firebase.google.com/docs/hosting/url-redirects-rewrites). Your index.html should not expect to have JavaScript written metadata.
3. You'll need a paid plan for Firebase (Flame or Blaze), otherwise the external network request to App Engine (or where ever you deployed the docker container) will fail.
4. Probably other things that I'm missing the day before a holiday.

Additional Notes / Questions

1. Initially, this was centered around Sam's initial concept samuelli/bot-render, which is what became Rendertron.
2. "Justin, do I have to run rendertron or can I just do this myself?" - You can run chrome headless on your desktop, you can use my chrome-headless docker image and talk to it via puppeteer (which utilizes Chrome DevTools Protocal).

pwa-firebase-functions-botrender's Issues

GoogleBot

Just found DO NOT ADD GOOGLEBOT. If you add Googlebot to this, you will not have a good day. comment in the code.

Could you please explain why? What happens if I add it?

How to SSR PSK-based PWAs for GoogleBot the right way?

Function executes but Twitter validator kicks out an error

Hi, Thank you for sharing - I haven't had much luck getting this to work unfortunately.

deployed Rendertron to app engine https://a-haan.appspot.com/
deployed my PWA to https://a-haan.firebaseapp.com
deployed function

firebase config variable set

All looks good, but when I try to use twitter validator with https://a-haan.firebaseapp.com/article/top_stories/google-pixel-buds or any other address I get an error:

INFO: Page fetched successfully
INFO: 2 metatags were found
ERROR: No card found (Card error)

Firebase Console shows that the functions executes:

Function execution took 5 ms, finished with status code: 200

---

Any help will be much appreciated

Cannot read property 'domain' of undefined at generateUrl

Hi,

I've setup everything as your readme suggests, when I try validating a page on Twitter, I get the following error:

TypeError: Cannot read property 'domain' of undefined at generateUrl (/user_code/index.js:23:31) at app.get (/user_code/index.js:61:23) at Layer.handle [as handle_request] (/user_code/node_modules/express/lib/router/layer.js:95:5) at next (/user_code/node_modules/express/lib/router/route.js:137:13) at Route.dispatch (/user_code/node_modules/express/lib/router/route.js:112:3) at Layer.handle [as handle_request] (/user_code/node_modules/express/lib/router/layer.js:95:5) at /user_code/node_modules/express/lib/router/index.js:281:22 at param (/user_code/node_modules/express/lib/router/index.js:354:14) at param (/user_code/node_modules/express/lib/router/index.js:365:14) at Function.process_params (/user_code/node_modules/express/lib/router/index.js:410:3)

myruntimeconfig.json shows:

{
  "site": {
    "domain": "a-haan.firebaseapp.com"
  },
  "rendertron": {
    "server": "https://render-tron.appspot.com"

Angular PWA Compatibility?

This looks awesome.

I've just started playing with Angular PWA. Making this work with Angular PWA structure shouldn't be hard but like I've said I don't have enough info on the subject yet. Is there an Angular example?

@davideast Have you seen this?

Enable edge caching

Cache-control headers should be set to ensure the firebase function doesn't get hit every time.

Example here: https://github.com/Polymer/hn-polymer-2/blob/master/functions/index.js#L15

Vary header will also need to be set.