alapisco/node-express-sequelize-nextjs-realworld-example-app

Node.js + Express.js + Sequelize + SQLite/PostgreSQL + Next.js fullstack static/SSG/ISG Example Realworld App

Node.js + Express.js + Sequelize + SQLite/PostgreSQL + Next.js fullstack static/SSR/SSG/ISG Example Realworld App

Got it running on Heroku as of April 2021 at https://************-realworld-next.herokuapp.com/

This is a branch of https://github.com/************/node-express-sequelize-realworld-example-app rebased on top of the master branch of that repository, we show it in a separate GitHub repository just to give it more visibility.

Any information that applies to both repositories will be kept in that README only.

Ideally we would like to have both demos on the same tree as they share all backend code, but Next.js and NPM impose too many restrictions which we don’t have the time to work around.

The frontend part of this repository is a fork of https://github.com/reck1ess/next-realworld-example-app/tree/66003772a42bf71c7b2771119121d4c9cf4b07d4 which was SSR only via API, not SSG/ISG.

We decided to merge it in-tree with the API rather than keep it a separate submodule because the fullstack implementation means that the barrier between front-end and back-end becomes is quite blurred, e.g. "front-end" files under /pages also contain functions with direct backend database access without going through the API.

Both Next.js and the backend API run on the same Express.js server via a Next.js custom server.

User-specific information such as "do I follow this article or not" is fetched by the client after it receives the user-agnostic statically generated page. The client then patches that page with this new user-specific information. See also: What works with JavaScript turned off.

The design goals of this repository are similar to https://dev.to/givehug/next-js-apollo-client-and-server-on-a-single-express-app-55l6 except that we are trying to implement the Realworld App :-) GraphQL would be ideal, but its need is greatly diminished by SSR.

This repository is a baseline for the more realworld/advanced/specialized: https://github.com/************/cirodown/tree/master/web We are trying to port back any problems that are noticed in that repository here. Some features will only be present there because of our constraint of not diverging from the Realworld spec.

• 1. Local development with SQLite
• 2. Local optimized frontend
  • 2.1. Local run as identical to deployment as possible
• 3. Heroku deployment
• 4. ISR vs SSR
• 5. What works with JavaScript turned off
• 6. TODO
  • 6.1. Single URL with multiple pages
  • 6.2. Personal user data in /user/settings
  • 6.3. TODO security
• 7. Known divergences
• 8. Bugs
• 9. Alternatives

1. Local development with SQLite

npm install
npm run dev

You can now visit http://localhost:3000 to view the website. Both API and pages are served from that single server.

2. Local optimized frontend

npm install
npm run build-dev
npm run start-dev

If you make any changes to the code, at least code under /pages for sure, you have to rebuild before they take effect in this mode, as Next.js appears to also run server-only code such as getStaticPaths from one of the webpack bundles.

Running next build is one very important test of the code, as it builds many of the most important pages of the website, and runs checks such as TypeScript type checking. You should basically run it after every single commit that touches the frontend.

2.1. Local run as identical to deployment as possible

First setup the database as described at: https://github.com/************/node-express-sequelize-realworld-example-app#local-run-as-identical-to-deployment-as-possible

Then run with:

npm run build-dev
npm run start-prod

3. Heroku deployment

The setup is analogous to: https://github.com/************/node-express-sequelize-realworld-example-app#heroku-deployment but instead of heroku git:remote -a ************-realworld-express you should use:

git remote add heroku-next https://git.heroku.com/************-realworld-next.git

This is done because this repository is normally developed as a branch of that one, which would lead to a conflicting name for the branch heroku.

You then have to add --app ************-realworld-next to any raw heroku commands to allow Heroku to differentiate between them, e.g.:

heroku run --app ************-realworld-next bash

for which we have the helper:

./heroku run bash

e.g. to delete, recreate and reseed the database:

./heroku run bin/generate-demo-data.js --force-production

We are not sure if Next.js ISR can be deployed reliably due to the ephemeral filesystem such as those in Heroku…​: https://stackoverflow.com/questions/67684780/how-to-set-where-the-prerendered-html-of-new-pages-generated-at-runtime-is-store

4. ISR vs SSR

ISR is very tempting, but is has some possibly drawbacks. Part of the goal of this repository is to decide if it is worth already or not.

With ISR, we want article contents and user pages to load instantly from a prerendered cache, as if the user were logged out.

Only after that will login-specific details be filled in by client JavaScript requests to the backend API, e.g. "have I starred/favorited this article or not".

This could lead to amazing article text loading performance, since this is the same for all users and can be efficiently cached.

The downside of that is that the user could see a two stage approach which is annoying, especially if there is no clear indication (first download, then wait, then if updates with personal details). This could be made even better by caching things client side, and userSWR which we already using likely makes that transparent, so there is hope. Even more amazing would be if it could cache across requests, e.g. from index page to an article! One day, one day, maybe with GraphQL.

Another big ISR limitation is that you can’t force a one-off immediate page update after the user edits a post, a manual refresh is generally needed: vercel/next.js#25677. However, this is not noticeable in this website, because in order to show login-specific information, we are already re-fetching the data from the API after every single request, so after a moment it gets updated to the latest version.

Our organizational principle is that all logged-in API data will be fetched from the toplevel element of each page. It will have the exact same form as the static rendering props, which come directly from the database during build rather than indirectly the API.

This data will correspond exactly to the static prerendered data, but with the user logged in. It will then simply replace the static rendered logged out version, and trigger a re-render.

This approach feels simple enough that it could even be automated in a framework manner. One day, one day.

It is true that the pass-down approach goes a bit against the phylosophy of useSWR, but there isn’t much we can do, e.g. / fetches all articles with /api/articles, and determines favorite states of multiple posts. Therefore, we if hardcoded useSWR for the article under FavoriteArticleButton, that would fetch the states from each article separately /api/articles/[slug]. We want that to happen on single article inspection however.

5. What works with JavaScript turned off

Due to ISR/SSR, all pages of the website that have distinct URLs, which includes e.g. articles and profiles but not "Your Feed" vs "Global Feed, look exactly the same with and without JavaScript for a logged out user.

For the pages without distinct URLs, we don’t know how to do this, the only way we can do it is by fetching the API with JavaScript.

SSR would require <a href elements to send custom headers, so that URLs won’t be changed, which is impossible:

• https://stackoverflow.com/questions/15835783/adding-http-request-header-to-a-a-href-link

• https://stackoverflow.com/questions/374885/can-i-change-the-headers-of-the-http-request-sent-by-the-browser

• https://softwareengineering.stackexchange.com/questions/250602/why-doesnt-the-html-dom-specification-allow-hyperlinks-to-set-an-accept-header

SSG would, in addition to the previous, require specific Next.js support for the above.

6. TODO

6.1. Single URL with multiple pages

We don’t know how to have multiple pages under a single URL in Next.js nicely. This is needed for tab navigation e.g. under / "Your Feed" vs "Global Feed", and for pagination:

• https://stackoverflow.com/questions/62628685/static-pagination-in-nextjs

• https://stackoverflow.com/questions/65471275/material-ui-tabs-with-nextjs

Note however that such functionality also makes it impossible to access those pages without JavaScript, therefore one of the main points of React, which is SEO without the need for JavaScript. Therefore maybe it doesn’t matter much.

reck1ess original implementation works around this by just fetching from the API and rendering, like a regular non-Next React app would, and this is the only way we know how to do it.

Global discussion at: gothinkster/realworld#691

6.2. Personal user data in /user/settings

reck1ess was using a mixture of SSR and client side redirects.

If you tried to access /user/settings directly e.g. by pasting it on the browser, it would redirect you to home even if you were logged in, and the server showed an error message:

Error: No router instance found.
You should only use "next/router" inside the client side of your app.

We patched to avoid that.

However, we are still currently just using data from the localStorage. This is bad because if the user changes details on another device, the data will be stale.

Also this is a very specific case of personal user data, so it doesn’t reflect the more general case of data that is not in localStorage.

Instead, we should handle /user/settings from Next.js server side, notably check JWT token there and 401 if not logged in.

6.3. TODO security

Use a markdown sanitizer, the marked library sanitize option was deprecated.

7. Known divergences

We aim to make this website look exactly like https://github.com/gothinkster/angular-realworld-example-app/tree/9e8c49514ee874e5e0bbfe53ffdba7d2fd0af36f pixel by pixel which we call "our reference implementation, and have the exact same DOM tree as much as possible, although that is hard because Angular adds a gazillion of fake nodes to the DOM it seems.

We test this by running this front/backend, and then also running angular in another browser tab. We then switch between browser tabs quickly back and forth which allows us to see even very small divergences on the UI.

Some known divergences:

• reference shows "Your Feed" for logged out user, click leads to login. This just feels wrong, not done anywhere else.

• gothinkster/angular-realworld-example-app#202 "No articles are here…​ yet" clearly broken on Angular

• reck1ess had implmented pagination really well with limits and previous/first/next/last, it would be a shame to revert that: gothinkster/realworld#684

Error messages due to API failures are too inconsistent across implementations to determine what is the correct behaviour, e.g. if you patch:

--- a/api/articles.js
+++ b/api/articles.js
@@ -104,6 +104,7 @@ router.get('/', auth.optional, async function(req, res, next) {

 router.get('/feed', auth.required, async function(req, res, next) {
   try {
+    asdf
     let limit = 20
     let offset = 0
     if (typeof req.query.limit !== 'undefined') {

• reference: shows "Loading articles…​" forever, does not inform user about error

• https://github.com/gothinkster/react-redux-realworld-example-app just throws an exception

8. Bugs

https://github.com/reck1ess/next-realworld-example-app has several UI bugs/missing functionality, some notable ones:

• reck1ess/next-realworld-example-app#22 Your Feed not working. We fixed it at d98637bb10af2bb111f0f2a6ccc72c1de6c8f351.

The implementation of reck1ess/next-realworld-example-app felt a bit quirky in a few senses:

• usage of useSWR even for data that can be already pre-rendered by Next.js such are articles. Presumably this is to give some kind of pool based realtime support? Or maybe it is just part of a workaround for the problem described at Single URL with multiple pages. But that is not what other implementations do, and neither should we. We don’t want data to update by surprise under a user’s feet.

• uses custom emotion-js CSS in addition to the global http://demo.productionready.io/main.css, which is also required since not everything was migrated to emotion.

  We later completely removed motion from this repository.

  And also has a global style.css.

  While this is good to illustrate that library, it also means that a lot of reimplementation is needed, and it is hard to be accurate at times.

  And if it were to use emotion, it should be emotion only, without the global CSS. Instead, that repo uses both, sometimes specifying the same CSS multiple times in two ways.

  It is also very annoying that they used separated defined components rather than in-tree emotion CSS which can be done as:

  <div css={css`
    font-weight: 300;
  `}>

  which leads to a much easier to read DOM tree, and less identifiers flying everywhere.

  It must be said that the port to emotion was made in a way that closely mimicks the original class/CSS structure. But still, it is just too much work, and mistakes popped up inevitably.

These are all points that we have or would like to address in this fork.

9. Alternatives

• https://github.com/lifeiscontent/realworld: Rails backend is a downside, as it adds another language to the mix besides JavaScript. But it has graphql, which is really really awesome technology.