Containerize Nginx

[codeforpdx]

Containerize Nginx

[erikjasso]

@wittejm Much of this task of configuring the web server and running it in a Docker container can be done while we figure out how the front-end code and assets are built. In fact this should be orthogonal to React. Independent of which front-end framework we use and how it is built we will still use a web server to serve the static content (as well as route requests of dynamic content to uWSGI). Does that make sense?

[erikjasso]

Right, we'll have a location block like that for the static web site and one for the Python API (as well as one or more server blocks listening on some port(s)).

Incoming requests will have the base form:

https://server_name/path

where

server_name is our domain name when deployed, or localhost/127.0.0.1 during development
path is a path to some resource (directory, file, or endpoint)

Nginx will look at path and try to match it with a location block. When path is to a static file or directory containing static files Nginx will return it to the client directly. When path is to an endpoint managed by our Flask application Nginx will route the request through the uWSGI server to the Flask app.

[wittejm]

From a conversation with @maxkwallace I realized that building the web app outside of docker and copying it in to make the docker image is the wrong way to go, and instead the builder should run in the container. I'll change it to that, (and claim the related issue #45 ) and then go ahead with a pull request when the thing works.

[wittejm]

Given @maxkwallace’s comment in issue #45, is it correct that the nginx server would not need to serve any static content? Meaning there’s no need to set up the additional docker volume?

Edit: nevermind, I realized this question made zero sense, I was completely misunderstanding several things.

[wittejm]

I realized that because of the reliance on react knowledge (which I don't have) and that it sounds like some react design questions are still unresolved (like issue #83 ), I should not actually be the one working on this.

[wittejm]

Ok, that partially makes sense. Initially I was misunderstanding the role of the nginx web server and thought it should only reroute http requests directly to the uWSGI server, which doesn't seem right. So, it sounds like instead the nginx configuration needs to expose uWSGI only for requests made by the frontend, which will still behave as though it is accessing a local socket. Is that correct?

The question of static content pathing is still unclear to me (even though it seems like it should be simple). It sounds like, for nginx to serve static content it only needs to specify the path from the server’s root directory to the public/ directory. Then the block in the nginx config file would be:

location / {
root /path/to/build/public/;
}
and that path is the destination of the react build.

[wittejm]

Ok, that helps. I’ll keep poking at this and see if I can get something working. Thanks Erik!

[maxkwallace]

@wittejm since you tagged me earlier I'll just jump in and add my $0.02.

Serving static content via nginx is one way to send the frontend code to the user's browser. But it's not the only way and it's not strictly necessary to use static content.

For instance, there could be a method in Flask that responds to all requests to the root URL by returning a string of HTML for the app's homepage. We could even take all our frontend code and paste it into a script tag in that HTML. This is one way to serve the frontend without using static content. But it's not a good way since we'd have to update the python code every time we updated the frontend code.

Another way is to have the frontend code in the repo, and have the Flask method read the HTML and JS files off the filesystem. That's basically "static content" except handled in Flask instead of nginx, so it'd be less performant. But it might be faster to code.

I'd say go ahead with what Erik discussed in his last comment. The main thing I wanted to say is just that "static content" is not such a well-defined concept and there are multiple ways to send the frontend code to the browser, all of which have their own advantages and disadvantages.

[wittejm]

Cool, thanks for that added context @maxkwallace . A point of confusion I had was believing that html code was still being generated and served by something like a node (and maybe react) process which would be running in the nginx container. Similar to your example where flask responds to requests with an html string, which is the structure that I've worked with in the past when building simple web apps.

But now it makes sense that react is only running to build the web pages beforehand, which then get stored to be served by nginx. So the nginx configuration will route any paths starting with /api/ to the uWSGI server, and every other path is just routed to the frontend build location.

I have two new questions (also for @erikjasso ) about calls to endpoints. First: about how the version number is handled in the path. Is the nginx configuration agnostic to version number? I think that would require the client page to specify the version in the path like:

server_name/api/<version>/endpoint_name

and nginx forwards the request with a path like

uwsgi_server_name:port/<version>/endpoint_name 

With that version.

OR, does the nginx configuration decide which version is expected by the frontend and determine the upstream path accordingly? That might require the configuration be updated manually as the frontend version changes which doesn’t seem correct.

Second question, implied by my path format above but just to verify explicitly : , “/api” should get dropped from the path that is forwarded to the uWSGI server, correct?

[maxkwallace]

@wittejm minor correction: any web pages that are built beforehand are created by Webpack or another build tool. React itself only runs in the user's browser (and generates the HTML they see) after they've downloaded the Javascript asset containing the React code, which must be referenced in a script tag in an HTML page sent by the server.

[wittejm]

@maxkwallace Ah gotcha, thanks.

[wittejm]

I made the listed file changes (except for changes that depend on the https cert) in a new branch on my forked repo, with readme here:
https://github.com/wittejm/recordexpungPDX/blob/nginx-stack-dev/config/nginx/README.md

@erikjasso I have several questions I hope we could look at at the meetup this week, before I file a pull request. The branch includes a lengthy readme of my changes and questions. It gets pretty granular, but hopefully if we can work through it together we’ll get this task mostly wrapped up.

[maxkwallace]

@wittejm let me know if you run into any issues related to the npm / React side of things! Some quick Googling let me to this article which seems to match our use case:

https://medium.com/thepeaklab/how-to-deploy-a-react-application-to-production-with-docker-multi-stage-builds-4da347f2d681

We are using create-react-app.

[wittejm]

@maxkwallace cool, thanks for the link!

Following @erikjasso 's suggestion of storing static content in a docker volume (mentioned above and in issue #45), I was looking into the idea of having a docker container separate from the web server that would run the build, save the result into the docker volume and then exit. I got a toy version of this working following this stackoverflow post:

https://stackoverflow.com/questions/38441159/docker-named-volume-not-updating#38444786

I suppose the main benefit of keeping the web app build process separate from the web server container is that we can update the web app files or re-deploy the web server independent of each other, because they're interacting only by way of the docker volume.

On the other hand, since the static web content would only change when there's a version update, it seems also reasonable to build it the way described in that tutorial -- namely that the static files exist only as part of the web server image, and there's no need for a docker volume (the multi-stage build feature is nice to know!). Your thoughts @erikjasso ?