lddlww/nginx-s3-gateway

NGINX S3 Caching Gateway

NGINX S3 Gateway

Introduction

This project provides a working configuration of NGINX configured to act as an authenticating and caching gateway for to AWS S3 or another S3 compatible service. This allows you to proxy a private S3 bucket without requiring users to authenticate to it. Within the proxy layer, additional functionality can be configured such as:

• Listing the contents of a S3 bucket
• Providing an authentication gateway using an alternative authentication system to S3
• Caching frequently accessed S3 objects for lower latency delivery and protection against S3 outages
• For internal/micro services that can't authenticate against the S3 API (e.g. don't have libraries available) the gateway can provide a means to accessing S3 objects without authentication
• Compressing objects (gzip, brotli) from gateway to end user
• Protecting S3 bucket from arbitrary public access and traversal
• Rate limiting S3 objects
• Protecting a S3 bucket with a WAF
• Serving static assets from a S3 bucket alongside a dynamic application endpoints all in a single RESTful directory structure

All such functionality can be enabled within a standard NGINX configuration because this project is nothing other than NGINX with additional configuration that allows for proxying S3. It can be used as-is if the predefined configuration is sufficient, or it can serve as a base example for a more customized configuration.

If the predefined configuration does not meet your needs, it is best to borrow from the patterns in this project and build your own configuration. For example, if you want to enable SSL/TLS and compression in your NGINX S3 gateway configuration, you will need to look at other documentation because this project does not enable those features of NGINX.

Usage

This project can be run as a stand-alone container or as a Systemd service. Both modes use the same NGINX configuration and are functionally equal in terms features. However, in the case of running as a Systemd service, other services can be configured that additional functionality such as certbot for Let's Encrypt support.

Getting Started

Refer to the Getting Started Guide for how to build and run the gateway.

Directory Structure and File Descriptions

common/                          contains files used by both NGINX OSS and Plus configurations
deployments/                     contains files used for deployment technologies such as
                                 CloudFormation
docs/                            contains documentation about the project
examples/                        contains additional `Dockerfile` examples that extend the base 
                                 configuration
oss/                             contains files used solely in NGINX OSS configurations
plus/                            contains files used solely in NGINX Plus configurations
test/                            contains automated tests for validang that the examples work
Dockerfile.oss                   Dockerfile that configures NGINX OSS to act as a S3 gateway
Dockerfile.plus                  Dockerfile that builds a NGINX Plus instance that is configured
                                 equivelently to NGINX OSS - instance is configured to act as a 
                                 S3 gateway with NGINX Plus additional features enabled
Dockerfile.buildkit.plus         Dockerfile with the same configuration as Dockerfile.plus, but
                                 with support for hiding secrets using Docker's Buildkit
Dockerfile.latest-njs            Dockerfile that inherits from the last build of the gateway and
                                 then builds and installs the latest version of njs from source
settings.example                 Docker env file example
standalone_ubuntu_oss_install.sh install script that will install the gateway as a Systemd service
test.sh                          test launcher

Development

Refer to the Development Guide for more information about extending or testing the gateway.

License

All code include is licensed under the Apache 2.0 license.