Giter Site home page Giter Site logo

aeternalis-ingenium / fastapi-backend-template Goto Github PK

View Code? Open in Web Editor NEW
558.0 8.0 84.0 659 KB

A backend project template with FastAPI, PostgreSQL with asynchronous SQLAlchemy 2.0, Alembic for asynchronous database migration, and Docker.

License: GNU General Public License v3.0

Shell 0.51% Python 96.42% Mako 1.05% Dockerfile 2.02%
asynchronous docker docker-compose fastapi postgresql python sqlalchemy codecov githubactions jwt

fastapi-backend-template's Introduction

FastAPI Backend Application Template


This is a template repository aimed to kick-start your project with a setup from a real-world application! This template utilizes the following tech stack:

When the Docker is started, these are the URL addresses:

  • Backend Application (API docs) $\rightarrow$ http://localhost:8001/docs
  • Database editor (Adminer) $\rightarrow$ http//localhost:8081

The backend API without Docker can be found in http://localhost:8000/docs.

Why the above Tech-Stack?

Well, the easy answer is Asynchronousity and Speed!

  • FastAPI is crowned as the fastest web framework for Python and thus we use it for our backend development.
  • The database of my choice is the asynchronous version of PostgreSQL (via SQLAlchemy 2.0). Read this blog from Packt if you want to educate yourself further about the topic Asynchronous, Synchronous, Concurrency, and Parallelism.
  • Docker is a technology that packages an application into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime.

Other Technologies

The above-listed technologies are just the main ones. There are other technologies utilized in this project template to ensure that your application is robust and provides the best possible development environment for your team! These technologies are:

  • TOML $\rightarrow$ The one-for-all configuration file. This makes it simpler to setup our project.
  • Pyenv $\rightarrow$ The simplest way to manage our Python versions.
  • Pyenv-VirtualEnv $\rightarrow$ The plugin for Pyenv to manage the virtual environment for our packages.
  • Pre-Commit $\rightarrow$ Git hook scripts to identify issues and quality of your code before pushing it to GitHub. These hooks are implemented for the following linting packages:
    • Black (Python) $\rightarrow$ Manage your code style with auto-formatting and parallel continuous integration runner for Python.
    • Isort (Python) $\rightarrow$ Sort your import for clarity. Also for Python.
    • MyPy (Python) $\rightarrow$ A static type checker for Python that helps you to write cleaner code.
  • Pre-Commit CI $\rightarrow$ Continuous integration for our Pre-Commit hook that fixes and updates our hook versions.
  • CodeCov $\rightarrow$ A platform that analyzes the result of your automated tests.
  • PyTest $\rightarrow$ The testing framework for Python code.
  • DBDiagram $\rightarrow$ A platform that lets your design your database by writing SQL and converting it into ERD. This platform provides a complete symbol for entity relationships (not like many other platforms!).
  • GitHub Actions $\rightarrow$ The platform to setup our CI/CD by GitHub.
  • SQLAlchemy 2.0 $\rightarrow$ The go-to database interface library for Python. The 2.0 is the most recent update where it provides an asynchronous setup.
  • CODEOWNERS $\rightarrow$ A file for distributing the responsibilities in our project to each team/teammate.

My choice for a project development workflow is usually the Trunk-Based Development because of the straightforward approach in the collaboration workflow, hence the name trunk for the main branch repository instead of master or main.

What Code is included?

For the backend application:

  • The project, linter, and test configurations in backend/pyproject.toml.
  • 3 settings classes (development, staging, production) with the super class in backend/src/config/settings/base.py.
  • Event logger in backend/src/config/events.py.
  • The Account object table model in backend/src/models/db/account.py.
  • The Account object schema model in backend/src/models/schemas/account.py.
  • PostgreSQL database connection with asynchronous SQLAlchemy 2.0 in backend/src/repository/database.py.
  • A custom SQLAlchemy Base class in backend/src/repository/table.py
  • PostgreSQL database connection with asynchronous SQLAlchemy 2.0 in backend/src/repository/database.py.
  • Database-related events e.g. database table registration by app startup in backend/src/repository/events.py.
  • C. R. U. D. methods for Account object in backend/src/repository/crud/account.py.
  • Table classes registration file in backend/src/repository/base.py.
  • Alembic setup for auto-generating asynchronous database migrations in backend/src/repository/migration/**.
  • Alembic main configuration file in backend/alembic.ini.
  • Dependency injection for database session and repository in backend/src/api/**.
  • API endpoints for Account signup and signin in backend/src/api/routes/authentication.py.
  • API endpoints for Account get all, get 1, update, and delete in backend/src/api/routes/account.py.
  • API endpoints registration file in backend/src/api/endpoints.
  • Hashing and JWT generators and simple verification functions in backend/src/securities/**.
  • Helper functions, string messages, and error handling in backend/src/utilities/**.
  • A comprehensive FastAPI application initialization in backend/src/main.py.

For testing, I have prepared the following simple code to kick-start your test-driven development:

  • A simple replication of the backend application for testing purposes and the asynchronous test client in backend/tests/conftest.py.
  • 2 simple test functions to test the backend application initialization in tests/unit_tests/test_src.py.

For the DevOps:

  • A simple build job to test the compilation of the source code for the backend application in .github/workflows/ci-backend.yaml.
  • A simple linting job called code-style with black, isort, flake8, and mypy in .github/workflows/ci-backend.yaml.
  • An automated testing with PyTest and an automated test reporting with Codecov in in .github/workflows/ci-backend.yaml.
  • A source code responsibility distribution file in .github/CODEOWNERS (Please change the username to your own).
  • A YAML file for an automated semantic commit message in .github/workflows/ci-backend.yaml.
  • An automated test report monitoring via Codecov in codecov.yaml
  • A CI for automatically updating all linter version in the pre-commit YAML file in .pre-commit-config.YAML.

For containerization:

  • A Docker configuration that utilizes the latest Python image in backend/Dockerfile.
  • A script that ensure the backend application will restart when postgres image hasn't started yet in backend/entrypoint.sh.
  • Setting up Postgres image for our database server, Adminer for our database editor, and backend_app for our backend application's container in docker-compose.yaml.

For the team development environment:

  • A pre-commit hooks for Black, Isort, and MyPy to ensure the conventional commit message before pushing an updated code into the remote repository in .pre-commit-config.YAML.
  • All secret variables are listed in .env.example. You need to copy these variables and set the values respectively to your need and save them in a new .env in the root directory.

Setup Guide

This backend application is setup with Docker. Nevertheless, you can see the full local setup without Docker in backend/README.md.

  1. Before setting up the backend app, please create a new directory called coverage for the testing report purpose:

    cd backend && mkdir coverage
    
  2. Backend app setup:

    # Creating VENV
    pyenv virtualenv 3.11.0 any_venv_name
    pyenv local any_venv_name
    
    # Install dependencies
    pip3 install -r requirements.txt
    
    # Test run your backend server
    uvicorn src.main:backend_app --reload
    
  3. Testing with PyTest: Make sure that you are in the backend/ directory.

    # For testing without Docker
    pytest
    
    # For testing within Docker
    docker exec backend_app pytest
    
  4. Pre-Commit setup:

    # Make sure you are in the ROOT project directory
    pre-commit install
    pre-commit autoupdate
    
  5. Backend app credentials setup: If you are not used to VIM or Linux CLI, then ignore the echo command and do it manually. All the secret variables for this template are located in .env.example.

    If you want to have another name for the secret variables, don't forget to change them also in:

    • backend/src/config/base.py
    • docker-compose.yaml
    # Make sure you are in the ROOT project directory
    touch .env
    
    echo "SECRET_VARIABLE=SECRET_VARIABLE_VALUE" >> .env
    
  6. CODEOWNERS setup: Go to .github/ and open CODEOWNERS file. This file is to assign the code to a specific team member so you can distribute the weights of the project clearly.

  7. Docker setup:

     # Make sure you are in the ROOT project directory
     chmod +x backend/entrypoint.sh
    
     docker-compose build
     docker-compose up
    
     # Every time you write a new code, update your container with:
     docker-compose up -d --build
    
  8. (IMPORTANT) Database setup:

     # (Docker) Generate revision for the database auto-migrations
     docker exec backend_app alembic revision --autogenerate -m "YOUR MIGRATION TITLE"
     docker exec backend_app alembic upgrade head    # to register the database classes
     
     # (Local) Generate revision for the database auto-migrations
     alembic revision --autogenerate -m "YOUR MIGRATION TITLE"
     alembic upgrade head    # to register the database classes
    
  9. Go to https://about.codecov.io/, and sign up with your github to get the CODECOV_TOKEN

  10. Go to your GitHub and register all the secret variables (look in .env.example) in your repository (settings $\rightarrow$ (scroll down a bit) Secrets $\rightarrow$ Actions $\rightarrow$ New repository secret)

IMPORTANT: Without the secrets registered in Codecov and GitHub, your CI will fail and life will be horrible ๐Ÿคฎ๐Ÿคฌ IMPORTANT: Remember to always run the container update every once in a while. Without the arguments -d --build, your Docker dashboard will be full of junk containers!

Project Structure

.github/
โ”œโ”€โ”€ workflows/
    โ”œโ”€โ”€ ci-backend.yaml                 # A CI file for the backend app that consists of `build`, `code-style`, and `test`
โ”œโ”€โ”€ CODEOWNERS                          # A configuration file to distribute code responsibility
โ”œโ”€โ”€ semantic.yaml                       # A configuration file for ensuring an automated semantic commit message

backend/
โ”œโ”€โ”€ coverage/
โ”œโ”€โ”€ src/
    โ”œโ”€โ”€ api/
        โ”œโ”€โ”€ dependencies/               # Dependency injections
            โ”œโ”€โ”€ session.py
            โ”œโ”€โ”€repository.py
        โ”œโ”€โ”€ routes/                     # Endpoints
            โ”œโ”€โ”€ account.py              # Account routes
            โ”œโ”€โ”€ authentication.py       # Signup and Signin routes
        โ”œโ”€โ”€ endpoints.py                # Endpoint registration
    โ”œโ”€โ”€ config/
        โ”œโ”€โ”€ settings/
            โ”œโ”€โ”€ base.py                 # Base settings / settings parent class
                โ”œโ”€โ”€ development.py      # Development settings
                โ”œโ”€โ”€ environments.py     # Enum with PROD, DEV, STAGE environment
                โ”œโ”€โ”€ production.py       # Production settings
                โ”œโ”€โ”€ staging.py          # Test settings
        โ”œโ”€โ”€ events.py                   # Registration of global events
        โ”œโ”€โ”€ manager.py                  # Manage get settings
    โ”œโ”€โ”€ models/
        โ”œโ”€โ”€ db/
            โ”œโ”€โ”€ account.py              # Account class for database entity
        โ”œโ”€โ”€ schemas/
            โ”œโ”€โ”€ account.py              # Account classes for data validation objects
            โ”œโ”€โ”€ base.py                 # Base class for data validation objects
    โ”œโ”€โ”€ repository/
        โ”œโ”€โ”€ crud/
            โ”œโ”€โ”€ account.py              # C. R. U. D. operations for Account entity
            โ”œโ”€โ”€ base.py                 # Base class for C. R. U. D. operations
        โ”œโ”€โ”€ migrations/
            โ”œโ”€โ”€ versions/
            โ”œโ”€โ”€ env.py                  # Generated via alembic for automigration
            โ”œโ”€โ”€ script.py.mako          # Generated via alembic
        โ”œโ”€โ”€ base.py                     # Entry point for alembic automigration
        โ”œโ”€โ”€ database.py                 # Database class with engine and session
        โ”œโ”€โ”€ events.py                   # Registration of database events
        โ”œโ”€โ”€ table.py                    # Custom SQLAlchemy Base class
    โ”œโ”€โ”€ security/
        โ”œโ”€โ”€ hashing/
            โ”œโ”€โ”€ hash.py                 # Hash functions with passlib
            โ”œโ”€โ”€ password.py             # Password generator with hash functions
        โ”œโ”€โ”€ authorizations/
            โ”œโ”€โ”€ jwt.py                  # Generate JWT tokens with python-jose
        โ”œโ”€โ”€ verifications/
            โ”œโ”€โ”€ credentials.py          # Check for attributes' availability
    โ”œโ”€โ”€ utilities/
        โ”œโ”€โ”€ exceptions/
            โ”œโ”€โ”€ http/
                โ”œโ”€โ”€ http_exc_400.py     # Custom 400 error handling functions
                โ”œโ”€โ”€ http_exc_401.py     # Custom 401 error handling functions
                โ”œโ”€โ”€ http_exc_403.py     # Custom 403 error handling functions
                โ”œโ”€โ”€ http_exc_404.py     # Custom 404 error handling functions
            โ”œโ”€โ”€ database.py             # Custom `Exception` class
            โ”œโ”€โ”€ password.py             # Custom `Exception` class
        โ”œโ”€โ”€ formatters/
            โ”œโ”€โ”€ datetime_formatter.py   # Reformat datetime into the ISO form
            โ”œโ”€โ”€ field_formatter.py      # Reformat snake_case to camelCase
        โ”œโ”€โ”€ messages/
            โ”œโ”€โ”€ http/
                โ”œโ”€โ”€ http_exc_details.py	# Custom message for HTTP exceptions
    โ”œโ”€โ”€ main.py                         # Our main backend server app
โ”œโ”€โ”€ tests/
    โ”œโ”€โ”€ end_to_end_tests/               # End-to-end tests
    โ”œโ”€โ”€ integration_tests/              # Integration tests
    โ”œโ”€โ”€ security_tests/                 # Security-related tests
    โ”œโ”€โ”€ unit_tests/                     # Unit tests
        โ”œโ”€โ”€ test_src.py                 # Testing the src directory's version
    โ”œโ”€โ”€ conftest.py                     # The fixture codes and other base test codes
โ”œโ”€โ”€ Dockerfile                          # Docker configuration file for backend application
โ”œโ”€โ”€ README.md                           # Documentation for backend app
โ”œโ”€โ”€ entrypoint.sh                       # A script to restart backend app container if postgres is not started
โ”œโ”€โ”€ alembic.ini                         # Automatic database migration configuration
โ”œโ”€โ”€ pyproject.toml                      # Linter and test main configuration file
โ”œโ”€โ”€ requirements.txt                    # Packages installed for backend app
.dockerignore                           # A file that list files to be excluded in Docker container
.gitignore                              # A file that list files to be excluded in GitHub repository
.pre-commit-config.yaml                 # A file with Python linter hooks to ensure conventional commit when committing
LICENSE.md                              # A license to use this template repository (delete this file after using this repository)
README.md                               # The main documentation file for this template repository
codecov.yaml                            # The configuration file for automated testing CI with codecov.io
docker-compose.yaml                     # The main configuration file for setting up a multi-container Docker

Final Step

You can delete these 3 files (or change its content based on your need):

  • LICENSE.md
  • README.md
  • backend/README.md

Enjoy your development and may your technology be forever useful to everyone ๐Ÿ˜‰๐Ÿš€๐Ÿงฌ


fastapi-backend-template's People

Contributors

aeternalis-ingenium avatar antonio-pedro99 avatar cclauss avatar duchuyvp avatar kumaravelabinbev avatar pre-commit-ci[bot] avatar timotheemm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fastapi-backend-template's Issues

Validation errors on Run

Show this errors in console at run

pydantic_core._pydantic_core.ValidationError: 11 validation errors for BackendDevSettings
BACKEND_SERVER_HOST
  Extra inputs are not permitted [type=extra_forbidden, input_value='127.0.0.1', input_type=str]
    For further information visit https://errors.pydantic.dev/2.0.3/v/extra_forbidden
BACKEND_SERVER_PORT
  Extra inputs are not permitted [type=extra_forbidden, input_value='8000', input_type=str]
    For further information visit https://errors.pydantic.dev/2.0.3/v/extra_forbidden
BACKEND_SERVER_WORKERS
  Extra inputs are not permitted [type=extra_forbidden, input_value='4', input_type=str]
    For further information visit https://errors.pydantic.dev/2.0.3/v/extra_forbidden

AttributeError: 'pydantic_core._pydantic_core.MultiHostUrl' object has no attribute 'replace'

backend_app | File "/usr/backend/src/repository/database.py", line 42, in
backend_app | async_db: AsyncDatabase = AsyncDatabase()
backend_app | ^^^^^^^^^^^^^^^
backend_app | File "/usr/backend/src/repository/database.py", line 19, in init
backend_app | url=self.set_async_db_uri,
backend_app | ^^^^^^^^^^^^^^^^^^^^^
backend_app | File "/usr/backend/src/repository/database.py", line 36, in set_async_db_uri
backend_app | self.postgres_uri.replace("postgresql://", "postgresql+asyncpg://")
backend_app | ^^^^^^^^^^^^^^^^^^^^^^^^^
backend_app | AttributeError: 'pydantic_core._pydantic_core.MultiHostUrl' object has no attribute 'replace'

TypeError: MultiHostUrl.__new__() got an unexpected keyword argument 'scheme'

backend_app | File "/usr/backend/src/api/dependencies/session.py", line 11, in
backend_app | from src.repository.database import async_db
backend_app | File "/usr/backend/src/repository/database.py", line 43, in
backend_app | async_db: AsyncDatabase = AsyncDatabase()
backend_app | ^^^^^^^^^^^^^^^
backend_app | File "/usr/backend/src/repository/database.py", line 15, in init
backend_app | self.postgres_uri: pydantic.PostgresDsn = pydantic.PostgresDsn(
backend_app | ^^^^^^^^^^^^^^^^^^^^^
backend_app | File "/usr/local/lib/python3.12/typing.py", line 1133, in call
backend_app | result = self.origin(*args, **kwargs)
backend_app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend_app | TypeError: MultiHostUrl.new() got an unexpected keyword argument 'scheme'

Running pytests

Hi.

In root dir README it says:
Make sure that you are in the backend/ directory

In backend/README.md:
INFO: For running the test, make sure you are in the root directory and NOT in the backend/ directory!

Try both ways i am not able to run tests,

    from src.main import backend_app
E   ModuleNotFoundError: No module named 'src'

TypeError: MultiHostUrl.__new__() got an unexpected keyword argument 'scheme'

backend_app | File "/usr/backend/src/api/dependencies/session.py", line 11, in
backend_app | from src.repository.database import async_db
backend_app | File "/usr/backend/src/repository/database.py", line 43, in
backend_app | async_db: AsyncDatabase = AsyncDatabase()
backend_app | ^^^^^^^^^^^^^^^
backend_app | File "/usr/backend/src/repository/database.py", line 15, in init
backend_app | self.postgres_uri: pydantic.PostgresDsn = pydantic.PostgresDsn(
backend_app | ^^^^^^^^^^^^^^^^^^^^^
backend_app | File "/usr/local/lib/python3.12/typing.py", line 1133, in call
backend_app | result = self.origin(*args, **kwargs)
backend_app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend_app | TypeError: MultiHostUrl.new() got an unexpected keyword argument 'scheme'

Authentication

I apologize if I didn't understand correctly, but it seems that the authentication itself was not implemented, and we do not have protected routes. Is that correct?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.