Wild Me - Houston Backend Server
Houston is a REST API component within the CODEX application suite. It is used to glue the frontend to Ecological Data Module (EDM) and Wildbook-IA.
For a high-level explanation of the application in relation to other CODEX applications read the documentation at CODEX - Houston.
About this implementation
This project showcases my vision on how the RESTful API server should be implemented.
The goals that were achieved in this example:
- RESTful API server should be self-documented using OpenAPI (fka Swagger) specifications, so interactive documentation UI is in place
- Authentication is handled with OAuth2 and using Resource Owner Password Credentials Grant (Password Flow) for first-party clients makes it usable not only for third-party "external" apps
- Permissions are handled (and automaticaly documented)
- PATCH method can be handled accordingly to RFC 6902
- Extensive testing with good code coverage.
The package Flask-RESTX has been patched (see flask_restx_patched
folder), so it can handle Marshmallow schemas and Webargs arguments.
Code Style and Development Guidelines
Project's Code Structure
Background and Periodic Tasks using Celery
See Background and Periodic Tasks
Installation
Using docker-compose (recommended)
Setup
git clone --recurse-submodules https://github.com/WildMeOrg/houston.git
# Option 1 - Activate Codex App
./scripts/codex/activate.sh
docker-compose up
# Option 2 - Use Codex Config Explicitly
docker-compose -f docker-compose.codex.yml --env-file .env.codex up
Surf to http://localhost:84/
Installing from source
Development Setup Note
Installation of Houston and the other components of Codex from source is intended to facilitate development leveraging the docker-compose environment.
See Development Environment section in Contributing to Houston for details. Full deployment of Codex outside docker-compose orchestration is not supported, and any changes should not be considered finished until they have been tested in the docker-compose environment.
Clone the Project
git clone --recurse-submodules https://github.com/WildMeOrg/houston.git
cd houston/
Setup Codex Environment
It is recommended to use virtualenv or Anaconda/Miniconda to manage Python dependencies. Please, learn details yourself. For quickstart purposes the following will set up a virtualenv for you:
./scripts/codex/venv.sh
source virtualenv/houston3.7/bin/activate
# To add bash-completion
export SCRIPT="$(pwd)/.invoke-completion.sh"
invoke --print-completion-script bash > $SCRIPT
echo "source $SCRIPT" >> virtualenv/houston3.7/bin/activate
Set up and install the package:
invoke dependencies.install
Run Server
NOTE: All dependencies and database migrations will be automatically handled, so go ahead and turn the server ON! (Read more details on this in Tips section)
export HOUSTON_APP_CONTEXT=codex
$ invoke app.run
Deploy Server
In general, you deploy this app as any other Flask/WSGI application. There are
a few basic deployment strategies documented in the ./deploy/
folder.
Usage
Open online interactive API documentation: http://127.0.0.1:5000/api/v1/
Autogenerated swagger config is always available from http://127.0.0.1:5000/api/v1/swagger.json
NOTE: Use On/Off switch in documentation to sign in.
Usage Tips
Once you have invoke, you can learn all available commands related to this project from:
$ invoke --list
Learn more about each command with the following syntax:
$ invoke --help <task>
For example:
$ invoke --help codex.run
Usage: inv[oke] [--core-opts] codex.run [--options] [other tasks here ...]
Docstring:
Run DDOTS RESTful API Server.
Options:
-d, --[no-]development
-h STRING, --host=STRING
-i, --[no-]install-dependencies
-p, --port
-u, --[no-]upgrade-db
Use the following command to enter ipython shell (ipython
must be installed):
$ invoke app.env.enter
codex.run
and app.env.enter
tasks automatically prepare all dependencies
(using pip install
) and migrate database schema to the latest version.
Database schema migration is handled via app.db.*
tasks group. The most
common migration commands are app.db.upgrade
(it is automatically run on
codex.run
), and app.db.migrate
(creates a new migration).
You can use better_exceptions
package to enable detailed tracebacks. Just add better_exceptions
to the
app/requirements.txt
and import better_exceptions
in the app/__init__.py
.
Dependencies
Project Dependencies
This project requires either Python >= 3.7 or Docker.
The tus portions of this application require Redis to facilitate resumable file uploads.
GitLab (community edition) is required for asset and submission storage and management.
Postgres is an optional dependency that can be used for a highly reliable scaled database solution.
Documentation
Build the documentation
To build and view the documentation use the following commands:
cd docs
pip install -r requirements.txt
make html
open _build/html/index.html
reCAPTCHA
Register at https://www.google.com/recaptcha/admin/create for reCAPTCHA v3
.
Add the site (public) key and secret key to docker-compose.override.yml
:
services:
houston:
environment:
RECAPTCHA_PUBLIC_KEY: "recaptcha-public-key"
RECAPTCHA_SECRET_KEY: "recaptcha-secret-key"
docker-compose up -d
to update running services.
Settings can also be set via SiteSetting
, the keys are
recaptchaPublicKey
and recaptchaSecretKey
.
License
This software is subject to the provisions of Apache License Version 2.0 (APL). See LICENSE
for details. Copyright (c) 2020 Wild Me