tedivm/tedivms-flask

Flask starter app with celery, bootstrap, and docker environment

tedivm's flask starter app

This code base serves as starting point for writing your next Flask application.

This package is designed to allow developers to start working on their specific features immediately while also making it simple to deploy the project into production. It contains a number of configured extensions and libraries as well as unique features specifically built for this package. It also is completely dockerized, with both a docker-compose testenv and the ability to easily make images off of the application for pushing into production.

Rob's Awesome Template

If you like this project you may also like my latest template, Rob's Awesome Python Template. It is updated with modern best practices and a variety of optional services.

Code characteristics

• Tested on Python 3.3, 3.4, 3.5, 3.6, and 3.7
• Complete docker environment.
• Images for both the web application and the celery worker.
• Full user management system.
• Server side session storage.
• An API system with API tokens and route decorators.
• Well organized directories with lots of comments.
• Includes test framework (py.test and tox)
• Includes database migration framework (alembic, using Flask-Migrate)
• Sends error emails to admins for unhandled exceptions

Configured Extensions and Libraries

With thanks to the following Flask extensions and libraries:

• Beaker for caching and session management.
• Celery for running asynchronous tasks on worker nodes.
• Click for the creation of command line tools.
• Flask the microframework framework which holds this all together.
• Flask-Login allows users to login and signout.
• Flask-Migrate integrates Alembic into Flask to handle database versioning.
• Flask-SQLAlchemy integrates SQLAlchemy into Flask for database modeling and access.
• Flask-User adds user management and authorization features.
• Flask-WTF integrates WTForms into Flask to handle form creation and validation.

In addition the front end uses the open source versions of:

• Bootstrap
• CoreUI
• Font Awesome

Unique Features

• Database or LDAP Authentication - Applications built with this project can use the standard database backed users or can switch to LDAP authentication with a few configuration settings.

• API Authentication and Authorization - this project can allow people with the appropriate role to generate API Keys, which in turn can be used with the roles_accepted_api decorator to grant API access to specific routes.

• Versatile Configuration System - this project can be configured with a combination of configuration files, AWS Secrets Manager configuration, and environmental variables. This allows base settings to be built into the deployment, secrets to be managed securely, and any configuration value to be overridden by environmental variables.

• A makefile with a variety of options to make common tasks easier to accomplish.

• A Celery based asynchronous task management system. This is extremely useful for long running tasks- they can be triggered in the web interface and then run on a worker node and take as long as they need to complete.

Setting up a development environment

First we recommend either cloning this repository with the "Use this template" button on Github.

We assume that you have make and docker.

# Clone the code repository into ~/dev/my_app
mkdir -p ~/dev
cd ~/dev
git clone https://github.com/tedivm/tedivms-flask my_app
cd my_app

# For the first run, and only the first run, we need to create the first round of SQLAlchemy models.
make init_db

# Create the 'my_app' virtual environment and start docker containers
make testenv

# Restart docker app container
docker-compose restart app

# Start a shell in the container running the application
docker-compose exec app /bin/bash

Configuration

Application configuration

To set default configuration values on the application level- such as the application name and author- edit ./app/settings.py. This should be done as a first step whenever using this application template.

Configuration File

A configuration file can be set with the environmental variable APPLICATION_SETTINGS.

AWS Secrets Manager

Configuration can be loaded from the AWS Secrets Manager by setting the environmental variables AWS_SECRETS_MANAGER_CONFIG and AWS_SECRETS_REGION.

Environmental Variables

Any environmental variables that have the same name as a configuration value in this application will automatically get loaded into the app's configuration.

Configuring LDAP

Any installation can run with LDAP as its backend with these settings.

USER_LDAP=true
LDAP_HOST=ldap://ldap
LDAP_BIND_DN=cn=admin,dc=example,dc=org
LDAP_BIND_PASSWORD=admin
LDAP_USERNAME_ATTRIBUTE=cn
LDAP_USER_BASE=ou=users,dc=example,dc=org
LDAP_GROUP_OBJECT_CLASS=posixGroup
LDAP_GROUP_ATTRIBUTE=cn
LDAP_GROUP_BASE=ou=groups,dc=example,dc=org
LDAP_GROUP_TO_ROLE_ADMIN=admin
LDAP_GROUP_TO_ROLE_DEV=dev
LDAP_GROUP_TO_ROLE_USER=user
LDAP_EMAIL_ATTRIBUTE=mail

Initializing the Database

# Initialize the database. This will create the `migrations` folder and is only needed once per project.
make init_db

# This creates a new migration. It should be run whenever you change your database models.
make upgrade_models

Running the app

# Start the Flask development web server
make testenv

Point your web browser to http://localhost/

You can make use of the following users:

• email [email protected] with password Password1.
• email [email protected] with password Password1.
• email [email protected] with password Password1.

Running the automated tests

# To run the test suite.
make run_tests

Acknowledgements

Flask-Dash was used as a starting point for this code repository. That project was based off of the Flask-User-starter-app.

Authors

• Robert Hafner (tedivms-flask) -- [email protected]

• Matt Hogan (flaskdash) -- matt AT twintechlabs DOT io
• Ling Thio (flask-user) -- ling.thio AT gmail DOT com

tedivms-flask's Issues

Celery worker exited.

Hi,

Thanks for this project, it seems to be a really great boilerplate.
Just one question : when I make testenv, during the launching of the network, I encounter an issue with the Celery worker.

worker_1    | Launching celery worker with beat enabled
worker_1    | celery worker -A app.worker:celery --loglevel=info --autoscale=10,2 -B -s /home/apprunner/celerybeat-schedule
worker_1    | Error:
worker_1    | Unable to load celery application.
worker_1    | The module app.tasks was not found.
worker_1    |
my_app_worker_1 exited with code 1

I use Docker v 19.03.2, and I don't get why the app.tasks module is not found. Is it an issue with rabbitmq-server?
Any help would be really appreciate.

email_validator missing from requirments.txt

make init_db fails because email_validator is missing

Steps to Replicate

1. Clone repo
2. make init_db

Steps to Fix

• add email_validator to requirements.txt, rerun make init_db
• or, install email_validator

Error when user not logged in

Accessing the site when not logged in throws this error:

app_1 | Traceback (most recent call last):
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2464, in call
app_1 | return self.wsgi_app(environ, start_response)
app_1 | File "/usr/local/lib/python3.7/site-packages/beaker/middleware.py", line 156, in call
app_1 | return self.wrap_app(environ, session_start_response)
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2450, in wsgi_app
app_1 | response = self.handle_exception(e)
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1867, in handle_exception
app_1 | reraise(exc_type, exc_value, tb)
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
app_1 | raise value
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
app_1 | response = self.full_dispatch_request()
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
app_1 | rv = self.handle_user_exception(e)
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1816, in handle_user_exception
app_1 | return self.handle_http_exception(e)
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1744, in handle_http_exception
app_1 | return handler(e)
app_1 | File "./app/init.py", line 286, in error_pagenotfound
app_1 | return show_error(404, 'Page not found.')
app_1 | File "./app/init.py", line 274, in show_error
app_1 | return render_template('pages/errors.html', error_code=status, message=message), status
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/templating.py", line 140, in render_template
app_1 | ctx.app,
app_1 | File "/usr/local/lib/python3.7/site-packages/flask/templating.py", line 120, in _render
app_1 | rv = template.render(context)
app_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
app_1 | self.environment.handle_exception()
app_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
app_1 | reraise(*rewrite_traceback_stack(source=source))
app_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
app_1 | raise value.with_traceback(tb)
app_1 | File "/app/app/templates/pages/errors.html", line 1, in top-level template code
app_1 | {% extends "common/dark_base.html" %} {# common/page_base.html extends layout.html #}
app_1 | File "/app/app/templates/common/dark_base.html", line 1, in top-level template code
app_1 | {% extends "dark_layout.html" %}
app_1 | File "/app/app/templates/dark_layout.html", line 33, in top-level template code
app_1 | {% block body %}
app_1 | File "/app/app/templates/dark_layout.html", line 67, in block "body"
app_1 | {% if current_user.has_role('admin') or current_user.has_role('dev') %}
app_1 | jinja2.exceptions.UndefinedError: 'flask_login.mixins.AnonymousUserMixin object' has no attribute 'has_role'

Add Celery Redbeat when redis is enabled

https://redbeat.readthedocs.io/en/latest/index.html

Can't delete API keys

File "./app/views/apikeys.py", line 44, in apikeys_delete
form = forms.Confirm(request.form)
AttributeError: module 'app.utils.forms' has no attribute 'Confirm'

Review and consider adding Flask Talisman

https://github.com/GoogleCloudPlatform/flask-talisman

Add `black` code formatter.

https://pypi.org/project/black/

Fix for ImportError: cannot import name 'url_encode' from 'werkzeug'

make init-db fails with above.

Reason: flask_wtf v0.14.2 says, from werkzeug import url_encode. Werkzeug changed the url_encode function from werkzeug.url_encode to werkzeug.urls.url_encode

Fix: update requirements.txt to Flask-WTF==0.14.3

Caveat: I haven't checked whether there are other incompatibilities introduced by bumping Flask-WTF (although it seems unlikely).

Can't connect to app ERR_CONNECTION_REFUSED

I'm on Ubuntu 18.04 and Docker version 19.03.6, build 369ce74a3c trying this a few times but can never connect to port 5000 after running 'make testenv'. I don't see any problems when I run 'make init_db'.

I've tried 'docker-compose restart app' but I get the same result.

I'm able to enter the container with 'docker-compose exec app /bin/bash' and can see nginx, supervisord and uwsgi running.

I've changed the basic defaults including SECRET_KEY in /app/settings.py with no luck.
I can see a container listening on 5000
docker-pr 20749 root 4u IPv6 3594430 0t0 TCP *:5000 (LISTEN)

log.txt