mexsonfernandes/asynchronous_tasks-django-celery-rabbitmq-redis

A Docker container for asynchronous processing of tasks in Django.

Django Celery Redis Rabbit SQLite Docker container

Django Celery Redis Rabbit SQLite project skeleton

Versions

App	Version
Python	3.6
Django	2.1
Celery	4.1.0
Redis	2.1
Rabbit	3.7.8

What is included

• Redis as a result backend
• RabbitMQ as a message broker
• pgAdmin to view the database schema
• Scalable Docker and Docker Compose configuration (HAProxy)
• Email support when job submitted (added in task file)

Docker Compose containers

• redis (Redis result backend for Celery)
• rabbit (RabbitMQ message broker for Celery)
• web (Django application)
• worker (Celery worker)
• lb (HAProxy load balancer)

Project structure

django-celery-skeleton/
|---src                                     # django project folder
|   |---dcs                                 # django root configuration
|   |   |---settings.py
|   |   |---urls.py
|   |   |---wsgi.py
|   |   |---celeryconf.py
|   |---files
|   |   |---tempaltes                       # view templates
|   |   |   |---index.html
|   |   |---media                           # images, sized cache and placeholder
|   |   |   |---.gitignore
|   |   |   |---placeholder.png
|   |   |   |---README.md
|   |   |---static
|   |   |   |---placeholder.png
|   |   |---README.md
|   |---manage.py
|---.gitignore
|---.dockerignore
|---Vagrantfile
|---Dockerfile
|---docker-compose.yml
|---requirements.txt
|---README.md

Setup & Run

Copy git repository

git clone https://github.com/
cd 

Email settings

# add the following info in src/dcs/settings.py
EMAIL_HOST_USER = '[email protected]'
EMAIL_HOST_PASSWORD = 'XXXXXXXXXX'

Run inside docker

# build docker containers
docker-compose build

# option 1: run 1 instance of web
docker-compose up

# option 2: run 3 instances of web over load balancer
docker-compose up --scale web=3

# option 3: run in background with -d
docker-compose up -d --scale web=3

You can also run manage.py commands using docker environment, for example tests.

docker-compose run web python ./manage.py test

See docker's logs

docker-compose logs --tail 5

Run from local machine

# Install requirements
virtualenv env
source env/bin/activate
pip install -r requirements.txt

# Move to 'src' folder
cd src

# Run worker
celery worker -A dcs.celeryconf -Q default -n default@%h

# Start application on another console
python manage.py runserver

Credits

Django Celery Skeleton (updated the repository with email support and pgAdmin container)

Django Celery Skeleton

asynchronous_tasks-django-celery-rabbitmq-redis's Issues

Add badges and contribution content

Anyone willing to contribute towards Hacktoberfest can add badges for repo along with contribution content for others .