Giter Site home page Giter Site logo

iv1t3 / django-middleware-fileuploadvalidation Goto Github PK

View Code? Open in Web Editor NEW
6.0 4.0 2.0 1.84 MB

A Django middleware to validate user file uploads and detect malicious content.

License: Apache License 2.0

Python 100.00%
django middleware file-upload security detection sanitization upload-sanitization file-upload-validation

django-middleware-fileuploadvalidation's Introduction

django-middleware-fileuploadvalidation (DMF)

This Django middleware provides robust validation and sanitization for file uploads. It is designed to ensure the security and integrity of files uploaded through Django applications by performing various checks, validations, and sanitization processes.

PyPI version Downloads GitHub

⚠️ Breaking Changes in Version 1.0.0: We've introduced a significant update to the upload configuration method. This change transitions from a per-path basis in the settings.py to a more flexible per-view basis using decorators. You can now configure uploads directly at the view level using decorators, offering more granular control. Please update your implementations accordingly to accommodate these changes. Examples of the new configuration method can be found in the Configuration section below.

Features

  • File Validation: Checks file types, sizes, and signatures to verify the authenticity and integrity of uploaded files.
  • File Sanitization: Cleans and modifies files to remove potentially harmful content, ensuring safe file handling.
  • Configurable Settings: Flexible configuration options to customize validation and sanitization rules based on specific needs.
  • Comprehensive Logging: Detailed logging for audit trails and debugging.
  • Support for Multiple File Types: Custom handlers for different file types (images, documents, etc.) with tailored validation and sanitization logic.

Installation

This package can be installed via pip:

pip install django-middleware-fileuploadvalidation

Then add django_middleware_fileuploadvalidation.middleware.FileUploadValidationMiddleware to the end of your MIDDLEWARE in settings.py.

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    ...,
    'django_middleware_fileuploadvalidation.middleware.FileUploadValidationMiddleware',
]

YARA rule matching

DMF supports the detection of malicious files based on custom YARA signatures. Common document/PDF signature matching is already supported due to the integration of the QuickSand framework. Custom YARA signatures can be placed under /vendor/yara/. A collection of useful YARA signatures can be found in the awesome-yara repository. The validation module will scan all files in the directory and compile the respective signatures.

ClamAV virus scanning

DMF also utilizes the ClamAV anti-virus engine. If you would like to enable ClamAV through DMF, follow our ClamAV installation instructions: ClamAV Install Guide

Configuration

By default, the upload configuration is set to the following:

{
    "clamav": False,
    "file_size_limit": None,
    "filename_length_limit": None,
    "keep_original_filename": False,
    "response_config": {
        "error_func": HttpResponseForbidden,
        "message": "File upload blocked",
        "redirect_on_block": None,
        "status": 403,
    },
    "sanitization": True,
    "uploadlogs_mode": "blocked",
    "whitelist_name": "RESTRICTED",
    "whitelist": [],
}

The middleware can be configured by adding a decorator to the respective view function that should be protected. Each field can be individually configured by passing the respective parameter to the decorator.

from django.http import HttpResponseForbidden

from django_middleware_fileuploadvalidation.decorators import file_upload_config

@file_upload_config()
def upload_default_view(request):
    # View logic for uploading files
    ...

@file_upload_config(
  file_size_limit=2000000,
  keep_original_filename=True,
  response_config={
      "error_func": HttpResponseForbidden,
      "message": "Please upload an image.",
      "status": 403,
  },
  whitelist=["application/pdf"]
)
def upload_pdf_view(request):
    # View logic for uploading PDF files
    ...

@file_upload_config(
  filename_length_limit=100,
  response_config={
      "message": "Please upload an image.",
      "redirect_on_block": 'file_list',
      "status": 403,
  },
  uploadlogs_mode='always',
  whitelist_name='IMAGE_ALL',
)
def upload_image_view(request):
    # View logic for uploading images
    ...

Options

  • clamav: ClamAV is an open source antivirus engine for detecting trojans, viruses, malware & other malicious threats. By default, ClamAV is disabled. However, if you want to enable it, you can do so by setting this to True.
  • file_size_limit: Defines the maximum allowed file size in kilobytes (kB). Files larger than this limit will be rejected. By default, there is no file size limit set.
  • filename_length_limit: Defines the maximum allowed character length of the file name. By default, there is no file length limit set.
  • keep_original_filename: By default, DMF will rename the uploaded file to a random string. If you would like to keep the original filename, set this to True.
  • response_config: Customizes the response behavior on file upload validation failure. Options include:
    • error_func: The Django HttpResponse function to execute on failure. Default: HttpResponseForbidden.
    • message: The message to display on failure. This utilizes the messaging framework of Django. The message can then be displayed in the redirected view. Default: "File upload blocked".
    • redirect_on_block: The URL name to redirect to on failure. If this is set, the error_func will be ignored.
    • Default: None.
    • status: The HTTP status code to return on failure. Default: 403.
  • sanitization: DMF supports sanitization of images and PDF documents. By default, DMF will block malicious files. However, activating the sanitization will instead sanitze files and upload them consequently.
  • uploadlogs_mode: Uploads can also be logged, to better analyze attempts afterwards. There are three different stages, which can be logged. By default, this setting is set to 'blocked'.
    • always: logs every upload attempt
    • success: logs only successful uploads
    • blocked: logs only blocked uploads
  • whitelist_name: DMF provides pre-defined whitelists. These can be used to prevent certain files from being uploaded. Each view can use an individual whitelist. This allows to have multiple upload forms with different whitelists. The following whitelists are available. By default, the whitelist is set to 'RESTRICTIVE'.
    • ALL: All files
      • AUDIO_ALL: All audio files
      • APPLICATION_ALL: All application files
      • IMAGE_ALL: All image files
      • TEXT_ALL: All text files
      • VIDEO_ALL: All video files
    • RESTRICTIVE: All restricted whitelists combined
      • AUDIO_RESTRICTIVE: audio/mpeg
      • APPLICATION_RESTRICTIVE: application/pdf
      • IMAGE_RESTRICTIVE: image/gif, image/jpeg, image/png, image/tiff
      • TEXT_RESTRICTIVE: text/plain
      • VIDEO_RESTRICTIVE: video/mp4, video/mpeg
  • whitelist: If you want to use a custom whitelist, you can define it here. The whitelist must be a list of strings. Each string must be a valid MIME type. For example: ["application/pdf", "image/png"]. This setting will override the whitelist_name setting.

django-middleware-fileuploadvalidation's People

Contributors

iv1t3 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

django-middleware-fileuploadvalidation's Issues

Support More Flexible Responses

Summary

Currently, our Django FileUploadValidation Middleware only returns HttpResponseBadRequest when it encounters a problem with a file upload. This behavior can be restrictive and may not suit all use cases in modern web applications. This issue proposes enhancements to make our response handling more flexible and adaptable to various needs.

Current Behavior

  • The middleware processes file uploads in POST requests.
  • If an issue is detected (e.g., file validation fails), the middleware returns HttpResponseBadRequest.
  • There's no option to customize this behavior or handle different scenarios in more nuanced ways.

Proposed Improvements

  • Configurable Response Types: Allow developers to configure the type of response the middleware should return in case of different types of upload errors. This could include options for different HTTP status codes or custom response messages.
  • Detailed Error Information: Instead of a generic error response, the middleware could provide detailed error messages or codes. This information can help frontend applications to better inform users about the specific issue with their upload.

Add Quicksand YARA matches to custom yara matches output

In order to better structure the reporter ouput, cluster all yara matches (from quicksand and custom) into a single dict in file.detection_results.yara_matches. Store other quicksand data in file.detection_results.quicksand.

Keyword-Search-Based Analysis: Shebangs

Besides PHP code (#17), the keyword-search-based analysis that is (to be) performed in a first step should also look for other possibly nasty stuff, such as Shebangs at the beginning of a file.

Add response detection

Look into response to detect possible exploits that have been executed and prevent information leakage.

Admin shouldn't have to add all upload configs

Currently, the web dev has to add all upload configurations to settings.py even if equal to default configuration. Implement functionality to find all missing configurations and fill with default.

Support multiple upload forms with different restrictions

Currently, upload restrictions can only be applied application-wide. The middleware should be able to differentiate between multiple upload forms and their designated usage to allow a variety of upload forms with different upload restrictions.

Django decorators could provide a suitable solution, since they could be used to notify the middleware which restrictions to apply. If no decorator was supplied, a default restriction set should be applied as a fallback option.

Implement Decorators for Easier Configuration of File Upload Validation in Django Views

Description

Currently, each Django view requires custom configuration for file upload validation. This process involves modifying the settings.py file to set various parameters for each upload type, like file size limits, whitelists, and others. This approach, while functional, can become cumbersome for developers, especially when dealing with multiple views and file types.

To streamline this process, I propose the implementation of decorators that can be applied directly to Django views. Decorators would allow developers to easily configure file upload validations on a per-view basis without the need to alter the settings.py file for each case. This would lead to cleaner, more maintainable code and a more straightforward implementation process.

Here's an example of how such a decorator could be used:

from django_middleware_fileuploadvalidation.decorators import file_upload_config

@file_upload_config(file_size_limit=2000000, keep_original_filename=True, whitelist=["application/pdf"])
def upload_pdf_view(request):
    # View logic for uploading PDF files
    ...

@file_upload_config(whitelist_name="IMAGES_ALL")
def upload_image_view(request):
    # View logic for uploading images
    ...

In this example, the file_upload_config decorator is used to define upload constraints directly above the view functions. This would override the default configurations set in the settings.py file for these specific views.

The benefits of this approach include:

  1. Improved clarity: Decorators make it explicit what configurations apply to which views.
  2. Enhanced flexibility: Developers can easily customize file upload constraints for individual views.
  3. Reduced boilerplate: Less need to repetitively define configurations in settings.py.

I believe this feature would greatly enhance the usability of the django-middleware-fileuploadvalidation project and I am eager to hear thoughts and suggestions from the community on this proposal.

Support more mimetypes

The python mimetype module only includes 130 types. Implement own mimetypes module with more types.

Core features:

  • get_all_mime_type() - return all available types
  • get_extension(mime_type) - return extension of given MIME
  • get_type(extension) - return MIME type of given extension

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.