icey-zhang/e2e-mfd

E2E-MFD-OOD

E2E-MFD

E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detection

The code is based on MMdetection 2.26.0, MMrotate 0.3.4 and MMCV-full 1.7.2. We modify its data loading, related classes, and functions. We revise the MMdetection and MMrotate to a multi-modal oriented detection framework to facilitate Multimodal Object Detection.

Overview

Getting Started

Installation

ref : mmrotate installation and mmdetection installation

Step 1: Clone the E2E-MFD repository:

To get started, first clone the E2E-MFD repository and navigate to the project directory:

git clone *****
cd *****

Step 2: Environment Setup:

E2E-MFD recommends setting up a conda environment and installing dependencies via pip. Use the following commands to set up your environment:

Create and activate a new conda environment

conda create -n E2E-MFD python=3.9.17
conda activate E2E-MFD

If you develop and run mmrotate directly, install it from source

pip install -v -e .

Install Dependencies

pip install -r requirements.txt

Prepare the dataset DroneVehicle

DroneVehicle is a publicly available dataset.

you can download the dataset at baiduyun with train (code:ngar) and test (code:tqwc).

root
├── DroneVehicle
│   ├── train
│   │   ├── rgb
│   │   │   ├── images
│   │   │   ├── labels
│   │   ├── ir
│   │   │   ├── images
│   │   │   ├── labels
│   ├── test
│   │   ├── rgb
│   │   │   ├── images
│   │   │   ├── labels
│   │   ├── ir
│   │   │   ├── images
│   │   │   ├── labels

Begin to train and test

Use the config file with this.

python ./tools/train.py
python ./tools/test.py

Generate fusion images

python ./tools/generate_fusion_image.py

Result

DroneVehicle weights
DroneVehicle logs

Future

The paper is under review, and this code repository is complete for rotating object detection.