mattherdelma/tensorrt-alpha

TensorRT-Alpha implements CUDA C accelerated deployment of more than 30 models, including but not limited to yolov8, yolov7, yolov6, yolov5, yolov4, yolov3, yolox, yolor and so on, the other 10 more CNN models are being sorted out, vision transformer related models will be updated in the future.CUDA is all you need, best wish!

TensorRT-Alpha

English | 简体中文

Introduce

This repository provides accelerated deployment cases of deep learning CV popular models, and cuda c accelerated methods for pre-processing and post-processing of mainstream models. Most of the model transformation process is torch->onnx->tensorrt. There are two ways to obtain onnx files:

• According to the network disk provided by this repository, download ONNX directly
• Follow the instructions provided in this repository to manually export ONNX from the relevant source code framework.

graph LR
    pytorch/tensorflow -->onnx-->tensorrt

Update

• 2023.01.01 🔥 update yolov3, yolov4, yolov5, yolov6
• 2023.01.04 🍅 update yolov7, yolox, yolor
• 2023.01.05 🎉 update u2net, libfacedetction
• 2023.01.08 🚀 The whole network is the first to support yolov8

Installation

platforms: windows and linux. The following environments have been tested：

ubuntu18.04

• cuda11.3
• cudnn8.2.0
• gcc7.5.0
• tensorrt8.4.2.4
• opencv3.x、4.x
• cmake3.10.2

windows10

• cuda11.3
• cudnn8.2.0
• visual studio 2017 and 2019
• tensorrt8.4.2.4
• opencv3.x、4.x

python dependent environment(optional）：

# install miniconda first
conda create -n tensorrt-alpha python==3.8 -y
conda activate tensorrt-alpha
git clone https://github.com/FeiYull/tensorrt-alpha
cd tensorrt-alpha
pip install -r requirements.txt  

Quick Start

ubuntu18.04

set your TensorRT_ROOT path:

git clone https://github.com/FeiYull/tensorrt-alpha
cd tensorrt-alpha/cmake
vim common.cmake
# set var TensorRT_ROOT to your path in line 20, eg:
# set(TensorRT_ROOT /root/TensorRT-8.4.2.4)

start to build project: For example:yolov7

windows10

waiting for update

Onnx

At present, more than 30 models have been implemented, and some onnx files of them are organized as follows:

model	weiyun	google driver
yolov3	weiyun	google driver
yolov4	weiyun	google driver
yolov5	weiyun	google driver
yolov6	weiyun	google driver
yolov7	weiyun	google driver
yolov8	weiyun	google driver
yolox	weiyun	google driver
yolor	weiyun	google driver
u2net	weiyun	google driver
libfacedet	weiyun	google driver
facemesh	weiyun	google driver
more...(🚀: I will be back soon!)

Visualization

some precision alignment renderings comparison:

yolov8n : Offical( left ) vs Ours( right )


yolov7-tiny : Offical( left ) vs Ours( right )


yolov5s : Offical( left ) vs Ours( right )


libfacedetction : Offical( left ) vs Ours( right topK:4000)

Reference

[0].https://github.com/NVIDIA/TensorRT
[1].https://github.com/onnx/onnx-tensorrt
[2].https://github.com/NVIDIA-AI-IOT/torch2trt
[3].https://github.com/shouxieai/tensorRT_Pro
[4].https://github.com/opencv/opencv_zoo