Giter Site home page Giter Site logo

rizhaocai / pytorch_onnx_tensorrt Goto Github PK

View Code? Open in Web Editor NEW
242.0 242.0 57.0 184 KB

A tutorial about how to build a TensorRT Engine from a PyTorch Model with the help of ONNX

License: MIT License

Jupyter Notebook 41.46% Python 58.54%
onnx pytorch-onnx-tensorrt tensorrt

pytorch_onnx_tensorrt's Introduction

  • 👋 Hi, I’m @RizhaoCai, a PhD student in ROSE Lab@LNTU

  • 🌱 I’m currently doing research on face anti-spoofing and domain generalization.

  • 👀 Beyond, I’m interested in Basketball, Guitar, Writing, and reading history books. Meanwhile, as a research staff of ROSE Lab, I have been managing and leading projects with several industrial partenrs, which are about using object detection, recognition, image generation and simulation. I am also seeking oppotunities of using AI in industry.

  • Reach me if you are interested:
    📫 [email protected]
    🏠 https://rizhaocai.github.io

pytorch_onnx_tensorrt's People

Contributors

rizhaocai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

pytorch_onnx_tensorrt's Issues

AdaptiveAvgPool2d not supported by tensorRT OnnxParser

Env:
ubuntu 16.04
Python 3.6
Pytorch 1.3.0
tensorRT:7.0.11

Detail:
the adaptiveavgpool2d operation is not supported by onnxparser, so it could lead to the failure of transformation of onnx to tensorrt.
you could use
for error in range(parser.num_errors):
print(parser.get_error(error))
after the parser read onnx model

Does onnx convert tensorRT need a data set for calibration?

I saw some quantization tutorials stating that a small part of the training data set is required for quantitative calibration to determine the range of activation values for activation and weights. Why is there no such part in the code you provided, or is it unnecessary? Thank you

error: Failed to parse ONNX model.

Hello,thank you for your work. I get a error when run the demo,but i just use the model_128.onnx and did not make any
changes.
What is the reason and how to solve it ?

error:
Please check if the ONNX model is compatible '
AssertionError: Failed to parse ONNX model.

AttributeError: 'NoneType' object has no attribute 'create_execution_context'

Connected to pydev debugger (build 181.5540.34)
Loading ONNX file from path ./models/onnx/model.onnx...
Beginning ONNX file parsing
WARNING: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
Successfully casted down to INT32.
Completed parsing of ONNX file
Building an engine from file ./models/onnx/model.onnx; this may take a while...
[TensorRT] ERROR: Network must have at least one output
Failed to create the engine

AssertionError: Failed to parse ONNX model. Please check if the ONNX model is compatible

trtexec --onnx=/home/guohao02/PyTorch_ONNX_TensorRT/model_128.onnx --explicitBatch

[10/16/2021-13:56:29] [I] === Model Options ===
[10/16/2021-13:56:29] [I] Format: ONNX
[10/16/2021-13:56:29] [I] Model: /home/guohao02/PyTorch_ONNX_TensorRT/model_128.onnx
[10/16/2021-13:56:29] [I] Output:
[10/16/2021-13:56:29] [I] === Build Options ===
[10/16/2021-13:56:29] [I] Max batch: explicit
[10/16/2021-13:56:29] [I] Workspace: 16 MB
[10/16/2021-13:56:29] [I] minTiming: 1
[10/16/2021-13:56:29] [I] avgTiming: 8
[10/16/2021-13:56:29] [I] Precision: FP32
[10/16/2021-13:56:29] [I] Calibration: 
[10/16/2021-13:56:29] [I] Safe mode: Disabled
[10/16/2021-13:56:29] [I] Save engine: 
[10/16/2021-13:56:29] [I] Load engine: 
[10/16/2021-13:56:29] [I] Inputs format: fp32:CHW
[10/16/2021-13:56:29] [I] Outputs format: fp32:CHW
[10/16/2021-13:56:29] [I] Input build shapes: model
[10/16/2021-13:56:29] [I] === System Options ===
[10/16/2021-13:56:29] [I] Device: 0
[10/16/2021-13:56:29] [I] DLACore: 
[10/16/2021-13:56:29] [I] Plugins:
[10/16/2021-13:56:29] [I] === Inference Options ===
[10/16/2021-13:56:29] [I] Batch: Explicit
[10/16/2021-13:56:29] [I] Iterations: 10 (200 ms warm up)
[10/16/2021-13:56:29] [I] Duration: 10s
[10/16/2021-13:56:29] [I] Sleep time: 0ms
[10/16/2021-13:56:29] [I] Streams: 1
[10/16/2021-13:56:29] [I] Spin-wait: Disabled
[10/16/2021-13:56:29] [I] Multithreading: Enabled
[10/16/2021-13:56:29] [I] CUDA Graph: Disabled
[10/16/2021-13:56:29] [I] Skip inference: Disabled
[10/16/2021-13:56:29] [I] === Reporting Options ===
[10/16/2021-13:56:29] [I] Verbose: Disabled
[10/16/2021-13:56:29] [I] Averages: 10 inferences
[10/16/2021-13:56:29] [I] Percentile: 99
[10/16/2021-13:56:29] [I] Dump output: Disabled
[10/16/2021-13:56:29] [I] Profile: Disabled
[10/16/2021-13:56:29] [I] Export timing to JSON file: 
[10/16/2021-13:56:29] [I] Export profile to JSON file: 
[10/16/2021-13:56:29] [I] 
----------------------------------------------------------------
Input filename:   /home/guohao02/PyTorch_ONNX_TensorRT/model_128.onnx
ONNX IR version:  0.0.6
Opset version:    9
Producer name:    pytorch
Producer version: 1.9
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.6) than this parser was built against (0.0.3).
While parsing node number 0 [Conv]:
ERROR: ModelImporter.cpp:296 In function importModel:
[5] Assertion failed: tensors.count(input_name)
[10/16/2021-13:56:29] [E] Failed to parse onnx file
[10/16/2021-13:56:29] [E] Parsing model failed
[10/16/2021-13:56:29] [E] Engine could not be created
&&&& FAILED TensorRT.trtexec # trtexec --onnx=/home/guohao02/PyTorch_ONNX_TensorRT/model_128.onnx --explicitBatch

代码与 tensorrt 8.4 不兼容

我正在使用 tensorrt 8.4的环境运行代码,报错
'tensorrt.tensorrt.Builder' object has no attribute 'build_cuda_engine',int8_mod, fp16_mode, int8_mode

在新的tensorrt中,使用config 接管了 builder的各种设置。

Run batch size with inputs[1].host

Hi, everyone
I define batch_size_max = 4 and input size onnx= 2 to run batch_size = 2 for model trt
When i convert model that Error show "list index out of range" in inputs[1].host
That mean inputs only have one elements.
How to fix when i want to run batch size > 1

Int8 implementation

Hi! Any idea or resources on how to implement the Int8? The documentation from Nvidia is too minimal to get how it is supposed to work. They mention creating two objects, but who knows from which classes and how.

problems occured when executing "python trt_int8_demo.py "

Building an engine from file model_128.onnx; this may take a while...
Traceback (most recent call last):
File "trt_int8_demo.py", line 138, in
main()
File "trt_int8_demo.py", line 92, in main
engine_int8 = trt_helper.get_engine(batch_size,onnx_model_path,engine_model_path, fp16_mode=False, int8_mode=True, calibration_stream=calibration_stream, save_engine=True)
File "/data/zhangyl/PyTorch_ONNX_TensorRT/helpers/trt_helper.py", line 95, in get_engine
return build_engine(max_batch_size, save_engine)
File "/data/zhangyl/PyTorch_ONNX_TensorRT/helpers/trt_helper.py", line 76, in build_engine
engine = builder.build_cuda_engine(network)
TypeError: read_calibration_cache() missing 1 required positional argument: 'length'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.