jkjung-avt/tensorrt_demos

TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet

tensorrt_demos's Issues

ERROR: ValueError: not enough values to unpack (expected 2, got 1)

I Have trained model with YOLO v3, I Have two files yolov3.weights and yolov3.cfg.
When I try to run yolov3_to_onnx.py with those files it throws these errors:

Traceback (most recent call last):
File "yolov3_to_onnx.py", line 833, in
main()
File "yolov3_to_onnx.py", line 791, in main
layer_configs = parser.parse_cfg_file(cfg_file_path)
File "yolov3_to_onnx.py", line 92, in parse_cfg_file
layer_dict, layer_name, remainder = self._next_layer(remainder)
File "yolov3_to_onnx.py", line 133, in _next_layer
layer_param_block, remainder = remainder.split('\n\n', 1)
ValueError: not enough values to unpack (expected 2, got 1)

Does tensorrt 4x support converting yolov3 model to onnx?

Hi,
Does tensorrt 4x support converting yolov3 model to onnx?

I have a jetson tx2 development board. My systems:
Ubuntu 16.04
TensorRT 4x
Cuda 9
CuDnn 7

I want to convert my special yolov3 model conveting to onnx model then convert to yolov3.engine.
But When I tried, I got some mistake.
Does tensorrt 4x support converting yolov3 model to onnx then yolov3.engine? and Which onnx version should I install with pip?

Thanks for your help..

ERROR: nvcc not found with tensorflow-1.15 and jetpack-4.3

Hi,
I used your wheel TensorFlow 1.15 file for install and correctly is install. but when I run ./install.sh in ssd folder, I get this error:

ERROR: nvcc not found
** Patch 'graphsurgeon.py' in TensorRT
patching file /usr/lib/python3.6/dist-packages/graphsurgeon/node_manipulation.py

** Making symbolic link of libflattenconcat.so
** Installation done

I comment this lines in install_pycuda.sh

if ! which nvcc > /dev/null; then
echo "ERROR: nvcc not found"
exit
fi

then I run the

python3 trt_ssd.py --model ssd_mobilenet_v2_egohands --image --filename hand.jpg

The output is ok and correctly I get the result but I get this error during the run the above python command :

[TensorRT] ERROR: Could not register plugin creator: FlattenConcat_TRT in namespace:

I get about 20 FPS when I run trt_ssd.py for ssd_mobilenet_v2_egohands model and I get about 2 27 FPS when I run trt_ssd_async.py, but I get more FPS with Tensorflow-1.12.2 Jetpack-4.2.2, even I get 30 FPS with run python3 trt_ssd.py --model ssd_mobilenet_v2_egohands --image --filename hand.jpg and 35 FPS with trt_ssd_async.py.

what's interesting that when I run the trt_ssd_async.py and trt_ssd.py with Tensorflow0-1.12.2 jetpack-4.2.2, I get almost unit FPS, that's mean, The FPS is changed between 29-32 FPS, but with Tensorflow-1.15.0 jetpack 4.3, The FPS is changed between 13-23 FPS for trt_ssd.py and even trt_ssd_aysnc.py, I don't know why these changes are so great, As my input image the same.

Possible to use other caffeemodel ?

Hi, first, thank you for your amazing work, got 15 ms of inference time on my nano, really suprised about it. Was wondering if I could use any other caffeemodel than the one you tell to use ? I would like to recognize stuff like person, cycle or suitcase for exemple.

Best regards !

failed to open camera....

is any fig setting wrong?

$ python3 camera_trt_googlenet.py --usb --vid 0 --width 1280 --height 720
Called with args:
Namespace(crop_center=False, image_height=720, image_width=1280, rtsp_latency=200, rtsp_uri=None, use_rtsp=False, use_usb=True, video_dev=0)
Failed to open camera!

I changed the following one, then it's working fine thought.

def open_cam_usb(dev, width, height):
    gst_str = ('v4l2src device=/dev/video{} ! '
               'video/x-raw, width=(int){}, height=(int){} ! '
               'videoconvert ! appsink').format(dev, width, height)
    # return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)
    return cv2.VideoCapture(0)

slow inference on jetson tx2

i have tested this demo on a jetson tx2 device and inference speed is at 22 fps. i expected better performance on a tx2 than a jetson nano. do you have any insights for achieving better results? and what are the expected speeds on a tx2?

anyone tried it on a tx2 and what were the results?

thanks

AttributeError: 'NoneType' object has no attribute 'serialize'

Hi~
I use https://github.com/ultralytics/yolov3 to trained a 1 class model. when use onnx_to tensorrt.py, meet below error.

ubuntu:~/tensorrt_demos/yolov3_onnx$ python3 onnx_to_tensorrt.py
Loading ONNX file from path yolov3-416.onnx...
Beginning ONNX file parsing
Completed parsing of ONNX file
Building an engine; this may take a while...
[TensorRT] ERROR: ../builder/cudnnBuilderWeightConverters.cpp (555) - Misc Error in operator(): 1 (Weights are outside of fp16 range. A possible fix is to retrain the model with regularization to bring the magnitude of the weights down.)
[TensorRT] ERROR: ../builder/cudnnBuilderWeightConverters.cpp (555) - Misc Error in operator(): 1 (Weights are outside of fp16 range. A possible fix is to retrain the model with regularization to bring the magnitude of the weights down.)
Completed creating engine
Traceback (most recent call last):
File "onnx_to_tensorrt.py", line 119, in
main()
File "onnx_to_tensorrt.py", line 115, in main
_ = build_engine(onnx_file_path, engine_file_path, args.verbose)
File "onnx_to_tensorrt.py", line 99, in build_engine
f.write(engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'

yolov3 - onnx library issue.

unable to build onnx on jetson nano.

ssdlite_mobilenet_v2_coco

Hi
I want to convert ssdlite_mobilenet_v2_coco with ./build_engines.sh, I get this error:
For ssd_mobilenet_v2_coco and athor models corectly convert.
AND I set num_classes = 91

python3: nmsPlugin.cpp:54: virtual nvinfer1::Dims nvinfer1::plugin::DetectionOutput::getOutputDimensions(int, const nvinfer1::Dims*, int): Assertion `nbInputDims == 3' failed.
./build_engines.sh: line 5:  7395 Aborted                 (core dumped) python3 build_engines.py ${model}

ssd_inception_v2_coco custom trained model with my own dataset.

I tried with ssd_inception_v2_coco custom trained model with my own dataset and the following error occurred: -

=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
input: "Cast"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 418
      }
      dim {
        size: 418
      }
    }
  }
}
]
=========================================

No. nodes: 721
UFF Output written to /home/sys-admin/Downloads/Projects/tensorrt_demos/ssd/tmp_inception_v2_coco.uff
UFF Text Output written to /home/sys-admin/Downloads/Projects/tensorrt_demos/ssd/tmp_inception_v2_coco.pbtxt
[TensorRT] ERROR: UffParser: Validator error: Cast: Unsupported operation _Cast
[TensorRT] ERROR: Network must have at least one output
Traceback (most recent call last):
  File "build_engine.py", line 216, in <module>
    main()
  File "build_engine.py", line 210, in main
    buf = engine.serialize()
AttributeError: 'NoneType' object has no attribute 'serialize'

Define New Activation Layer

Hi,
I want to use mobilenet_v3 module in my network, and in this module is used HardSwitch activation such this:

def relu6(x):
    return tf.nn.relu(x)

def hard_swish(x):
    return x * tf.nn.relu(x + 3.0) / 6.0

def return_activation(x, nl):
    if nl == 'HS':
        x = Activation(hard_swish)(x)
    if nl == 'RE':
        x = Activation(relu6)(x)
    return x

If I want to convert to UFF and TensorRT, I should use a plugin for these layers? If so, How do I do for writing a custom plugin?

Built TensorRT UFF Faster RCNN

Hi,
Did you test and experience with build TensorRT UFF Faster RNN?

The problem of yolov3_onnx

Followed the Demo #4: YOLOv3
when i execute this command at step3
python3 yolov3_to_onnx.py --model yolov3-tiny-416
I got the error below figure

I already update my protobuf to a recent version
Do you have any suggestions ?

yolov3_onnx, onnx_to_tensorrt.py <tensorrt.tensorrt.Builder object at 0x7f5d678a78>, 0 bug

Hi
I tried this steps.

$ cd ${HOME}/project/tensorrt_demos/yolov3_onnx $ ./download_yolov3.sh $ python3 yolov3_to_onnx.py --model yolov3-tiny-416 $ python3 onnx_to_tensorrt.py --model yolov3-tiny-416

"python3 onnx_to_tensorrt.py --model yolov3-tiny-416" in this step I get the error:

Traceback (most recent call last): File "onnx_to_tensorrt.py", line 119, in <module> main() File "onnx_to_tensorrt.py", line 115, in main _ = build_engine(onnx_file_path, engine_file_path, args.verbose) File "onnx_to_tensorrt.py", line 69, in build_engine with trt.Builder(TRT_LOGGER) as builder, builder.create_network(EXPLICIT_BATCH) as network, trt.OnnxParser(network, TRT_LOGGER) as parser: TypeError: create_network(): incompatible function arguments. The following argument types are supported: 1. (self: tensorrt.tensorrt.Builder) -> tensorrt.tensorrt.INetworkDefinition Invoked with: <tensorrt.tensorrt.Builder object at 0x7f5d678a78>, 0

and I encountered this output during this command. I don't know if there is a problem " python3 yolov3_to_onnx.py --model yolov3-tiny-416"

Layer of type yolo not supported, skipping ONNX node generation. Layer of type yolo not supported, skipping ONNX node generation. graph yolov3-tiny-416 ( %000_net[FLOAT, 1x3x416x416] ) initializers ( %001_convolutional_bn_scale[FLOAT, 16] %001_convolutional_bn_bias[FLOAT, 16] %001_convolutional_bn_mean[FLOAT, 16] %001_convolutional_bn_var[FLOAT, 16] %001_convolutional_conv_weights[FLOAT, 16x3x3x3] %003_convolutional_bn_scale[FLOAT, 32] %003_convolutional_bn_bias[FLOAT, 32] %003_convolutional_bn_mean[FLOAT, 32] %003_convolutional_bn_var[FLOAT, 32] %003_convolutional_conv_weights[FLOAT, 32x16x3x3] %005_convolutional_bn_scale[FLOAT, 64] %005_convolutional_bn_bias[FLOAT, 64] %005_convolutional_bn_mean[FLOAT, 64] %005_convolutional_bn_var[FLOAT, 64] %005_convolutional_conv_weights[FLOAT, 64x32x3x3] %007_convolutional_bn_scale[FLOAT, 128] %007_convolutional_bn_bias[FLOAT, 128] %007_convolutional_bn_mean[FLOAT, 128] %007_convolutional_bn_var[FLOAT, 128] %007_convolutional_conv_weights[FLOAT, 128x64x3x3] %009_convolutional_bn_scale[FLOAT, 256] %009_convolutional_bn_bias[FLOAT, 256]

Can you help me pls. I couldn't convert my yolov3 model to onnx_model.
I plan to use this onnx model to convert it to TRT C++ API

test on file problem

when i test on video ( .mp4 file),it did not stop when video ended,it continued rerun video again

UUF Convertor

Hi
I have some question:
1- What's the UUF converter? why we use this converter before converting to the TensorRT engine?
As you know we can convert the Tensorflow forzen graph (.pb) model to TernsorRT engine model directly and no need for UFF convertor, with TensorRT API:

trt_graph = trt.create_inference_graph(
    input_graph_def,
    outputs,
    max_batch_size,
    max_workspace_size_bytes,
    minimum_segment_size, 
    precision_mode="FP16") 

My quesion is that what's advantage of converting .pb (Tensorflow) >> . bin (UFF) >> .pb (TensorRT) instead .pb (Tensorflow) >> .pb (TensorRT)?
2- The TensorRT library is optimization package for optimizing the graph, so the UFF library is optimizer also?
3- what's host memory and cuda memory:

           size = trt.volume(self.engine.get_binding_shape(binding)) * self.engine.max_batch_size
            host_mem = cuda.pagelocked_empty(size, np.float32) 
            cuda_mem = cuda.mem_alloc(host_mem.nbytes)  

as far as I know, binding is the each of graph layers, and we get the size of each graph layers assigne the memory for each layers, but I don't what's host_mem?

mtcnn optimization is input dependent

Your optimization of mtcnn is specifically designed for 720p input image. While other size may be scaled or padded to fit into 720p. The failure case may be an input of portrait rather than landscape. In this case, images are heavily squeezed and make it impossible to detect small faces. Is there any plan to make it input independent?

Uff parameters for INT_8 precision?

Hey @jkjung-avt , so how do we create an .uff model with int8 precision forr SSD models? In build_engine.py file-->https://github.com/jkjung-avt/tensorrt_demos/blob/master/ssd/build_engine.py

It would be great if you could point out the relevant changes to be made.

Thanks

my finetuned model is much slower

Hi JK,

I tried the model in your repo in Nano, the ssd_mobilenet_v1_egohands, the fps is 20. Then I trained/fine tuned the model with my own data from the same model (from model zoon) and with the same config file (from your hand detection repo). After converting to TRT in nano, the fps is only 12.
Do you have any ideas about what factors that will affect the speed?

more Facial landmark in mtcnn

@jkjung-avt ,
Thanks for your work.

how to get 68 points facial landmark (as like dlib) using MTCNN.
currently, it's giving only 5 facial points.
Since I want to do head pose estimation I am need of the other facial points

please advise.

Could not install ONNX on jetson nano

when I execute the below command I am getting below error,

sudo pip3 install onnx==1.4.1

sudo pip3 install onnx==1.4.1
WARNING: The directory '/home/svnano/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting onnx==1.4.1
Downloading onnx-1.4.1.tar.gz (2.9 MB)
|████████████████████████████████| 2.9 MB 6.4 MB/s
Requirement already satisfied: protobuf in /usr/local/lib/python3.6/dist-packages (from onnx==1.4.1) (3.8.0)
Requirement already satisfied: numpy in /home/svnano/.local/lib/python3.6/site-packages (from onnx==1.4.1) (1.18.1)
Requirement already satisfied: six in /home/svnano/.local/lib/python3.6/site-packages (from onnx==1.4.1) (1.14.0)
Requirement already satisfied: typing>=3.6.4 in /usr/local/lib/python3.6/dist-packages (from onnx==1.4.1) (3.7.4.1)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.6/dist-packages (from onnx==1.4.1) (3.7.4.1)
Requirement already satisfied: setuptools in /home/svnano/.local/lib/python3.6/site-packages (from protobuf->onnx==1.4.1) (45.1.0)
Building wheels for collected packages: onnx
Building wheel for onnx (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-4t58g49u/onnx/setup.py'"'"'; file='"'"'/tmp/pip-install-4t58g49u/onnx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-2qq9qqta
cwd: /tmp/pip-install-4t58g49u/onnx/
Complete output (64 lines):
fatal: not a git repository (or any of the parent directories): .git
running bdist_wheel
running build
running build_py
running create_version
running cmake_build
-- Build type not set - defaulting to Release
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:217 (message):
Protobuf compiler not found
Call Stack (most recent call first):
CMakeLists.txt:248 (relative_protobuf_generate_cpp)

-- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-4t58g49u/onnx/.setuptools-cmake-build/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-4t58g49u/onnx/setup.py", line 328, in
'backend-test-tools = onnx.backend.test.cmd_tools:main',
File "/home/svnano/.local/lib/python3.6/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 204, in run
self.run_command('build')
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-install-4t58g49u/onnx/setup.py", line 203, in run
self.run_command('cmake_build')
File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-install-4t58g49u/onnx/setup.py", line 190, in run
subprocess.check_call(cmake_args)
File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/cmake', '-DPYTHON_INCLUDE_DIR=/usr/include/python3.6m', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-DBUILD_ONNX_PYTHON=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cpython-36m-aarch64-linux-gnu.so', '/tmp/pip-install-4t58g49u/onnx']' returned non-zero exit status 1.

ERROR: Failed building wheel for onnx
Running setup.py clean for onnx
Failed to build onnx
Installing collected packages: onnx
Running setup.py install for onnx ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-4t58g49u/onnx/setup.py'"'"'; file='"'"'/tmp/pip-install-4t58g49u/onnx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-tqgfr9k6/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6/onnx
cwd: /tmp/pip-install-4t58g49u/onnx/
Complete output (51 lines):
fatal: not a git repository (or any of the parent directories): .git
running install
running build
running build_py
running create_version
running cmake_build
CMake Error at CMakeLists.txt:217 (message):
Protobuf compiler not found
Call Stack (most recent call first):
CMakeLists.txt:248 (relative_protobuf_generate_cpp)

-- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-4t58g49u/onnx/.setuptools-cmake-build/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-4t58g49u/onnx/setup.py", line 328, in <module>
    'backend-test-tools = onnx.backend.test.cmd_tools:main',
  File "/home/svnano/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/svnano/.local/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/usr/lib/python3.6/distutils/command/install.py", line 589, in run
    self.run_command('build')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-4t58g49u/onnx/setup.py", line 203, in run
    self.run_command('cmake_build')
  File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-4t58g49u/onnx/setup.py", line 190, in run
    subprocess.check_call(cmake_args)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/cmake', '-DPYTHON_INCLUDE_DIR=/usr/include/python3.6m', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-DBUILD_ONNX_PYTHON=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cpython-36m-aarch64-linux-gnu.so', '/tmp/pip-install-4t58g49u/onnx']' returned non-zero exit status 1.
----------------------------------------

ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-4t58g49u/onnx/setup.py'"'"'; file='"'"'/tmp/pip-install-4t58g49u/onnx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-tqgfr9k6/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6/onnx Check the logs for full command output.

please advise

Install/Upgrade TensorRT6 to 7

Hi,
I want to install/upgrade TensorRT6 to 7 on jetson nano with jetpack-4.3.
How do I do?

FasterRCNN possible? [Potential Future Dev]

Hello Jk!

Like usual, awesome work on this repo. I am always learning tons from the work you create.

I have tried to replicate what you did with SSD obj det in this repo but for FasterRCNN... but am not having much luck.. Have you tried to use any Tensorflow object detection API Faster RCNN models with the new Python TensorRT Api?

Please let me know if you were successful or if there is a plan to include this in the next addition to the repo.

Thank you for all your hard work 😄

How to inference own yolo model ?

I have my own yolov3 model
I want to using tensorrt inference on jetson nano
How can i do ?
Appreciate

Running MTCNN with tensorrt

I have some confusion here.
I run MTCNN without tensorrt by demo in here : jetson_nano_demo
and compare with your code for MTCNN with tensorrt
I realize when use tensorrt FPS is nearly same

build issue when using my own model

Thank you for creating this repo. It's great!

I met an issue when I try to replace your model 'ssd_mobilenet_v1_egohands.pb' with my own trained model. I think the two model should be the same as I also download from TF detection zoon and just retrained with my own data.

When doing the build_engin.sh, I got the following error:
Do you have any ideas about what i need to modify? Many thanks.

amvi@nvidia-nano:~/zhen/tensorrt_demos/ssd$ ./build_engines.sh

• for model in ssd_mobilenet_v1_egohands
• python3 build_engine.py ssd_mobilenet_v1_egohands
  [TensorRT] INFO: Plugin Creator registration succeeded - GridAnchor_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - NMS_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - Reorg_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - Region_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - Clip_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - LReLU_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - PriorBox_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - Normalize_TRT
  [TensorRT] INFO: Plugin Creator registration succeeded - RPROI_TRT
  WARNING:tensorflow:From /usr/lib/python3.6/dist-packages/graphsurgeon/StaticGraph.py:123: FastGFile.init (from tensorflow.python.platform.gfile) is deprecated and will be removed in a future version.
  Instructions for updating:
  Use tf.gfile.GFile.
  WARNING: To create TensorRT plugin nodes, please use the create_plugin_node function instead.
  UFF Version 0.5.5
  === Automatically deduced input nodes ===
  [name: "Input"
  op: "Placeholder"
  attr {
  key: "dtype"
  value {
  type: DT_FLOAT
  }
  }
  attr {
  key: "shape"
  value {
  shape {
  dim {
  size: 1
  }
  dim {
  size: 3
  }
  dim {
  size: 300
  }
  dim {
  size: 300
  }
  }
  }
  }
  ]
  =========================================

Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_loc as custom op: FlattenConcat_TRT
Warning: No conversion function registered for layer: GridAnchor_TRT yet.
Converting MultipleGridAnchorGenerator as custom op: GridAnchor_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_conf as custom op: FlattenConcat_TRT
No. nodes: 450
UFF Output written to /home/camvi/zhen/tensorrt_demos/ssd/tmp_v1_egohands.uff
UFF Text Output written to /home/camvi/zhen/tensorrt_demos/ssd/tmp_v1_egohands.pbtxt
[TensorRT] INFO: UFFParser: parsing Input
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_0/weights
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_0/Conv2D
[TensorRT] INFO: UFFParser: Convolution: add Padding Layer to support asymmetric padding
[TensorRT] INFO: UFFParser: Convolution: Left: 0
[TensorRT] INFO: UFFParser: Convolution: Right: 1
[TensorRT] INFO: UFFParser: Convolution: Top: 0
[TensorRT] INFO: UFFParser: Convolution: Bottom: 1
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_0/BatchNorm/gamma
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_0/BatchNorm/beta
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_0/BatchNorm/moving_mean
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_0/BatchNorm/moving_variance
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_0/BatchNorm/FusedBatchNorm
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_0/Relu6
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 9) to 0.0472441
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 11) to 0.0472441
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_1_depthwise/depthwise_weights
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_1_depthwise/depthwise
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_1_depthwise/BatchNorm/gamma
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_1_depthwise/BatchNorm/beta
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_1_depthwise/BatchNorm/moving_mean
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_1_depthwise/BatchNorm/moving_variance
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_1_depthwise/BatchNorm/FusedBatchNorm
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_1_depthwise/Relu6
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 19) to 0.0472441
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 21) to 0.0472441
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_1_pointwise/weights
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_1_pointwise/Conv2D
....
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_10_depthwise/BatchNorm/moving_variance
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_10_depthwise/BatchNorm/FusedBatchNorm
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_10_depthwise/Relu6
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 201) to 0.0472441
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 203) to 0.0472441
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_10_pointwise/weights
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_10_pointwise/Conv2D
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_10_pointwise/BatchNorm/gamma
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_10_pointwise/BatchNorm/beta
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_10_pointwise/BatchNorm/moving_mean
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_10_pointwise/BatchNorm/moving_variance
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_10_pointwise/BatchNorm/FusedBatchNorm
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_10_pointwise/Relu6
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 211) to 0.0472441
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 213) to 0.0472441
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_depthwise/depthwise_weights
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_11_depthwise/depthwise
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_depthwise/BatchNorm/gamma
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_depthwise/BatchNorm/beta
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_depthwise/BatchNorm/moving_mean
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_depthwise/BatchNorm/moving_variance
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_11_depthwise/BatchNorm/FusedBatchNorm
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_11_depthwise/Relu6
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 221) to 0.0472441
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 223) to 0.0472441
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_pointwise/weights
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_11_pointwise/Conv2D
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_pointwise/BatchNorm/gamma
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_pointwise/BatchNorm/beta
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_pointwise/BatchNorm/moving_mean
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/Conv2d_11_pointwise/BatchNorm/moving_variance
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_11_pointwise/BatchNorm/FusedBatchNorm
[TensorRT] INFO: UFFParser: parsing FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_11_pointwise/Relu6
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 231) to 0.0472441
[TensorRT] INFO: Setting Dynamic Range for (Unnamed ITensor* 233) to 0.0472441
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/BoxEncodingPredictor/weights
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/BoxEncodingPredictor/Conv2D
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/BoxEncodingPredictor/biases
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/BoxEncodingPredictor/BiasAdd
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/Shape
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/strided_slice/stack
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/strided_slice/stack_1
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/strided_slice/stack_2
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/strided_slice
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/Reshape/shape/1
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/Reshape/shape/2
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/Reshape/shape/3
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/Reshape/shape
[TensorRT] INFO: UFFParser: parsing BoxPredictor_0/Reshape
[TensorRT] ERROR: UFFParser: Parser error: BoxPredictor_0/Reshape: Reshape: -1 dimension specified more than 1 time
[TensorRT] ERROR: Network must have at least one output
Traceback (most recent call last):
File "build_engine.py", line 218, in
main()
File "build_engine.py", line 212, in main
buf = engine.serialize()
AttributeError: 'NoneType' object has no attribute 'serialize'

Why config.py different from ssd_mobilenet_v1_coco and ssd_mobilenet_v2_coco?

When compare config.py between ssd_mobilenet_v1_coco and ssd_mobilenet_v2_coco ,
There some different context
like NMS input order and V1MultipleGridAnchorGenerator/Concatenate": concat_priorbox、V2 "Concatenate": concat_priorbox,.

But why should use this? How to know these different code?

Compatibility of TensorRT optimized engine with deepstream-app

Thanks to the Demo #3: SSD, I've successfully made a TensorRT optimized 'engine' of SSD.
However, I got the following error message when I tried to use the engine with NVIDIA's deepstream-app (the deepstream-app can dedicate the TensorRT engine to be used in its pipeline) :

deepstream-app: nvdsiplugin_ssd.cpp:72: FlattenConcat::FlattenConcat(const void, size_t): Assertion `mConcatAxisID == 1 || mConcatAxisID == 2 || mConcatAxisID == 3' failed.
Aborted (core dumped)*

I'm not familiar with the deepstream-app plugin structure and hope that any expert can explain what is the main cause of this problem and what I should do to use any TensorRT optimized engine in the deepstream pipeline.
(Before, I expected that the deepstream can perform inference by a user dedicated TensorRT optimized engine without any coding or plugin library building.. )

Unsupported operation Cast in the ssd models

Hi,
your codes run very well, but when I convert the ssdmobilenetv1/2 from tensorflow model zoo, I got this error.
Because this layer unsupported in the TensorRT, I want to convert the .pb model to .onnx model and then convert .onnx model to .uff and .bin, but I got some errors when converting to onnx model.

`NOTE: UFF has been tested with TensorFlow 1.12.0. Other versions are not guaranteed to work
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.3
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
input: "Cast"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
dim {
size: 1
}
dim {
size: 3
}
dim {
size: 300
}
dim {
size: 300
}
}
}
}
]

Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
WARNING:tensorflow:From /usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:179: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_conf as custom op: FlattenConcat_TRT
Warning: No conversion function registered for layer: Cast yet.
Converting Cast as custom op: Cast
Warning: No conversion function registered for layer: GridAnchor_TRT yet.
Converting GridAnchor as custom op: GridAnchor_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_loc as custom op: FlattenConcat_TRT
No. nodes: 451
UFF Output written to tmp.uff
[TensorRT] ERROR: UffParser: Validator error: Cast: Unsupported operation _Cast
[TensorRT] ERROR: Network must have at least one output
Traceback (most recent call last):
File "main.py", line 44, in
buf = engine.serialize()
AttributeError: 'NoneType' object has no attribute 'serialize'`

ssd_inception_v2_coco(custom trained model)

@jkjung-avt - I am trying to convert - ssd_inception_v2_coco(custom trained model) but during runtime its throwing following error.

I am using:-
tensorflow==1.14.0
jetson nano==Jetpack 4.3 [L4T 32.3.1]

for reference:

No. nodes: 720
UFF Output written to /home/sys-admin/Downloads/Projects/tensorrt_demos/ssd/tmp_inception_v2_coco.uff
UFF Text Output written to /home/sys-admin/Downloads/Projects/tensorrt_demos/ssd/tmp_inception_v2_coco.pbtxt
[libprotobuf FATAL /externals/protobuf/aarch64/10.0/include/google/protobuf/repeated_field.h:1408] CHECK failed: (index) < (current_size_):
Traceback (most recent call last):
File "build_engine.py", line 219, in
main()
File "build_engine.py", line 210, in main
parser.parse(spec['tmp_uff'], network)
RuntimeError: CHECK failed: (index) < (current_size_):

Issue while trying with ssd_inception_v2_coco from tensorflow zoo

while converting frozen_inference_graph.pb file to uff format, I downloaded from TensorFlow zoo I got key error: 'image_tensor', can you tell why i am getting this error.

for reference:

UFF Version 0.6.5
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
input: "image_tensor:0"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 300
      }
      dim {
        size: 300
      }
    }
  }
}
]

update README

I think you should check README and update it
in MTCNN section,it should be in MTCNN folder instead of GOOGLENET

jetson tx2 yolov3_to_onnx.py bug

I run the script yolov3_to_onnx.py on jetsontx2 with jetpack4.2.1 onnx=1.4.1 and it still show the above code without change.

train my caffemodel error

I want to train MobileNet-YOLOv3 caffemodel so that I can transplant it on TX2.
But when I use MobileNet-YOLOv3 to train my dataset, error occurred.

Opened lmdb /home/gzz/data/VOCdevkit/MyDataSet/lmdb/MyDataSet_trainval_lmdb/
I1018 09:27:19.771530 17056 annotated_data_layer.cpp:78] output data size: 4,3,608,608
[libprotobuf ERROR google/protobuf/text_format.cc:274] Error parsing text-format caffe.YoloSegLabel: 4:15: Message type "caffe.YoloSegItem" has no field named "display_name".
F1018 09:27:19.771678 17056 annotated_data_layer.cpp:134] Check failed: ReadProtoFromTextFile(label_map_file_, &label_map_) Failed to read label map file.

How should I do?

trt_outputs = [output.reshape(shape) for output, shape ValueError: cannot reshape array of size 20577 into shape (1,255,19,19)

(base) tim@tim-System-Product-Name:~/workspace/tensorrt_demos$ python trt_yolov3.py --model yolov3-608 --usb --vid 0
[TensorRT] INFO: Glob Size is 128825344 bytes.
[TensorRT] INFO: Added linear block of size 47316992
[TensorRT] INFO: Added linear block of size 23658496
[TensorRT] INFO: Added linear block of size 11829248
[TensorRT] INFO: Added linear block of size 2957312
[TensorRT] INFO: Added linear block of size 1478656
[TensorRT] INFO: Added linear block of size 739328
[TensorRT] INFO: Found Creator ResizeNearest
[TensorRT] INFO: Found Creator ResizeNearest
[TensorRT] INFO: Deserialize required 1140534 microseconds.
Traceback (most recent call last):
File "trt_yolov3.py", line 96, in
main()
File "trt_yolov3.py", line 88, in main
loop_and_detect(cam, trt_yolov3, conf_th=0.3, vis=vis)
File "trt_yolov3.py", line 56, in loop_and_detect
boxes, confs, clss = trt_yolov3.detect(img, conf_th)
File "/home/tim/workspace/tensorrt_demos/utils/yolov3.py", line 473, in detect
in zip(trt_outputs, self.output_shapes)]
File "/home/tim/workspace/tensorrt_demos/utils/yolov3.py", line 472, in
trt_outputs = [output.reshape(shape) for output, shape
ValueError: cannot reshape array of size 20577 into shape (1,255,19,19)

Mtcnn don't work?

Yolov3 Performance on Xavier Jetpack 4.2.2 TensorRT5

Hi Jk!

What an update you made! Just wanted to let you know that there is no issues with the yolo models... I just want to give you some more numbers all tested on the Xavier.

Yolo-288 (FP16) = ~31 FPS
Yolo-416 (FP16) = ~20 FPS
Yolo-608 (FP16) = ~12 FPS

Thank you for all your hard work and awesome documentation.

Cheers 😄

TF-TRT vs UFF-TensorRT

Hi,
I found that we can optimize the Tensorflow model in several ways. If I am mistaken, please tell me.

1- Using TF-TRT, This API developer by tensorflow and integreted TensoRT to Tensorflow and this API called as :
from tensorflow.python.compiler.tensorrt import trt_convert as trt
This API can be applied to any tensorflow models (new and old version models) without any converting error, because If this API don't support any new layers, don't consider these layers for TensorRT engines and these layers remain for Tensorflow engine and run on Tensorflow. right?

2- Using TensorRT, This API by developed by NVIDA and is independent of Tenorflow library (Not integrated to Tensorflow), and this API called as:
import tensorrt as trt
If we want to use this api, first, we must converting the tensorflow graph to UFF using uff-convertor and then parse the UFF graph to this API.
In this case, If the Tensorflow graph have unsupported layers we must use plugin or custom code for these layers, right?

3- I don't know, when we work with Tensorflow models, Why we use UFF converter then TensorRT, we can use directly TF-TRT API, right? If so, Are you tested the Tensorflow optimization model from these two method to get same performance? what's advantage of this UFF converter method?

I have some question about the two cases above:
4- I convert the ssd_mobilenet_v2 using two cases, In the case 1, I achieve slight improvement in speed but in the case 2, I achieve more improvement, why?
My opinion is that, In the case 1, The API only consider converting the precision (FP32 to FP16) and merging the possible layers together, But in the case 2, the graph is clean by UFF such as remove any redundant nodes like Asserts and Identity and then converted to tensorrt graph, right?

5- when we convert the trained model files like .ckpt and .meta, ... to frozen inference graph(.pb file), These layers don't remove from graph? only loss states and optimizer states , ... are removed?

PyCuda logic error on jetson nano with latest image

I have carefully followed your instruction and I can able to run the python yolo/ssd sample code.

But When I integrate the same code into my multi-threaded application I am getting the below error.

yolov3.py", line 282, in allocate_buffers

stream = cuda.Stream()
pycuda._driver.LogicError: explicit_context_dependent failed: invalid device context - no currently active context?

please advise

Unable to get tegra-cam-caffe.py tegra-cam.py？

I read your blog 'How to Capture Camera Video and Do Caffe Inferencing with Python on Jetson TX2' and 'How to Capture and Display Camera Video with Python on Jetson TX2 '
These two files cannot be obtained. I don't want to go through the firewall. Can you send me files by email? I suggest you place a new link without going through the firewall. Thanks!

slow with batch size > 1

Hi,
I set max_batch_size = 3 and I want to speed up model with 3 input image as parallel instead serial,
But I converted the model with batch_size = 3 correctly, when I run trt_ssd.py, I achieve this results:
for 1 batch_size : process time is 0.002 sec with 1080 TI,
for 3 batch_size : process time is 0.006 sec with 1080 TI,
That means the system process as serial not parallel, why?

ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco

Hi,
Have you experience the converting the ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco to uff and bin files?

the video stream delays for 300ms

hello, I run this code on my jetson nano for a live stream detect but the video has about 300ms delay, and the fps for me is about 10 with the trt_ssd.py. Do you have there situations, Thanks. I use the jetpack4.2 and tensorrt5.0.6.3,

How to inference own yolo model ?

@jkjung-avt
I try your thoughts in this Disqus post , and put my custom yolo weight & cfg in /yolov3_onnx.
Then excute python3 yolov3_to_onnx.py --model yolov3-416
I got the error below the figure

Can you give me some advice ?
Thanks you a lot ...

Dynamic Batch Sizes

Hi @jkjung-avt - awesome work with this repo. I think this is the most useful demo I've seen on the nano so far, and your work with the async function is really well done.

I'm wondering if there's an easy way to rebuild the SSD detector engines to take a batch size of (2,3,300,300), or even better to take a dynamic batch size input?

GStreamer: unable to start pipeline!

I read your blog"How to Capture and Display Camera Video with Python on Jetson TX2"
run command error!
"python3 tegra-cam.py --usb --vid 1 --width 1280 --height 720"

Called with args:
Namespace(image_height=720, image_width=1280, rtsp_latency=200, rtsp_uri=None, use_rtsp=False, use_usb=True, video_dev=1)
OpenCV version: 3.4.0
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 890
VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:

/home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:890: error: (-2) GStreamer: unable to start pipeline
in function cvCaptureFromCAM_GStreamer

Failed to open camera!

build ssd engine failed

[TensorRT] ERROR: Could not register plugin creator: FlattenConcat_TRT in namespace:
WARNING: To create TensorRT plugin nodes, please use the create_plugin_node function instead.
NOTE: UFF has been tested with TensorFlow 1.14.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.5
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
attr {
key: "shape"
value {
shape {
dim {
size: 1
}
dim {
size: 3
}
dim {
size: 300
}
dim {
size: 300
}
}
}
}
]

Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
Warning: No conversion function registered for layer: FlattenConcat_TRT yet.
Converting concat_box_conf as custom op: FlattenConcat_TRT
Traceback (most recent call last):
File "build_engine.py", line 204, in
main()
File "build_engine.py", line 187, in main
debug_mode=DEBUG_UFF)
File "/usr/local/lib/python3.6/site-packages/uff/converters/tensorflow/conversion_helpers.py", line 178, in from_tensorflow
debug_mode=debug_mode)
File "/usr/local/lib/python3.6/site-packages/uff/converters/tensorflow/converter.py", line 94, in convert_tf2uff_graph
uff_graph, input_replacements, debug_mode=debug_mode)
File "/usr/local/lib/python3.6/site-packages/uff/converters/tensorflow/converter.py", line 79, in convert_tf2uff_node
op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes, debug_mode=debug_mode)
File "/usr/local/lib/python3.6/site-packages/uff/converters/tensorflow/converter.py", line 47, in convert_layer
return cls.registry_[op](name, tf_node, inputs, uff_graph, **kwargs)
File "/usr/local/lib/python3.6/site-packages/uff/converters/tensorflow/converter_functions.py", line 21, in convert_placeholder
dtype = tf2uff.convert_tf2numpy_dtype(tf_node.attr['dtype'].type)
File "/usr/local/lib/python3.6/site-packages/uff/converters/tensorflow/converter.py", line 103, in convert_tf2numpy_dtype
return tf.as_dtype(dtype).as_numpy_dtype
File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py", line 712, in as_dtype
raise TypeError("Cannot convert value %r to a TensorFlow DType." % type_value)
TypeError: Cannot convert value 0 to a TensorFlow DType.

linux 16.04
cuda10.0
cudnn7.6.5
Tensorrt 6.0.1.5
tensorflow 1.12
python 3.6

command line: python3 build_engine.py ssd_mobilenet_v2_egohands

Multiple BBoxes with SSD

Hello Jk,

I have confirmed SSD works with Xavier and is running at 130FPS using async!! Amazing.

Question: When running the SSD demo, are using NMS? It seems when I run it with a live camera feed, there are multiple boxes around each object. This isnt the same issue as when using a single image.

Thanks!

Reshape Error while running Inference

Hi , i followed your code and i tried to covert for my model all went good. i used my .cfg file and .weights file, it given all .onnx and .trt file. But the thing is what i cant able to run inference its showing cant be reshape i dont know why? I changed the categories =19 as my number of classes in utils/yolov3.py file

trt_outputs = [output.reshape(shape) for output, shape Value-error: cannot reshape array of size 12168 into shape (1,255,13,13)

please give a detail explanation about the error.

Mtcnn accuracy problem

Thanks for your work.

I have carefully followed your build instructions and when I run the TensorRT mtcnn face detector on my jetson nano, I am getting low accuracy as like below
code detecting my hand as a face (continuously for some seconds ).

please advise.