ouyangjunyuan/pointcloud-3d-detector-tensorrt

The first tensorrt implementation for point-based 3d detector, i.e., 3DSSD,SASA,IA-SSD.

pointcloud-3d-detector-tensorrt's Issues

compile error on ./plugins/src/nms3DPlugin/nms3D.cu

Dear @OuyangJunyuan,

I got some problem during building source.

The error messages are as follows,

[build] /usr/include/c++/10/tuple(582): error: pack "_UElements" does not have the same number of elements as "_Elements"
[build] detected during:
[build] instantiation of "__nv_bool std::tuple<_Elements...>::__nothrow_constructible<_UElements...>() [with _Elements=<const int &>, _UElements=<>]"
[build] /usr/include/c++/10/bits/stl_map.h(502): here
[build] instantiation of "std::map<_Key, _Tp, _Compare, _Alloc>::mapped_type &std::map<_Key, _Tp, _Compare, _Alloc>::operator[](const std::map<_Key, _Tp, _Compare, _Alloc>::key_type &) [with _Key=int, _Tp=cub::CachingDeviceAllocator::TotalBytes, _Compare=std::less, _Alloc=std::allocator<std::pair<const int, cub::CachingDeviceAllocator::TotalBytes>>]"
[build] /usr/local/cuda/include/cub/util_allocator.cuh(418): here
[build]
[build] 1 error detected in the compilation of "/home/ldk/test_ws/src/pointcloud-3d-detector-tensorrt-devel/plugins/src/nms3DPlugin/nms3D.cu".

And I googled it, they say It's because of gcc version(error on gcc-9, should downgrade to gcc-8), so I downgrade the gcc version to 8, but the error isn't solved.

Did you face the error before, or do you have any solution for this?

My configuration is as follows,
OS: Ubuntu 20.04
Cuda: 11.1
Cudnn: 8.6.0
TensorRT: 8.4.0.6 or 8.4.1.5 (Both cause the error)

Thanks.

IASSD deployment issues on jetson orin

First of all, thank you for your work on the IASSD deployment. I am having problems trying to deploy the IASSD algorithm on jetson orin. When I compile the project no error is generated, but when I publish the point cloud I get stuck and can't end the process with no result or error, can you please give me some help?
Results of cmake run:
-- read TENSORRT_DIR from environment variable
-- read CUDNN_DIR from environment variable
-- Found CUDA: /usr/local/cuda (found suitable version "11.4", minimum required is "11.4")
-- GPU_ARCHS is not defined. Generating CUDA code for default SMs: 53;60;61;70;75;80;86
-- read CUDNN_DIR from environment variable
-- read TENSORRT_DIR from environment variable
-- Using CATKIN_DEVEL_PREFIX: /home/xx/data/pointcloud-3d-detector-tensorrt/build/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/noetic
-- This workspace overlays: /opt/ros/noetic
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.10", minimum required is "3")
-- Using PYTHON_EXECUTABLE: /usr/bin/python3
-- Using Debian Python package layout
-- Found PY_em: /usr/lib/python3/dist-packages/em.py
-- Using empy: /usr/lib/python3/dist-packages/em.py
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/xx/data/pointcloud-3d-detector-tensorrt/build/test_results
-- Forcing gtest/gmock from source, though one was otherwise available.
-- Found gtest sources under '/usr/src/googletest': gtests will be built
-- Found gmock sources under '/usr/src/googletest': gmock will be built
CMake Deprecation Warning at /usr/src/googletest/CMakeLists.txt:4 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

Update the VERSION argument value or use a ... suffix to tell
CMake that the project does not need compatibility with older versions.

-- The C compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
CMake Deprecation Warning at /usr/src/googletest/googlemock/CMakeLists.txt:45 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

Update the VERSION argument value or use a ... suffix to tell
CMake that the project does not need compatibility with older versions.

CMake Deprecation Warning at /usr/src/googletest/googletest/CMakeLists.txt:56 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.

Update the VERSION argument value or use a ... suffix to tell
CMake that the project does not need compatibility with older versions.

-- Found PythonInterp: /usr/bin/python3 (found version "3.8.10")
-- Using Python nosetests: /usr/bin/nosetests3
-- catkin 0.8.10
-- BUILD_SHARED_LIBS is on
-- Checking for module 'eigen3'
-- Found eigen3, version 3.3.7
-- Found Eigen: /usr/include/eigen3 (Required is at least version "3.1")
-- Eigen found (include: /usr/include/eigen3, version: 3.3.7)
-- Checking for module 'flann'
-- Found flann, version 1.9.1
-- Found FLANN: /usr/lib/aarch64-linux-gnu/libflann_cpp.so
CMake Warning (dev) at /opt/cmake-3.23.0/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to find_package_handle_standard_args (PCL_COMMON)
does not match the name of the calling package (PCL). This can lead to
problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
/usr/lib/aarch64-linux-gnu/cmake/pcl/PCLConfig.cmake:616 (find_package_handle_standard_args)
CMakeLists.txt:17 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Found PCL_COMMON: /usr/lib/aarch64-linux-gnu/libpcl_common.so
CMake Warning (dev) at /opt/cmake-3.23.0/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to find_package_handle_standard_args (PCL_KDTREE)
does not match the name of the calling package (PCL). This can lead to
problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
/usr/lib/aarch64-linux-gnu/cmake/pcl/PCLConfig.cmake:616 (find_package_handle_standard_args)
CMakeLists.txt:17 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Found PCL_KDTREE: /usr/lib/aarch64-linux-gnu/libpcl_kdtree.so
CMake Warning (dev) at /opt/cmake-3.23.0/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to find_package_handle_standard_args (PCL_OCTREE)
does not match the name of the calling package (PCL). This can lead to
problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
/usr/lib/aarch64-linux-gnu/cmake/pcl/PCLConfig.cmake:616 (find_package_handle_standard_args)
CMakeLists.txt:17 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Found PCL_OCTREE: /usr/lib/aarch64-linux-gnu/libpcl_octree.so
CMake Warning (dev) at /opt/cmake-3.23.0/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to find_package_handle_standard_args (PCL_SEARCH)
does not match the name of the calling package (PCL). This can lead to
problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
/usr/lib/aarch64-linux-gnu/cmake/pcl/PCLConfig.cmake:616 (find_package_handle_standard_args)
CMakeLists.txt:17 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Found PCL_SEARCH: /usr/lib/aarch64-linux-gnu/libpcl_search.so
CMake Warning (dev) at /opt/cmake-3.23.0/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to find_package_handle_standard_args
(PCL_SAMPLE_CONSENSUS) does not match the name of the calling package
(PCL). This can lead to problems in calling code that expects
find_package result variables (e.g., _FOUND) to follow a certain
pattern.
Call Stack (most recent call first):
/usr/lib/aarch64-linux-gnu/cmake/pcl/PCLConfig.cmake:616 (find_package_handle_standard_args)
CMakeLists.txt:17 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Found PCL_SAMPLE_CONSENSUS: /usr/lib/aarch64-linux-gnu/libpcl_sample_consensus.so
CMake Warning (dev) at /opt/cmake-3.23.0/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to find_package_handle_standard_args
(PCL_FILTERS) does not match the name of the calling package (PCL). This
can lead to problems in calling code that expects find_package result
variables (e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
/usr/lib/aarch64-linux-gnu/cmake/pcl/PCLConfig.cmake:616 (find_package_handle_standard_args)
CMakeLists.txt:17 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Found PCL_FILTERS: /usr/lib/aarch64-linux-gnu/libpcl_filters.so
-- ******** Summary ********
-- CMake version : 3.23.0
-- CMake command : /opt/cmake-3.23.0/bin/cmake
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- C++ compiler version : 9.4.0
-- Build type : Release
-- CXX flags :
-- CMAKE_PREFIX_PATH : /home/xx/data/pointcloud-3d-detector-tensorrt/build/devel;/opt/ros/noetic
-- CMAKE_INSTALL_PREFIX : /usr/local
-- CMAKE_MODULE_PATH : /usr/lib/aarch64-linux-gnu/cmake/pcl/Modules

-- CUDA_VERSION : 11.4
-- CUDA_TOOLKIT_ROOT_DIR : /usr/local/cuda
-- CUDA_LIBRARIES : /usr/local/cuda/lib64/libcudart_static.a;Threads::Threads;dl;/usr/lib/aarch64-linux-gnu/librt.so
-- CUDA_INCLUDE_DIRS: : /usr/local/cuda/include
-- CUDART_LIB : /usr/local/cuda/lib64/libcudart_static.a
-- CUBLAS_LIB : /usr/local/cuda/lib64/libcublas.so
-- TENSORRT_LIB : /usr/src/tensorrt/lib/libnvinfer.so;/usr/lib/aarch64-linux-gnu/libnvonnxparser.so
-- TENSORRT_INCLUDE_DIR : /usr/src/tensorrt/include
-- CUDNN_LIB : /usr/lib/aarch64-linux-gnu/libcudnn.so
-- CUBLASLT_LIB : /usr/local/cuda/lib64/libcublasLt.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/xx/data/pointcloud-3d-detector-tensorrt/build

make -j10 did not report an error and compiled successfully，The message after running is as follows:
./devel/lib/point_detection/point_detector
loading /home/xx/data/pointcloud-3d-detector-tensorrt/plugins/lib/librd3d_trt_plugin.so ...
try loading TensorRT engine file: /home/xx/data/pointcloud-3d-detector-tensorrt/iassd_hvcsx2_gq_4x8_80e_kitti_3cls(export_fp16).engine
====== model infos ======
-> points(1, 16384, 4)
<- boxes(1, 256, 8)
<- scores(1, 256)
<- nums(1, 1)

However, when I publish the point cloud, the program is stuck and can't end. When I do a debug print, I find that it is stuck when the data enters the detector and the GPU occupancy stays at 99%.

My environment is as follows:
CUDA 11.4.14

TensorRT 8.4.1

cuDNN 8.4.1

OpenCV 4.5.4

GCC 9.4.0

Model loading does not finish

After building the workspace, I tried to run the node using:

 rosrun point_detection point_detector

and got these logs:

loading /home/iassd_user/iassd_ws/src/pointcloud-3d-detector-tensorrt/plugins/lib/librd3d_trt_plugin.so ...
try loading TensorRT engine file: /home/iassd_user/iassd_ws/src/pointcloud-3d-detector-tensorrt/config/iassd_hvcsx2_gq_4x8_80e_kitti_3cls(export_fp16).engine
====== model infos ======
-> points(1, 16384, 4)
<- boxes(1, 256, 8)
<- scores(1, 256)
<- nums(1, 1)
=========================
CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars

I set CUDA_MODULE_LOADING=LAZY to get rid of the warning.
but the model is not loading further. I tried to debug it and I think here is the problem.
Any suggestions?
Thanks

How to run the HAV sampler?

Hi, thank you for your great works! I just want to run the HAV sampler of your paper, but the code is a bit complicated, can you tell me how I can do it? Input point cloud to get the down sampling point cloud.

How to deal with Error Code 3: API Usage Error

====== model infos ======
-> points(1, 16384, 4)
<- boxes(1, 256, 8)
<- scores(1, 256)
<- nums(1, 1)

3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)

About Catkin_make and CUBLAS_LIB

Hi,
I tried to build this package with catkin_make and got this cuda-related error:

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 1 packages in topological order:
-- ~~  - point_detection
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'point_detection'
-- ==> add_subdirectory(pointcloud-3d-detector-tensorrt)
-- read TENSORRT_DIR from environment variable
-- read CUDNN_DIR from environment variable
CMake Error at pointcloud-3d-detector-tensorrt/plugins/CMakeLists.txt:34 (find_library):
  Could not find CUBLAS_LIB using the following names: cublas


-- Configuring incomplete, errors occurred!
Invoking "cmake" failed

I think that cudnn dir. is not correct in my case. But, I tried all possible options without luck.
When I search for libcublas:

sudo find /usr -name "libcublas*"

I got these paths:

/usr/local/lib/python3.8/dist-packages/nvidia/cublas/lib/libcublasLt.so.11
/usr/local/lib/python3.8/dist-packages/nvidia/cublas/lib/libcublas.so.11
/usr/local/cuda-12.1/targets/x86_64-linux/lib/stubs/libcublasLt.so
/usr/local/cuda-12.1/targets/x86_64-linux/lib/stubs/libcublas.so
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so.12
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so.12
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublas.so.12.1.0.26
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcublasLt.so.12.1.0.26
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcublasLt.so.11.11.3.6
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcublas.so.11.11.3.6
/usr/local/cuda-11.8/targets/x86_64-linux/lib/stubs/libcublasLt.so
/usr/local/cuda-11.8/targets/x86_64-linux/lib/stubs/libcublas.so
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcublasLt.so
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcublasLt.so.11
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcublas.so.11
/usr/local/cuda-11.8/targets/x86_64-linux/lib/libcublas.so
/usr/share/doc/libcublas-dev-12-1
/usr/share/doc/libcublas-11-8
/usr/share/doc/libcublas-12-1
/usr/share/doc/libcublas-dev-11-8

Could you please direct me to solve this issue?

Thanks

cublas error

Hello, I tried to your project. @OuyangJunyuan

I had error like below in build step.

pointcloud-3d-detector-tensorrt/./inc/model.h:175:25: error: variable ‘std::ofstream p’ has initializer but incomplete type
175 | std::ofstream p(engine_file.c_str(), std::ios::binary);

Question about datasets

Hi!
Thanks for sharing this great work!
I wonder if you plan to add ONNX and configs for models trained on other datasets (Waymo, nuScenes,...). If not, will you provide torch to onnx conversion script for IA-SSD?

Thanks