nvidia-isaac-ros / isaac_ros_object_detection Goto Github PK

View Code? Open in Web Editor NEW

86.0 2.0 18.0 144 KB

Hardware-accelerated, deep learned model support for object detection including YOLOv8 and DetectNet

Home Page: https://developer.nvidia.com/isaac-ros-gems

License: Apache License 2.0

CMake 4.24% C++ 78.26% Shell 3.07% C 14.43%

ros2 object-detection inference deep-learning nvidia triton machine-learning tensorrt ros2-humble ros

isaac_ros_object_detection's Introduction

Isaac ROS Object Detection

Hardware-accelerated, deep learned model support for object detection including DetectNet.

bounding box predictions using DetectNet

Overview

Isaac ROS Object Detection contains an ROS 2 package to perform object detection. isaac_ros_detectnet provides a method for spatial classification using bounding boxes with an input image. Classification is performed by a GPU-accelerated DetectNet model. The output prediction can be used by perception functions to understand the presence and spatial location of an object in an image.

isaac_ros_detectnet is used in a graph of nodes to provide a bounding box detection array with object classes from an input image. A DetectNet model is required to produce the detection array. Input images may need to be cropped and resized to maintain the aspect ratio and match the input resolution of DetectNet; image resolution may be reduced to improve DNN inference performance, which typically scales directly with the number of pixels in the image. isaac_ros_dnn_image_encoder provides a DNN encoder to process the input image into Tensors for the DetectNet model. Prediction results are clustered in the DNN decoder to group multiple detections on the same object. Output is provided as a detection array with object classes.

DNNs have a minimum number of pixels that need to be visible on the object to provide a classification prediction. If a person cannot see the object in the image, it’s unlikely the DNN will. Reducing input resolution to reduce compute may reduce what is detected in the image. For example, a 1920x1080 image containing a distant person occupying 1k pixels (64x16) would have 0.25K pixels (32x8) when downscaled by 1/2 in both X and Y. The DNN may detect the person with the original input image, which provides 1K pixels for the person, and fail to detect the same person in the downscaled resolution, which only provides 0.25K pixels for the person.

Note

DetectNet is similar to other popular object detection models such as YOLOV3, FasterRCNN, and SSD, while being efficient at detecting multiple object classes in large images.

Object detection classifies a rectangle of pixels as containing an object, whereas image segmentation provides more information and uses more compute to produce a classification per pixel. Object detection is used to know if, and where in a 2D image, the object exists. If a 3D spacial understanding or size of an object in pixels is required, use image segmentation.

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

Performance

Sample Graph	Input Size	AGX Orin	Orin NX	Orin Nano 8GB	x86_64 w/ RTX 4060 Ti
DetectNet Object Detection Graph	544p	232 fps 11 ms	105 fps 15 ms	74.2 fps 22 ms	644 fps 5.6 ms

Documentation

Please visit the Isaac ROS Documentation to learn how to use this repository.

Packages

Latest

Update 2023-10-18: Adding NITROS YOLOv8 decoder.

isaac_ros_object_detection's People

Contributors

Stargazers

Watchers

Forkers

terrisgo nithaelizabeth datatecyl mstiehm-nf kanghitea wizyke georgehu0815 weirdsim14 abdullahgm1 sumerudataanalaytics nctu-auv ut-ras hornet9software r4hul77 leon-seidel space-manipulator-project annemsony

isaac_ros_object_detection's Issues

Could NOT find CUDA (missing: CUDA_INCLUDE_DIRS) (found version "11.4")

admin@thesis-desktop:/workspaces/isaac_ros-dev$ cd /workspaces/isaac_ros-dev && \

colcon build --symlink-install &&
source install/setup.bash

error:
Starting >>> isaac_ros_test
Starting >>> isaac_ros_tensor_list_interfaces
Finished <<< isaac_ros_test [9.12s]
Starting >>> isaac_ros_nitros
--- stderr: isaac_ros_nitros
CMake Error at /usr/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find CUDA (missing: CUDA_INCLUDE_DIRS) (found version "11.4")
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake-3.23/Modules/FindCUDA.cmake:1264 (find_package_handle_standard_args)
CMakeLists.txt:67 (find_package)

Failed <<< isaac_ros_nitros [15.5s, exited with code 1]
Aborted <<< isaac_ros_tensor_list_interfaces [50.3s]

Summary: 1 package finished [51.8s]
1 package failed: isaac_ros_nitros
1 package aborted: isaac_ros_tensor_list_interfaces
1 package had stderr output: isaac_ros_nitros
18 packages not processed

is there any mistake in my installations? please let me know

Thanks

isaac_ros_common/scripts/run_dev.sh your_ws failed

Hi expert,
I I try to test isaac_ros_common/scripts/run_dev.sh your_ws on xavier AGX with sys 4.6.1 followed guide , during run command : errors report
you can check log in attachments ,thanks!

Collecting protobuf<=3.20.1,>=3.12.2 (from onnx)
Downloading https://files.pythonhosted.org/packages/c6/1c/f18d97fc479b4fb6f72bbb0e41188575362e3bbd31014cf294ef0fdec8bf/protobuf-3.19.4-py2.py3-none-any.whl (162kB)
Building wheels for collected packages: onnx
Running setup.py bdist_wheel for onnx: started
�[91m Failed building wheel for onnx
�[0m Running setup.py bdist_wheel for onnx: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-hofm4dj7/onnx/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/tmpdbtxuctapip-wheel- --python-tag cp36:
fatal: not a git repository (or any of the parent directories): .git
/usr/local/lib/python3.6/dist-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
SetuptoolsDeprecationWarning,
running bdist_wheel
r
test.txt

System throttled due to over current when running Yolov8 on Orin-NX 16GB

Greetings.

I have a Jetson Oin NX 16GB with carrier board J401 from seedstudio.

When I run isaac_ros_yolov8, I continuously get the following warning 'System throttled due to over-current.'

I am using the MAXN power mode.

Here is a log file generated by the Jetson Power GUI.
yolo_over_current_issue_orin_nx16.csv

I also noticed that the cashed memory gets large when I run this package! Here is a screenshot from jtop

Here is a small video that shows the output of jtop
jtop1.webm

Here are the commands I run with their terminal outputs.

command

ros2 launch isaac_ros_yolov8 realsense.launch.py

realsense.launch.py

# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

import launch
from launch_ros.actions import ComposableNodeContainer, Node
from launch_ros.descriptions import ComposableNode


def generate_launch_description():
    """Launch file which brings up visual slam node configured for RealSense."""
    realsense_camera_node = Node(
        name='camera',
        namespace='camera',
        package='realsense2_camera',
        executable='realsense2_camera_node',
        parameters=[{
                'align_depth.enable': True,
                'enable_infra1': False,
                'enable_infra2': False,
                'enable_color': True,
                'enable_depth': True,
                'depth_module.emitter_enabled': 0,
                'rgb_camera.profile': '1280x720x30', 
                'depth_module.profile': '640x480x90',
                'enable_gyro': False,
                'enable_accel': False,
                'gyro_fps': 200,
                'accel_fps': 200,
                'unite_imu_method': 2
        }],
        #remappings=[('color/image_raw', '/image')
        #]
    )
    
    image_resize_node_color = ComposableNode(
        package='isaac_ros_image_proc',
        plugin='nvidia::isaac_ros::image_proc::ResizeNode',
        name='image_resize_right',
        parameters=[{
                'output_width': 640,
                'output_height': 640,
                'keep_aspect_ratio': True
        }],
        remappings=[
            ('camera_info', '/camera/color/camera_info'),
            ('image', '/camera/color/image_raw'),
            ('resize/camera_info', '/camera/color/camera_info_resize'),
            ('resize/image', '/image')]
    )
    
    resize_launch_container = ComposableNodeContainer(
        name='resize_launch_container',
        namespace='',
        package='rclcpp_components',
        executable='component_container',
        composable_node_descriptions=[
            image_resize_node_color
        ],
        output='screen'
    )

    return launch.LaunchDescription([realsense_camera_node, resize_launch_container])
   # return launch.LaunchDescription([realsense_camera_node])

output

[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-11-30-08-39-54-
528665-ubuntu-563055
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [563066]
[INFO] [component_container-2]: process started with pid [563068]
[component_container-2] [INFO] [1701322795.155558550] [resize_launch_container]: Load
Library: /workspaces/isaac_ros-dev/install/isaac_ros_image_proc/lib/libresize_node.so
[realsense2_camera_node-1] [INFO] [1701322795.179117503] [camera.camera]: RealSense
ROS v4.51.1
[realsense2_camera_node-1] [INFO] [1701322795.179345798] [camera.camera]: Built with
LibRealSense v2.54.2
[realsense2_camera_node-1] [INFO] [1701322795.179368646] [camera.camera]: Running with
LibRealSense v2.54.2
[component_container-2] [INFO] [1701322795.215812099] [resize_launch_container]: Found
class: rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::image_proc::ResizeNode&gt;
[component_container-2] [INFO] [1701322795.215926118] [resize_launch_container]:
Instantiate class:
rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::image_proc::ResizeNode&gt;
[component_container-2] [INFO] [1701322795.262946420] [NitrosContext]: [NitrosContext]
Creating a new shared context
[component_container-2] [INFO] [1701322795.263211420] [image_resize_right]: [NitrosNode]
Initializing NitrosNode
[component_container-2] [INFO] [1701322795.265203222] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/std/libgxf_std.so
[component_container-2] [INFO] [1701322795.283212574] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container-2] [INFO] [1701322795.287727520] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/libgxf_sight.so
[component_container-2] [INFO] [1701322795.306618338] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/libgxf_atlas.so
[component_container-2] [INFO] [1701322795.324723757] [NitrosContext]: [NitrosContext]
Loading application: &#39;/workspaces/isaac_ros-
dev/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_grap
h.yaml&#39;
[component_container-2] [INFO] [1701322795.326122357] [NitrosContext]: [NitrosContext]
Initializing application...

[component_container-2] [INFO] [1701322795.342824728] [NitrosContext]: [NitrosContext]
Running application...
[component_container-2] 2023-11-30 08:39:55.342 WARN gxf/std/program.cpp@514: No
system specified. Nothing to do
[component_container-2] [INFO] [1701322795.353971930] [image_resize_right]: [NitrosNode]
Starting NitrosNode
[component_container-2] [INFO] [1701322795.354067165] [image_resize_right]: [NitrosNode]
Loading built-in preset extension specs
[component_container-2] [INFO] [1701322795.357669349] [image_resize_right]: [NitrosNode]
Loading built-in extension specs
[component_container-2] [INFO] [1701322795.357795816] [image_resize_right]: [NitrosNode]
Loading preset extension specs
[component_container-2] [INFO] [1701322795.369142448] [image_resize_right]: [NitrosNode]
Loading extension specs
[component_container-2] [INFO] [1701322795.369308085] [image_resize_right]: [NitrosNode]
Loading generator rules
[component_container-2] [INFO] [1701322795.369807075] [image_resize_right]: [NitrosNode]
Loading extensions
[component_container-2] [INFO] [1701322795.372091589] [image_resize_right]:
[NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container-2] [INFO] [1701322795.374778387] [image_resize_right]:
[NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container-2] [INFO] [1701322795.377150327] [image_resize_right]:
[NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container-2] [INFO] [1701322795.399408506] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container-2] [INFO] [1701322795.409400379] [image_resize_right]: [NitrosNode]
Loading graph to the optimizer
[component_container-2] [INFO] [1701322795.423973344] [image_resize_right]: [NitrosNode]
Running optimization
[realsense2_camera_node-1] [INFO] [1701322795.550378595] [camera.camera]: Device with
serial number 207322250752 was found.
[realsense2_camera_node-1]
[realsense2_camera_node-1] [INFO] [1701322795.550516743] [camera.camera]: Device with
physical ID 2-1.3-3 was found.
[realsense2_camera_node-1] [INFO] [1701322795.550533736] [camera.camera]: Device with
name Intel RealSense D455 was found.
[realsense2_camera_node-1] [INFO] [1701322795.551027542] [camera.camera]: Device with
port number 2-1.3 was found.
[realsense2_camera_node-1] [INFO] [1701322795.551053079] [camera.camera]: Device USB
type: 3.2
[realsense2_camera_node-1] [INFO] [1701322795.567075653] [camera.camera]:
getParameters...
[realsense2_camera_node-1] [INFO] [1701322795.573704453] [camera.camera]: JSON file is

not provided
[realsense2_camera_node-1] [INFO] [1701322795.575166287] [camera.camera]: Device
Name: Intel RealSense D455
[realsense2_camera_node-1] [INFO] [1701322795.575854211] [camera.camera]: Device Serial
No: 207322250752
[realsense2_camera_node-1] [INFO] [1701322795.576485301] [camera.camera]: Device
physical port: 2-1.3-3
[realsense2_camera_node-1] [INFO] [1701322795.576511222] [camera.camera]: Device FW
version: 5.14.0
[realsense2_camera_node-1] [INFO] [1701322795.576524630] [camera.camera]: Device
Product ID: 0x0B5C
[realsense2_camera_node-1] [INFO] [1701322795.576535831] [camera.camera]: Sync Mode:
Off
[component_container-2] [INFO] [1701322795.579101473] [image_resize_right]: [NitrosNode]
Obtaining graph IO group info from the optimizer
[component_container-2] [INFO] [1701322795.583747303] [image_resize_right]: [NitrosNode]
Creating negotiated publishers/subscribers
[component_container-2] [INFO] [1701322795.631932567] [image_resize_right]: [NitrosNode]
Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node &#39;/image_resize_right&#39; in
container &#39;/resize_launch_container&#39;
[component_container-2] [INFO] [1701322795.647637116] [image_resize_right]: Negotiating
[component_container-2] [INFO] [1701322795.651419818] [image_resize_right]: Negotiating
[component_container-2] [INFO] [1701322796.640272703] [image_resize_right]: [NitrosNode]
Starting post negotiation setup
[component_container-2] [INFO] [1701322796.640406851] [image_resize_right]: [NitrosNode]
Getting data format negotiation results
[component_container-2] [INFO] [1701322796.640440036] [image_resize_right]:
[NitrosPublisher] Use the negotiated data format: &quot;nitros_image_rgb8&quot;
[component_container-2] [INFO] [1701322796.640456644] [image_resize_right]:
[NitrosPublisher] Negotiation ended with no results
[component_container-2] [INFO] [1701322796.640469125] [image_resize_right]:
[NitrosPublisher] Use only the compatible publisher:
topic_name=&quot;/camera/color/camera_info_resize&quot;, data_format=&quot;nitros_camera_info&quot;
[component_container-2] [INFO] [1701322796.640480805] [image_resize_right]:
[NitrosSubscriber] Negotiation ended with no results
[component_container-2] [INFO] [1701322796.640492517] [image_resize_right]:
[NitrosSubscriber] Use the compatible subscriber: topic_name=&quot;/camera/color/image_raw&quot;,
data_format=&quot;nitros_image_bgr8&quot;
[component_container-2] [INFO] [1701322796.640597480] [image_resize_right]:
[NitrosSubscriber] Negotiation ended with no results
[component_container-2] [INFO] [1701322796.640610217] [image_resize_right]:
[NitrosSubscriber] Use the compatible subscriber: topic_name=&quot;/camera/color/camera_info&quot;,
data_format=&quot;nitros_camera_info&quot;

[component_container-2] [INFO] [1701322796.640669962] [image_resize_right]:
[NitrosPublisherSubscriberGroup] Adjusted the compatible format of the component
&quot;input_compositor/image_in&quot; (type=&quot;nvidia::gxf::DoubleBufferReceiver&quot;) from
&quot;nitros_image_bgr8&quot; to &quot;nitros_image_rgb8&quot;
[component_container-2] [INFO] [1701322796.641014836] [image_resize_right]: [NitrosNode]
Exporting the final graph based on the negotiation results
[component_container-2] [INFO] [1701322796.648775253] [image_resize_right]: [NitrosNode]
Wrote the final top level YAML graph to
&quot;/tmp/isaac_ros_nitros/graphs/JTILYGRDJC/JTILYGRDJC.yaml&quot;
[component_container-2] [INFO] [1701322796.648877431] [image_resize_right]: [NitrosNode]
Calling user&#39;s pre-load-graph callback
[component_container-2] [INFO] [1701322796.648893048] [image_resize_right]: [NitrosNode]
Loading application
[component_container-2] [INFO] [1701322796.648910040] [image_resize_right]:
[NitrosContext] Loading application:
&#39;/tmp/isaac_ros_nitros/graphs/JTILYGRDJC/JTILYGRDJC.yaml&#39;
[component_container-2] [INFO] [1701322796.655138892] [image_resize_right]: [NitrosNode]
Linking Nitros pub/sub to the loaded application
[component_container-2] [INFO] [1701322796.656533845] [image_resize_right]: [NitrosNode]
Calling user&#39;s post-load-graph callback
[component_container-2] [INFO] [1701322796.656981858] [image_resize_right]: [ResizeNode]
postLoadGraphCallback().
[component_container-2] [INFO] [1701322796.657452719] [image_resize_right]:
[NitrosContext] Initializing application...
[component_container-2] [INFO] [1701322797.361305149] [image_resize_right]:
[NitrosContext] Running application...
[component_container-2] [INFO] [1701322797.361897198] [image_resize_right]: [NitrosNode]
Starting a heartbeat timer (eid=17)
[realsense2_camera_node-1] [WARN] [1701322800.764950763] [camera.camera]: re-enable
the stream for the change to take effect.
[realsense2_camera_node-1] [WARN] [1701322802.145945524] [camera.camera]: re-enable
the stream for the change to take effect.
[realsense2_camera_node-1] [INFO] [1701322802.212011146] [camera.camera]: Stopping
Sensor: Stereo Module
[realsense2_camera_node-1] [INFO] [1701322802.422906682] [camera.camera]: Starting
Sensor: Stereo Module
[realsense2_camera_node-1] [INFO] [1701322802.686489277] [camera.camera]: Open profile:
stream_type: Depth(0), Format: Z16, Width: 640, Height: 480, FPS: 90
[realsense2_camera_node-1] [INFO] [1701322802.686779301] [camera.camera]: Stopping
Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1701322802.715066711] [camera.camera]: Starting
Sensor: RGB Camera
[realsense2_camera_node-1] [WARN] [1701322802.740877985] [camera.camera]:
[realsense2_camera_node-1] [INFO] [1701322802.747071828] [camera.camera]: Open profile:

stream_type: Color(0), Format: RGB8, Width: 1280, Height: 720, FPS: 30
[realsense2_camera_node-1] [INFO] [1701322802.748940746] [camera.camera]: RealSense
Node Is Up!

Command

ros2 launch isaac_ros_yolov8 isaac_ros_yolov8_visualize.launch.py model_file_path:=/workspaces/isaac_ros-dev/src/onnx/drone_detection_v21_int8.onnx engine_file_path:=/workspaces/isaac_ros-dev/src/onnx/drone_detection_v21_int8.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 force_engine_update:=False image_mean:=[0.0,0.0,0.0] image_stddev:=[1.0,1.0,1.0] input_image_width:=640 input_image_height:=640 confidence_threshold:=0.25 nms_threshold:=0.45

Output

[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-11-30-08-39-38-
677274-ubuntu-563002
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [563013]
[INFO] [isaac_ros_yolov8_visualizer.py-2]: process started with pid [563015]
[component_container_mt-1] [INFO] [1701322779.374554982] [tensor_rt_container]: Load
Library: /workspaces/isaac_ros-
dev/install/isaac_ros_dnn_image_encoder/lib/libdnn_image_encoder_node.so
[component_container_mt-1] [INFO] [1701322779.422003422] [tensor_rt_container]: Found
class:
rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::dnn_inference::DnnImageEncod
erNode&gt;
[component_container_mt-1] [INFO] [1701322779.422146370] [tensor_rt_container]:
Instantiate class:
rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::dnn_inference::DnnImageEncod
erNode&gt;
[component_container_mt-1] [INFO] [1701322779.428988104] [NitrosContext]:
[NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1701322779.429260784] [dnn_image_encoder]:
[NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1701322779.430896223] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1701322779.439542456] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-1] [INFO] [1701322779.446296123] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-1] [INFO] [1701322779.451424463] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-1] [INFO] [1701322779.456436031] [NitrosContext]:
[NitrosContext] Loading application: &#39;/workspaces/isaac_ros-
dev/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_grap
h.yaml&#39;
[component_container_mt-1] [INFO] [1701322779.457807943] [NitrosContext]:
[NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1701322779.462127075] [NitrosContext]:
[NitrosContext] Running application...

[component_container_mt-1] 2023-11-30 08:39:39.462 WARN gxf/std/program.cpp@514: No
system specified. Nothing to do
[component_container_mt-1] [INFO] [1701322779.463995417] [dnn_image_encoder]:
[NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1701322779.464062075] [dnn_image_encoder]:
[NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1701322779.467213334] [dnn_image_encoder]:
[NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1701322779.467351162] [dnn_image_encoder]:
[NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1701322779.469741855] [dnn_image_encoder]:
[NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1701322779.469813601] [dnn_image_encoder]:
[NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1701322779.470381681] [dnn_image_encoder]:
[NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1701322779.471138759] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container_mt-1] [INFO] [1701322779.473786516] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1701322779.475630569] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1701322779.482131012] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container_mt-1] [INFO] [1701322779.496469250] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1701322779.498295478] [dnn_image_encoder]:
[NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1701322779.506178682] [dnn_image_encoder]:
[NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1701322779.755889691] [dnn_image_encoder]:
[NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1701322779.765680566] [dnn_image_encoder]:
[NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1701322779.793095948] [dnn_image_encoder]:
[NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node &#39;/dnn_image_encoder&#39; in
container &#39;/tensor_rt_container&#39;
[component_container_mt-1] [INFO] [1701322779.800116631] [tensor_rt_container]: Load
Library: /workspaces/isaac_ros-dev/install/isaac_ros_tensor_rt/lib/libtensor_rt_node.so
[component_container_mt-1] [INFO] [1701322779.854147021] [tensor_rt_container]: Found
class:
rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::dnn_inference::TensorRTNode&gt;
[component_container_mt-1] [INFO] [1701322779.854299506] [tensor_rt_container]:

Instantiate class:
rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::dnn_inference::TensorRTNode&gt;
[component_container_mt-1] [INFO] [1701322779.870576999] [tensor_rt]: [NitrosNode]
Initializing NitrosNode
[component_container_mt-1] [INFO] [1701322779.873076367] [tensor_rt]: [NitrosNode]
Starting NitrosNode
[component_container_mt-1] [INFO] [1701322779.873117968] [tensor_rt]: [NitrosNode]
Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1701322779.876811227] [tensor_rt]: [NitrosNode]
Loading built-in extension specs
[component_container_mt-1] [INFO] [1701322779.876909438] [tensor_rt]: [NitrosNode]
Loading preset extension specs
[component_container_mt-1] [INFO] [1701322779.878036990] [tensor_rt]: [NitrosNode]
Loading extension specs
[component_container_mt-1] [INFO] [1701322779.878084640] [tensor_rt]: [NitrosNode]
Loading generator rules
[component_container_mt-1] [INFO] [1701322779.878297030] [tensor_rt]: [NitrosNode]
Loading extensions
[component_container_mt-1] [INFO] [1701322779.882214999] [tensor_rt]: [NitrosContext]
Loading extension: gxf/tensor_rt/libgxf_tensor_rt.so
[component_container_mt-1] [INFO] [1701322779.886617110] [tensor_rt]: [NitrosNode]
Loading graph to the optimizer
[component_container_mt-1] [INFO] [1701322779.890131195] [tensor_rt]: [NitrosNode]
Running optimization
[component_container_mt-1] [INFO] [1701322779.920149533] [tensor_rt]: [NitrosNode]
Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1701322779.927303211] [tensor_rt]: [NitrosNode]
Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1701322779.928922938] [tensor_rt]:
[NitrosPublisherSubscriberGroup] Pinning the component &quot;inference/rx&quot;
(type=&quot;nvidia::gxf::DoubleBufferReceiver&quot;) to use its compatible format only:
&quot;nitros_tensor_list_nchw_rgb_f32&quot;
[component_container_mt-1] [INFO] [1701322779.929025181] [tensor_rt]:
[NitrosPublisherSubscriberGroup] Pinning the component &quot;sink/sink&quot;
(type=&quot;nvidia::isaac_ros::MessageRelay&quot;) to use its compatible format only:
&quot;nitros_tensor_list_nhwc_rgb_f32&quot;
[component_container_mt-1] [INFO] [1701322779.937657302] [tensor_rt]: [NitrosNode]
Starting negotiation...
[component_container_mt-1] [INFO] [1701322779.939187458] [dnn_image_encoder]:
Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node &#39;/tensor_rt&#39; in container
&#39;/tensor_rt_container&#39;
[component_container_mt-1] [INFO] [1701322779.943489278] [tensor_rt_container]: Load
Library: /workspaces/isaac_ros-dev/install/isaac_ros_yolov8/lib/libyolov8_decoder_node.so

[component_container_mt-1] [INFO] [1701322779.967102151] [tensor_rt_container]: Found
class:
rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::yolov8::YoloV8DecoderNode&gt;
[component_container_mt-1] [INFO] [1701322779.968357835] [tensor_rt_container]:
Instantiate class:
rclcpp_components::NodeFactoryTemplate&lt;nvidia::isaac_ros::yolov8::YoloV8DecoderNode&gt;
[component_container_mt-1] [INFO] [1701322779.991981876]
[yolov8_decoder_node.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[component_container_mt-1] [INFO] [1701322779.992318590] [tensor_rt]: Negotiating
[component_container_mt-1] [INFO] [1701322779.993687781] [tensor_rt]: Could not
negotiate
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node &#39;/yolov8_decoder_node&#39; in
container &#39;/tensor_rt_container&#39;
[component_container_mt-1] [INFO] [1701322780.795652960] [dnn_image_encoder]:
[NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1701322780.795782660] [dnn_image_encoder]:
[NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1701322780.795815109] [dnn_image_encoder]:
[NitrosPublisher] Use the negotiated data format: &quot;nitros_tensor_list_nchw_rgb_f32&quot;
[component_container_mt-1] [INFO] [1701322780.795831846] [dnn_image_encoder]:
[NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1701322780.795843174] [dnn_image_encoder]:
[NitrosSubscriber] Use the compatible subscriber: topic_name=&quot;/image&quot;,
data_format=&quot;nitros_image_bgr8&quot;
[component_container_mt-1] [INFO] [1701322780.796284371] [dnn_image_encoder]:
[NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1701322780.912005116] [dnn_image_encoder]:
[NitrosNode] Wrote the final top level YAML graph to
&quot;/tmp/isaac_ros_nitros/graphs/JNFIVUAIDJ/JNFIVUAIDJ.yaml&quot;
[component_container_mt-1] [INFO] [1701322780.914359872] [dnn_image_encoder]:
[NitrosNode] Calling user&#39;s pre-load-graph callback
[component_container_mt-1] [INFO] [1701322780.915398238] [dnn_image_encoder]: In DNN
Image Encoder Node preLoadGraphCallback().
[component_container_mt-1] [INFO] [1701322780.916077970] [dnn_image_encoder]:
[NitrosNode] Loading application
[component_container_mt-1] [INFO] [1701322780.916602529] [dnn_image_encoder]:
[NitrosContext] Loading application:
&#39;/tmp/isaac_ros_nitros/graphs/JNFIVUAIDJ/JNFIVUAIDJ.yaml&#39;
[component_container_mt-1] [INFO] [1701322780.926448733] [dnn_image_encoder]:
[NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1701322780.928209296] [dnn_image_encoder]:
[NitrosNode] Calling user&#39;s post-load-graph callback
[component_container_mt-1] [INFO] [1701322780.928670525] [dnn_image_encoder]: In DNN
Image Encoder Node postLoadGraphCallback().

[component_container_mt-1] [INFO] [1701322780.929245582] [dnn_image_encoder]:
[NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1701322780.939570583] [tensor_rt]: [NitrosNode]
Starting post negotiation setup
[component_container_mt-1] [INFO] [1701322780.939669530] [tensor_rt]: [NitrosNode]
Getting data format negotiation results
[component_container_mt-1] [INFO] [1701322780.939688123] [tensor_rt]: [NitrosPublisher]
Negotiation ended with no results
[component_container_mt-1] [INFO] [1701322780.939704251] [tensor_rt]: [NitrosPublisher]
Use only the compatible publisher: topic_name=&quot;/tensor_sub&quot;,
data_format=&quot;nitros_tensor_list_nhwc_rgb_f32&quot;
[component_container_mt-1] [INFO] [1701322780.939790558] [tensor_rt]: [NitrosSubscriber]
Use the negotiated data format: &quot;nitros_tensor_list_nchw_rgb_f32&quot;
[component_container_mt-1] [INFO] [1701322780.940029957] [tensor_rt]: [NitrosNode]
Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1701322780.944896561] [tensor_rt]: [NitrosNode]
Wrote the final top level YAML graph to
&quot;/tmp/isaac_ros_nitros/graphs/IQPYNYPNPH/IQPYNYPNPH.yaml&quot;
[component_container_mt-1] [INFO] [1701322780.945559780] [tensor_rt]: [NitrosNode]
Calling user&#39;s pre-load-graph callback
[component_container_mt-1] [INFO] [1701322780.945582277] [tensor_rt]: [NitrosNode]
Loading application
[component_container_mt-1] [INFO] [1701322781.087987504] [dnn_image_encoder]:
[NitrosContext] Running application...
[component_container_mt-1] [INFO] [1701322781.089617119] [tensor_rt]: [NitrosContext]
Loading application: &#39;/tmp/isaac_ros_nitros/graphs/IQPYNYPNPH/IQPYNYPNPH.yaml&#39;
[component_container_mt-1] [INFO] [1701322781.093182182] [tensor_rt]: [NitrosNode]
Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1701322781.093334283] [tensor_rt]: [NitrosNode]
Calling user&#39;s post-load-graph callback
[component_container_mt-1] [INFO] [1701322781.093362635] [tensor_rt]: In TensorRTNode
postLoadGraphCallback().
[component_container_mt-1] 2023-11-30 08:39:41.092 WARN
gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter &#39;dummy_rx&#39; in component
&#39;&#39;.
[component_container_mt-1] 2023-11-30 08:39:41.093 WARN
gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter &#39;dev_id&#39; in component
&#39;stream&#39;.
[component_container_mt-1] [INFO] [1701322781.102424465] [dnn_image_encoder]:
[NitrosNode] Starting a heartbeat timer (eid=25)
[component_container_mt-1] [INFO] [1701322784.080123808] [tensor_rt]: Read tensor shape
information from TRT Model Engine: /workspaces/isaac_ros-
dev/src/onnx/drone_detection_v22_Yolov8n_int8.plan
[component_container_mt-1] [INFO] [1701322784.081762767] [tensor_rt]: Tensors 201600

bytes, num outputs 40 x tensors per output 1 = 40 blocks
[component_container_mt-1] [INFO] [1701322784.082625256] [tensor_rt]: [NitrosContext]
Initializing application...
[component_container_mt-1] [INFO] [1701322784.088133479] [tensor_rt]: [NitrosContext]
Running application...
[component_container_mt-1] [INFO] [1701322784.090031262] [tensor_rt]: [NitrosNode]
Starting a heartbeat timer (eid=110)
[component_container_mt-1] [INFO] [1701322795.652244513] [dnn_image_encoder]:
Negotiating

list of nodes

/camera/camera
/dnn_image_encoder
/image_resize_right
/launch_ros_563002
/launch_ros_563055
/resize_launch_container
/tensor_rt
/tensor_rt_container
/yolov8_decoder_node
/yolov8_visualizer

list of topics

/camera/aligned_depth_to_color/camera_info
/camera/aligned_depth_to_color/image_raw
/camera/aligned_depth_to_color/image_raw/compressed
/camera/aligned_depth_to_color/image_raw/compressedDepth
/camera/aligned_depth_to_color/image_raw/theora
/camera/color/camera_info
/camera/color/camera_info/nitros
/camera/color/camera_info_resize
/camera/color/camera_info_resize/nitros
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressedDepth
/camera/color/image_raw/nitros
/camera/color/image_raw/theora

/camera/color/metadata
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/theora
/camera/depth/metadata
/camera/extrinsics/depth_to_color
/camera/extrinsics/depth_to_depth
/camera/imu
/detections_output
/image
/image/nitros
/image/nitros/nitros_image_rgb8
/parameter_events
/rosout
/tensor_pub
/tensor_pub/nitros
/tensor_sub
/tensor_sub/nitros
/tf_static
/yolov8_processed_image

I hope I can get some help on how to solve this as there would be more applications running alongside the yolov8.

Thanks.

loading different yolov8 model other then the standard downloadable lead to crash of ros2 composable_node

hello everyone,
i did train a yolov8s model by my self but when i send it as launch file parameter, everithing goes well unitll i start the image_publisher node, that lead to the composable_node to crash without any infos about the crash in itself.
Can you help me?
I'll provide further infos if necessary.
Thanks in advance for you time,
Franzhd, member of Team Roboto.

Seeking advice on 'process has died' error

          @mzahana Thank you for your help and advice, I have gained some insights after looking into netron again. I did changed the number of classes in the script and I think the error is caused by the number of outputs my model gives, which consists of output0 and output1. I managed to fix the mismatch error after changing `output_tensor_names`  and `output_binding_names` to `[output0, output1]`. However, another error occurred which I'm not sure of what could be the cause of it.

@kajananchinniahNV Thank you for your help, I have changed the parameters as mentioned above and managed to fix the mismatch error. Unfortunately another error occurred. The process has died without additional log on its cause. it'd be great if you could share some insights or advice on how to troubleshoot the cause of it.

Really appreciate both of your help, thank you!

Originally posted by @immaixq in #32 (comment)

Using OpenCV NMS in decoder

I was using detectnet in ROS2 foxy but I want to update to the latest release in Humble. Before I was not using the DBscan clustering on the bounding boxes but I implemented opencv NMS in the decoder which gave me a bit better results and was faster. Now I want to do something similar but it done somewhere in the gxf library which is more difficult to change.

So my question is if I can use DetectNet with only NMS somewhat and no DBscan.

Modifying YOLOv8 Decoder Node for Segmentation Mask Extraction

I am currently exploring the use of a segmentation model for my project. To integrate this model, it appears necessary for me to adapt or develop a new decoder node. The current decoder node in the isaac_ros_yolov8 repository is designed for extracting bounding boxes, which does not align with my requirement for segmentation mask extraction.

Would it be feasible to modify the YOLOv8 decoder node, specifically the section responsible for bounding box extraction, to enable segmentation mask extraction instead? Or does this task require a more comprehensive overhaul than initially anticipated?

Any guidance or recommendations on how to approach this modification would be greatly appreciated.

isaac_ros_object_detection node doesn't install parameters file

This line installs the launch file but does not install the parameters file.

isaac_ros_object_detection/isaac_ros_detectnet/CMakeLists.txt

Line 83 in 237774a

ament_auto_package(INSTALL_TO_SHARE launch)

So a user has to use symlink build to launch the node or to copy the file himself.

ament_auto_package(INSTALL_TO_SHARE launch params)
solves the issue

Wrong detections using PeopleNet and Triton

Hello, I have been testing the object detection pipeline using PeopleNet and my camera input (also a rosbag captured from this camera) on the AGX Orin. The camera has a resolution of 640x480. For this I have written my own config to be used with tao-converter

name: "detectnet"
platform: "tensorrt_plan"
max_batch_size: 16
input [
  {
    name: "input_1"
    data_type: TYPE_FP32
    format: FORMAT_NCHW
    dims: [ 3, 480, 640 ]
  }
]
output [
  {
        name: "output_bbox/BiasAdd"
        data_type: TYPE_FP32
        dims: [ 12, 30, 40]
    },
    {
        name: "output_cov/Sigmoid"
        data_type: TYPE_FP32
        dims: [ 3, 30, 40]
    }
]

and use the following command to generate the model plan

setup_model.sh --height 480 --width 640 --config-file /workspaces/detectnet.pbtxt

I have also modified the launch file accordingly (change encoder input and output size)

def generate_launch_description():
    """Generate launch description for testing relevant nodes."""
    launch_dir_path = os.path.dirname(os.path.realpath(__file__))
    config = launch_dir_path + '/../config/params.yaml'
    model_dir_path = '/tmp/models'

    # Read labels from text file
    labels_file_path = f'{model_dir_path}/detectnet/1/labels.txt'
    with open(labels_file_path, 'r') as fd:
        label_list = fd.read().strip().splitlines()

    encoder_node = ComposableNode(
        name='dnn_image_encoder',
        package='isaac_ros_dnn_image_encoder',
        plugin='nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode',
        parameters=[{
            'input_image_width': 640,
            'input_image_height': 480,
            'network_image_width': 640,
            'network_image_height': 480,
            'image_mean': [0.0, 0.0, 0.0],
            'image_stddev': [1.0, 1.0, 1.0],
            'enable_padding': False
        }],
        remappings=[('encoded_tensor', 'tensor_pub'),
                    ('image', 'v4l/camera/image_raw')]
    )

    triton_node = ComposableNode(
        name='triton_node',
        package='isaac_ros_triton',
        plugin='nvidia::isaac_ros::dnn_inference::TritonNode',
        parameters=[{
            'model_name': 'detectnet',
            'model_repository_paths': [model_dir_path],
            'input_tensor_names': ['input_tensor'],
            'input_binding_names': ['input_1'],
            'input_tensor_formats': ['nitros_tensor_list_nchw_rgb_f32'],
            'output_tensor_names': ['output_cov', 'output_bbox'],
            'output_binding_names': ['output_cov/Sigmoid', 'output_bbox/BiasAdd'],
            'output_tensor_formats': ['nitros_tensor_list_nhwc_rgb_f32'],
            'log_level': 0
        }])

    detectnet_decoder_node = ComposableNode(
        name='detectnet_decoder_node',
        package='isaac_ros_detectnet',
        plugin='nvidia::isaac_ros::detectnet::DetectNetDecoderNode',
        parameters=[config,
                    {
                        'label_list': label_list
                    }]
    )

    container = ComposableNodeContainer(
        name='detectnet_container',
        namespace='detectnet_container',
        package='rclcpp_components',
        executable='component_container_mt',
        composable_node_descriptions=[
            encoder_node, triton_node, detectnet_decoder_node],
        output='screen'
    )

    return launch.LaunchDescription([container])

When I run the pipeline with just me in the frame, I get my main objects detected correctly (i.e., 1 person and 1 face within the right bounding boxes), however I am also getting a lot of wrong detections (~10 bounding boxes which appear and disappear quickly near the other two boxes). I tested the pipeline using the same model and input video in DeepStream and the detection is working correctly.

Cuda bindings mismatch error using custom trained model

Description:
Encounter error using yolov8_tensor_rt.launch.py and isaac_ros_yolov8_visualize.launch.py. The scripts are run on jetson orin nano. The custom trained model is exported to onnx as suggested by documentation using ultralytics, I have also tried exporting using trtexec but still face the same error.

Thank you for your help.

Inference not proper when deploying with this pipeline

Hello,

I have trained the detectnetV2 model on custom dataset using TAO toolkit. However, when I deploy the model with Isaac ROS Gems pipeline, the output inference is not reflecting the output metrics and the inference that I am seeing while using the TAO pipeline.

Later, I deployed the same model with deepstream and noticed that inference was as per the output metrics which I noticed while training the model.

I tried matching the hyper-parameters in the params.yaml file to match the config file of deepstream which I had used but there was no significant improvement in output inference. I am attaching the output images for i) TAO inference ii) Isaac ROS Gems inference iii) Deepstream inference for reference

Model was trained on 6000 images with 500 epochs which gave average precision(mAP) of ~95%. The config file used for training is same provided with DetectnetV2 for TAO training, no changes were done.

Isaac ROS Gems:

TAO inference:

Deep-stream:

Thanks,
Mayank

Using different model with this pipeline

Hello.

I have a trained YOLO model which I want to run using this object detection pipeline for ROS application. My question is:

Is it possible to run any other models using this pipeline
If yes, what changes do I need to make in encoder, triton node and decoder file
Do I need to make any changes from Nitros side or Nitros is generic for any running models

Also, Why this detectnetV2 model is not possible to run on TensorRT ?

Thanks in advance.
Mayank

Can't visualize output at step 8-9 of quickstart guide

Hi!

I'm going through the steps of the quickstart guide but I'm stuck at steps 8-9. Basically, I don't see anything in the rqt_image_view and the output is the following:

[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-05-16-12-05-37-570212-romolo-3913
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [ros2-1]: process started with pid [3924]
[INFO] [component_container_mt-2]: process started with pid [3926]
[INFO] [isaac_ros_detectnet_visualizer.py-3]: process started with pid [3928]
[INFO] [rqt_image_view-4]: process started with pid [3930]
[ros2-1] stdin is not a terminal device. Keyboard handling disabled.[INFO] [1684231538.076278976] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
[ros2-1] [INFO] [1684231538.076344491] [rosbag2_player]: Set rate to 1
[ros2-1] [INFO] [1684231538.083847767] [rosbag2_player]: Adding keyboard callbacks.
[ros2-1] [INFO] [1684231538.083907681] [rosbag2_player]: Press SPACE for Pause/Resume
[ros2-1] [INFO] [1684231538.083928386] [rosbag2_player]: Press CURSOR_RIGHT for Play Next Message
[ros2-1] [INFO] [1684231538.083942719] [rosbag2_player]: Press CURSOR_UP for Increase Rate 10%
[ros2-1] [INFO] [1684231538.083956176] [rosbag2_player]: Press CURSOR_DOWN for Decrease Rate 10%
[ros2-1] [INFO] [1684231538.084578303] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
[ros2-1] [INFO] [1684231543.537188336] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
[ros2-1] [INFO] [1684231548.992522895] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
.
.
.

What could be the problem?

Thanks

Isuue with the sizes of bounding boxes

Hey! so when I publish images by the size 640x640 everything seems to be OK.
But when I publish an image with dimensions 1280x720 the bounding boxes get "smooshed" they don't fit not the 640x640 image nor do they fit 1280x720 image.
Of course I'm using the image dimension parameters in the launch file, but is there a hardcover dimensions somewhere? Except for the fact yolo works with 640x640..

In yolov5 issac ros I just used to push the 1280x720 image and it would get resized in the input of the yolonn

Can't Visualize from Isaac Sim

I can easly install and work on Quickstart example but when I try it on Isaac Sim I can't get any output. Firstly, I suspected I didnt install nvidia container but I believe that is not the problem. Not sure...

I also faced the same problem on VSLAM repo.

Quickstart Example:
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection#quickstart
Isaac Sim Example:
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_object_detection/blob/main/docs/tutorial-isaac-sim.md#tutorial-walkthrough

Any suggestions?

[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-02-20-10-38-19-647207-openzeka-test-22662
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [22673]
[INFO] [isaac_ros_detectnet_visualizer.py-2]: process started with pid [22675]
[INFO] [rqt_image_view-3]: process started with pid [22677]
[component_container_mt-1] [INFO] [1676889500.028942264] [detectnet_container.detectnet_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_dnn_encoders/lib/libdnn_image_encoder_node.so
[component_container_mt-1] [INFO] [1676889500.039107248] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1676889500.039243115] [detectnet_container.detectnet_container]: Found class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::DnnImageEncoderNode
[component_container_mt-1] [INFO] [1676889500.039274293] [detectnet_container.detectnet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::DnnImageEncoderNode
[component_container_mt-1] [INFO] [1676889500.041487616] [dnn_image_encoder]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1676889500.041768563] [dnn_image_encoder]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1676889500.041780112] [dnn_image_encoder]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1676889500.042874622] [dnn_image_encoder]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1676889500.042890917] [dnn_image_encoder]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1676889500.044003013] [dnn_image_encoder]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1676889500.044019705] [dnn_image_encoder]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1676889500.044235413] [dnn_image_encoder]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1676889500.044326936] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/std/libgxf_std.so
[component_container_mt-1] [INFO] [1676889500.046043117] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1676889500.047111142] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1676889500.048479167] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/tensorops/libgxf_tensorops.so
[component_container_mt-1] [INFO] [1676889500.052016290] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/libgxf_message_compositor.so
[component_container_mt-1] [INFO] [1676889500.052427077] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1676889500.052574387] [dnn_image_encoder]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1676889500.055342293] [dnn_image_encoder]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1676889500.166559999] [dnn_image_encoder]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1676889500.168850362] [dnn_image_encoder]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1676889500.175614666] [dnn_image_encoder]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/dnn_image_encoder' in container '/detectnet_container/detectnet_container'
[component_container_mt-1] [INFO] [1676889500.176942643] [detectnet_container.detectnet_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_triton/lib/libisaac_ros_triton_node.so
[component_container_mt-1] [INFO] [1676889500.177786261] [detectnet_container.detectnet_container]: Found class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::TritonNode
[component_container_mt-1] [INFO] [1676889500.177799764] [detectnet_container.detectnet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::TritonNode
[component_container_mt-1] [INFO] [1676889500.179776011] [triton_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1676889500.180062364] [triton_node]: [TritonNode] Set input data format to: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1676889500.180071966] [triton_node]: [TritonNode] Set output data format to: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1676889500.180106136] [triton_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1676889500.180110576] [triton_node]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1676889500.181218905] [triton_node]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1676889500.181235782] [triton_node]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1676889500.182165595] [triton_node]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1676889500.182175651] [triton_node]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1676889500.182290012] [triton_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1676889500.182479427] [triton_node]: [NitrosContext] Loading extension: gxf/triton/libgxf_triton_ext.so
[component_container_mt-1] [INFO] [1676889500.214162780] [triton_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1676889500.215535200] [triton_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1676889500.286901187] [triton_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1676889500.295080111] [triton_node]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1676889500.295516115] [triton_node]: [NitrosPublisherSubscriberGroup] Pinning the component "triton_request/input" (type="nvidia::gxf::DoubleBufferReceiver") to use its compatible format only: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1676889500.297590624] [triton_node]: [NitrosPublisherSubscriberGroup] Pinning the component "vault/vault" (type="nvidia::gxf::Vault") to use its compatible format only: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1676889500.297927094] [triton_node]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1676889500.298056136] [dnn_image_encoder]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/triton_node' in container '/detectnet_container/detectnet_container'
[component_container_mt-1] [INFO] [1676889500.299277928] [detectnet_container.detectnet_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_detectnet/lib/libdetectnet_decoder_node.so
[component_container_mt-1] [INFO] [1676889500.301595109] [detectnet_container.detectnet_container]: Found class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::detectnet::DetectNetDecoderNode
[component_container_mt-1] [INFO] [1676889500.301623949] [detectnet_container.detectnet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::detectnet::DetectNetDecoderNode
[component_container_mt-1] [INFO] [1676889500.303967386] [detectnet_decoder_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1676889500.305013886] [detectnet_decoder_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1676889500.305031625] [detectnet_decoder_node]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1676889500.306238589] [detectnet_decoder_node]: [NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1676889500.306258321] [detectnet_decoder_node]: [NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1676889500.306452578] [detectnet_decoder_node]: [NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1676889500.306461690] [detectnet_decoder_node]: [NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1676889500.306464947] [detectnet_decoder_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1676889500.306712610] [detectnet_decoder_node]: [NitrosContext] Loading extension: gxf/libgxf_detectnet.so
[component_container_mt-1] [INFO] [1676889500.307096282] [detectnet_decoder_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1676889500.307734685] [detectnet_decoder_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1676889500.308943725] [detectnet_decoder_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1676889500.309091762] [detectnet_decoder_node]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1676889500.312390116] [detectnet_decoder_node]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1676889500.312611238] [triton_node]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/detectnet_decoder_node' in container '/detectnet_container/detectnet_container'
[component_container_mt-1] [INFO] [1676889500.312631831] [triton_node]: Could not negotiate
[component_container_mt-1] [INFO] [1676889501.175848196] [dnn_image_encoder]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1676889501.175903489] [dnn_image_encoder]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1676889501.175915430] [dnn_image_encoder]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1676889501.175922878] [dnn_image_encoder]: [NitrosSubscriber] Negotiation failed
[component_container_mt-1] [INFO] [1676889501.175928520] [dnn_image_encoder]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/rgb_left", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1676889501.175961419] [dnn_image_encoder]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1676889501.184186750] [dnn_image_encoder]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/isaac_ros-dev/install/isaac_ros_dnn_encoders/share/isaac_ros_dnn_encoders/KJXJSQBGVX.yaml"
[component_container_mt-1] [INFO] [1676889501.184245436] [dnn_image_encoder]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1676889501.184697404] [dnn_image_encoder]: In DNN Image Encoder Node preLoadGraphCallback().
[component_container_mt-1] [INFO] [1676889501.184739201] [dnn_image_encoder]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1676889501.184751907] [dnn_image_encoder]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_dnn_encoders/share/isaac_ros_dnn_encoders/KJXJSQBGVX.yaml'
[component_container_mt-1] [INFO] [1676889501.186930498] [dnn_image_encoder]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1676889501.187011516] [dnn_image_encoder]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1676889501.187019191] [dnn_image_encoder]: In DNN Image Encoder Node postLoadGraphCallback().
[component_container_mt-1] [INFO] [1676889501.187069564] [dnn_image_encoder]: [NitrosContext] Initializing applicaiton...
[component_container_mt-1] [INFO] [1676889501.295121372] [dnn_image_encoder]: [NitrosContext] Running appliation...
[component_container_mt-1] [INFO] [1676889501.298106752] [triton_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1676889501.298136009] [triton_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1676889501.298144950] [triton_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1676889501.298160580] [triton_node]: [NitrosPublisher] Negotiation failed
[component_container_mt-1] [INFO] [1676889501.298167583] [triton_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/tensor_sub", data_format="nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1676889501.298176600] [triton_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1676889501.312571806] [detectnet_decoder_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1676889501.312622392] [detectnet_decoder_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1676889501.312630144] [detectnet_decoder_node]: [NitrosPublisher] Negotiation failed
[component_container_mt-1] [INFO] [1676889501.312637144] [detectnet_decoder_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/detectnet/detections", data_format="nitros_detection2_d_array"
[component_container_mt-1] [INFO] [1676889501.312645118] [detectnet_decoder_node]: [NitrosSubscriber] Negotiation failed
[component_container_mt-1] [INFO] [1676889501.312652408] [detectnet_decoder_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/tensor_sub", data_format="nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1676889501.312672077] [detectnet_decoder_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1676889501.313789960] [triton_node]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/isaac_ros-dev/install/isaac_ros_triton/share/isaac_ros_triton/HXZGBBNAWY.yaml"
[component_container_mt-1] [INFO] [1676889501.313819128] [triton_node]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1676889501.313823607] [triton_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1676889501.313828947] [triton_node]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_triton/share/isaac_ros_triton/HXZGBBNAWY.yaml'
[component_container_mt-1] [INFO] [1676889501.313997528] [detectnet_decoder_node]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/EHAGNYUTRS.yaml"
[component_container_mt-1] [INFO] [1676889501.314016751] [detectnet_decoder_node]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1676889501.314025644] [detectnet_decoder_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1676889501.315185499] [triton_node]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1676889501.315191026] [detectnet_decoder_node]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/EHAGNYUTRS.yaml'
[component_container_mt-1] [INFO] [1676889501.316175337] [detectnet_decoder_node]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1676889501.316221092] [detectnet_decoder_node]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1676889501.316221089] [triton_node]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1676889501.316245585] [triton_node]: In TritonNode postLoadGraphCallback().
[component_container_mt-1] [INFO] [1676889501.316354525] [triton_node]: [NitrosContext] Initializing applicaiton...
[component_container_mt-1] WARNING: infer_trtis_server.cpp:1212 NvDsTritonServerInit suggest to set model_control_mode:none. otherwise may cause unknow issues.
[component_container_mt-1] I0220 10:38:21.375859 22673 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f49ac000000' with size 268435456
[component_container_mt-1] I0220 10:38:21.376048 22673 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
[component_container_mt-1] I0220 10:38:21.376860 22673 server.cc:563]
[component_container_mt-1] +------------------+------+
[component_container_mt-1] | Repository Agent | Path |
[component_container_mt-1] +------------------+------+
[component_container_mt-1] +------------------+------+
[component_container_mt-1]
[component_container_mt-1] I0220 10:38:21.376875 22673 server.cc:590]
[component_container_mt-1] +---------+------+--------+
[component_container_mt-1] | Backend | Path | Config |
[component_container_mt-1] +---------+------+--------+
[component_container_mt-1] +---------+------+--------+
[component_container_mt-1]
[component_container_mt-1] I0220 10:38:21.376889 22673 server.cc:633]
[component_container_mt-1] +-------+---------+--------+
[component_container_mt-1] | Model | Version | Status |
[component_container_mt-1] +-------+---------+--------+
[component_container_mt-1] +-------+---------+--------+
[component_container_mt-1]
[component_container_mt-1] I0220 10:38:21.389281 22673 metrics.cc:864] Collecting metrics for GPU 0: NVIDIA RTX A4500
[component_container_mt-1] I0220 10:38:21.389441 22673 metrics.cc:757] Collecting CPU metrics
[component_container_mt-1] I0220 10:38:21.389527 22673 tritonserver.cc:2264]
[component_container_mt-1] +----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[component_container_mt-1] | Option | Value |
[component_container_mt-1] +----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[component_container_mt-1] | server_id | triton |
[component_container_mt-1] | server_version | 2.26.0 |
[component_container_mt-1] | server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics trace logging |
[component_container_mt-1] | model_repository_path[0] | /tmp/models |
[component_container_mt-1] | model_control_mode | MODE_EXPLICIT |
[component_container_mt-1] | strict_model_config | 1 |
[component_container_mt-1] | rate_limit | OFF |
[component_container_mt-1] | pinned_memory_pool_byte_size | 268435456 |
[component_container_mt-1] | cuda_memory_pool_byte_size{0} | 67108864 |
[component_container_mt-1] | response_cache_byte_size | 0 |
[component_container_mt-1] | min_supported_compute_capability | 6.0 |
[component_container_mt-1] | strict_readiness | 1 |
[component_container_mt-1] | exit_timeout | 30 |
[component_container_mt-1] +----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[component_container_mt-1]
[component_container_mt-1] [INFO] [1676889501.389797883] [triton_node]: [NitrosContext] Running appliation...
[component_container_mt-1] [INFO] [1676889501.389999445] [detectnet_decoder_node]: [NitrosContext] Initializing applicaiton...
[component_container_mt-1] [INFO] [1676889501.390286804] [detectnet_decoder_node]: [NitrosContext] Running appliation...
[component_container_mt-1] I0220 10:38:21.391044 22673 model_lifecycle.cc:459] loading: detectnet:1
[component_container_mt-1] I0220 10:38:21.420699 22673 tensorrt.cc:5442] TRITONBACKEND_Initialize: tensorrt
[component_container_mt-1] I0220 10:38:21.420714 22673 tensorrt.cc:5452] Triton TRITONBACKEND API version: 1.10
[component_container_mt-1] I0220 10:38:21.420717 22673 tensorrt.cc:5458] 'tensorrt' TRITONBACKEND API version: 1.10
[component_container_mt-1] I0220 10:38:21.420719 22673 tensorrt.cc:5486] backend configuration:
[component_container_mt-1] {"cmdline":{"auto-complete-config":"false","min-compute-capability":"6.000000","backend-directory":"/opt/tritonserver/backends","default-max-batch-size":"4"}}
[component_container_mt-1] I0220 10:38:21.420981 22673 tensorrt.cc:5591] TRITONBACKEND_ModelInitialize: detectnet (version 1)
[component_container_mt-1] I0220 10:38:21.421832 22673 tensorrt.cc:5640] TRITONBACKEND_ModelInstanceInitialize: detectnet (GPU device 0)
[component_container_mt-1] I0220 10:38:22.031302 22673 logging.cc:49] Loaded engine size: 22 MiB
[component_container_mt-1] I0220 10:38:22.129810 22673 logging.cc:49] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +21, now: CPU 0, GPU 21 (MiB)
[component_container_mt-1] I0220 10:38:22.133307 22673 logging.cc:49] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +704, now: CPU 0, GPU 725 (MiB)
[component_container_mt-1] W0220 10:38:22.133317 22673 logging.cc:46] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See CUDA_MODULE_LOADING in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
[component_container_mt-1] I0220 10:38:22.133620 22673 tensorrt.cc:1556] Created instance detectnet on GPU 0 with stream priority 0
[component_container_mt-1] I0220 10:38:22.133738 22673 model_lifecycle.cc:693] successfully loaded 'detectnet' version 1

No quickstart_config.pbtxt

Hi guys,

I am addressing an issue running the isaac_ros_detectnet tutorial using the example bagfile. In few words, the setup_model script expect the ROS2 package to be in the src subfolder of the isaac_ros workspace.
From the docker version, the src workspace doesn't exist. Do you mind where I am wrong?

Regards,

Huge gxf_log file fills disc space

When running the human detector for a couple days in a row, it will at some point crash because there is not enough disc space on my NVIDIA jetson. This is because Isaac Ros write a huger log file named gxf_log in /tmp. I have been looking for option to disable the logging but haven't found it.

Currently we made some additions so the file gets deleted but we would prefer an option to just turn it off. Are you aware of this logging?

detections topic -- no time listed in header

hey i've been trying to measure the fps of the detentions
and there is no time in the header of the topic /detectnet/detections:

compilation error

Hey , I'm trying to compile the new release.
I have this error after trying to colcon build:

--- stderr: isaac_ros_nitros_correlated_timestamp_type
In file included from /workspaces/isaac_ros-dev/src/isaac_ros_nitros/isaac_ros_nitros_type/isaac_ros_nitros_correlated_timestamp_type/src/nitros_correlated_timestamp.cpp:25:
/workspaces/isaac_ros-dev/src/isaac_ros_nitros/isaac_ros_nitros_type/isaac_ros_nitros_correlated_timestamp_type/include/isaac_ros_nitros_correlated_timestamp_type/nitros_correlated_timestamp.hpp:31:10: fatal error: isaac_ros_nova_interfaces/msg/correlated_timestamp.hpp: No such file or directory
   31 | #include "isaac_ros_nova_interfaces/msg/correlated_timestamp.hpp"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

jetson xavier nx
jetpack 5.0.2

YOLOv8 rebuilding CUDA engine not finishing

When executing the commands from the Quickstart the following comes up and inference never seems to start:

2024-02-27 14:15:34.115 WARN /data/jenkins/ros/workspace/ly_debian-compatible-release-2.1/isaac_ros-dev/ros_ws/src/isaac_ros_dnn_inference/isaac_ros_tensor_rt/gxf/tensor_rt/tensor_rt_inference.cpp@278: Rebuilding CUDA engine /tmp/yolov8s.plan (forced by config). Note: this process may take up to several minutes.

I waited over an hour on an Orin NX without any further messages. As this is probably the TensorRT conversion I also tried exporting to TensorRT with yolo export model=yolov8s.pt format='engine' outside the container, which works just fine. However, using the resulting TensorRT model inside the container did not work as well. Any ideas on this?

Process has died error when Running detectnet_quickstart.launch.py

Running the quickstart guide,

cd /workspaces/isaac_ros-dev && \
  ros2 launch isaac_ros_detectnet isaac_ros_detectnet_quickstart.launch.py

[component_container_mt-2] I0414 00:52:52.763286 1497463 logging.cc:49] Loaded engine size: 21 MiB
[component_container_mt-2] I0414 00:52:52.859469 1497463 logging.cc:49] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +21, now: CPU 0, GPU 21 (MiB)
[component_container_mt-2] I0414 00:52:53.041190 1497463 logging.cc:49] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +579, now: CPU 0, GPU 600 (MiB)
[component_container_mt-2] I0414 00:52:53.088781 1497463 tensorrt.cc:1549] Created instance detectnet on GPU 0 with stream priority 0
[component_container_mt-2] I0414 00:52:53.090651 1497463 model_repository_manager.cc:1352] successfully loaded &apos;detectnet&apos; version 1
[ERROR] [component_container_mt-2]: process has died [pid 1497463, exit code -11, cmd &apos;/opt/ros/humble/install/lib/rclcpp_components/component_container_mt --ros-args -r __node:=detectnet_container -r __ns:=/detectnet_container&apos;].
[ros2-1] [INFO] [1681433575.534087803] [rosbag2_storage]: Opened database &apos;/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3&apos; for READ_ONLY.
[ros2-1] [INFO] [1681433580.993063529] [rosbag2_storage]: Opened database &apos;/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3&apos; for READ_ONLY.

I Modified the quickstart launch file to launch everything before the triton node, and it works OK, just when the triton is added it crashes.
I'm not sure what is causing this error but would love to get this working

AGX Orin Developer kit 32GB, running through docker.

Here is the full terminal output

admin@tom-Orin:/workspaces/isaac_ros-dev$ ros2 launch isaac_ros_detectnet isaac_ros_detectnet_quickstart.launch.py 
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-04-14-12-52-49-222770-tom-Orin-1497443
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [ros2-1]: process started with pid [1497461]
[INFO] [component_container_mt-2]: process started with pid [1497463]
[INFO] [rqt_image_view-3]: process started with pid [1497465]
[component_container_mt-2] [INFO] [1681433569.805349019] [detectnet_container.detectnet_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_dnn_encoders/lib/libdnn_image_encoder_node.so
[component_container_mt-2] [INFO] [1681433569.870840364] [detectnet_container.detectnet_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode>
[component_container_mt-2] [INFO] [1681433569.870967758] [detectnet_container.detectnet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncoderNode>
[component_container_mt-2] [INFO] [1681433569.879228924] [NitrosContext]: [NitrosContext] Creating a new shared context
[component_container_mt-2] [INFO] [1681433569.879439646] [dnn_image_encoder]: [NitrosNode] Initializing NitrosNode
[component_container_mt-2] [INFO] [1681433569.880261766] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-2] [INFO] [1681433569.884371950] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-2] [INFO] [1681433569.887335594] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-2] [INFO] [1681433569.890351175] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-2] [INFO] [1681433569.894516078] [NitrosContext]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml'
[component_container_mt-2] [INFO] [1681433569.895502712] [NitrosContext]: [NitrosContext] Initializing application...
[component_container_mt-2] [INFO] [1681433569.897404682] [NitrosContext]: [NitrosContext] Running application...
[component_container_mt-2] 2023-04-14 12:52:49.897 WARN  gxf/std/program.cpp@456: No system specified. Nothing to do
[component_container_mt-2] [INFO] [1681433569.899179803] [dnn_image_encoder]: [NitrosNode] Starting NitrosNode
[component_container_mt-2] [INFO] [1681433569.899226491] [dnn_image_encoder]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-2] [INFO] [1681433569.901699571] [dnn_image_encoder]: [NitrosNode] Loading built-in extension specs
[component_container_mt-2] [INFO] [1681433569.901749140] [dnn_image_encoder]: [NitrosNode] Loading preset extension specs
[component_container_mt-2] [INFO] [1681433569.903604965] [dnn_image_encoder]: [NitrosNode] Loading extension specs
[component_container_mt-2] [INFO] [1681433569.903646758] [dnn_image_encoder]: [NitrosNode] Loading generator rules
[component_container_mt-2] [INFO] [1681433569.904014633] [dnn_image_encoder]: [NitrosNode] Loading extensions
[component_container_mt-2] [INFO] [1681433569.904918546] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-2] [INFO] [1681433569.907513003] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-2] [INFO] [1681433569.910470439] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container_mt-2] [INFO] [1681433569.914476333] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container_mt-2] [INFO] [1681433569.915804026] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-2] [INFO] [1681433569.916455168] [dnn_image_encoder]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-2] [INFO] [1681433569.922143990] [dnn_image_encoder]: [NitrosNode] Running optimization
[ros2-1] stdin is not a terminal device. Keyboard handling disabled.[INFO] [1681433570.055582545] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
[ros2-1] [INFO] [1681433570.055710354] [rosbag2_player]: Set rate to 1
[ros2-1] [INFO] [1681433570.079987418] [rosbag2_player]: Adding keyboard callbacks.
[ros2-1] [INFO] [1681433570.080092123] [rosbag2_player]: Press SPACE for Pause/Resume
[ros2-1] [INFO] [1681433570.080112379] [rosbag2_player]: Press CURSOR_RIGHT for Play Next Message
[ros2-1] [INFO] [1681433570.080126043] [rosbag2_player]: Press CURSOR_UP for Increase Rate 10%
[ros2-1] [INFO] [1681433570.080136027] [rosbag2_player]: Press CURSOR_DOWN for Decrease Rate 10%
[ros2-1] [INFO] [1681433570.081057572] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
[component_container_mt-2] [INFO] [1681433570.226029964] [dnn_image_encoder]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-2] [INFO] [1681433570.238651685] [dnn_image_encoder]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-2] [INFO] [1681433570.276785009] [dnn_image_encoder]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/dnn_image_encoder' in container '/detectnet_container/detectnet_container'
[component_container_mt-2] [INFO] [1681433570.288955269] [detectnet_container.detectnet_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_triton/lib/libtriton_node.so
[component_container_mt-2] [INFO] [1681433570.299410281] [detectnet_container.detectnet_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TritonNode>
[component_container_mt-2] [INFO] [1681433570.299528842] [detectnet_container.detectnet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TritonNode>
[component_container_mt-2] [INFO] [1681433570.332442276] [triton_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-2] [INFO] [1681433570.337636118] [triton_node]: [TritonNode] Set input data format to: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-2] [INFO] [1681433570.343138026] [triton_node]: [TritonNode] Set output data format to: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-2] [INFO] [1681433570.344135988] [triton_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-2] [INFO] [1681433570.344686649] [triton_node]: [NitrosNode] Loading built-in preset extension specs
[component_container_mt-2] [INFO] [1681433570.347153041] [triton_node]: [NitrosNode] Loading built-in extension specs
[component_container_mt-2] [INFO] [1681433570.348045881] [triton_node]: [NitrosNode] Loading preset extension specs
[component_container_mt-2] [INFO] [1681433570.350364751] [triton_node]: [NitrosNode] Loading extension specs
[component_container_mt-2] [INFO] [1681433570.351515930] [triton_node]: [NitrosNode] Loading generator rules
[component_container_mt-2] [INFO] [1681433570.351936350] [triton_node]: [NitrosNode] Loading extensions
[component_container_mt-2] [INFO] [1681433570.353025129] [triton_node]: [NitrosContext] Loading extension: gxf/triton/libgxf_triton_ext.so
[component_container_mt-2] [INFO] [1681433570.477305419] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.480875597] [triton_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-2] [INFO] [1681433570.482347611] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.483721800] [triton_node]: [NitrosNode] Running optimization
[component_container_mt-2] [INFO] [1681433570.566863106] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.580003040] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.666947486] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.670585185] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.681330503] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.742211628] [triton_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-2] [INFO] [1681433570.757969027] [triton_node]: [NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-2] [INFO] [1681433570.758771467] [triton_node]: [NitrosPublisherSubscriberGroup] Pinning the component "triton_request/input" (type="nvidia::gxf::DoubleBufferReceiver") to use its compatible format only: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-2] [INFO] [1681433570.769463793] [triton_node]: [NitrosPublisherSubscriberGroup] Pinning the component "vault/vault" (type="nvidia::gxf::Vault") to use its compatible format only: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-2] [INFO] [1681433570.773799962] [triton_node]: [NitrosNode] Starting negotiation...
[component_container_mt-2] [INFO] [1681433570.774273087] [dnn_image_encoder]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/triton_node' in container '/detectnet_container/detectnet_container'
[component_container_mt-2] [INFO] [1681433570.856836051] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.869501772] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.872067172] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.883145294] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.970848787] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.973194506] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433570.984249107] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.071758583] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.075300792] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.085391833] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.172460344] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.176520735] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.186378365] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.273510780] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.277498402] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set.
[component_container_mt-2] [INFO] [1681433571.279731160] [dnn_image_encoder]: [NitrosNode] Starting post negotiation setup
[component_container_mt-2] [INFO] [1681433571.279803512] [dnn_image_encoder]: [NitrosNode] Getting data format negotiation results
[component_container_mt-2] [INFO] [1681433571.279835353] [dnn_image_encoder]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-2] [INFO] [1681433571.279854521] [dnn_image_encoder]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-2] [INFO] [1681433571.279864537] [dnn_image_encoder]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/image", data_format="nitros_image_bgr8"
[component_container_mt-2] [INFO] [1681433571.279947706] [dnn_image_encoder]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-2] [INFO] [1681433571.311964363] [dnn_image_encoder]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/isaac_ros-dev/install/isaac_ros_dnn_encoders/share/isaac_ros_dnn_encoders/QNZTXRSQFA.yaml"
[component_container_mt-2] [INFO] [1681433571.312086156] [dnn_image_encoder]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-2] [INFO] [1681433571.312109357] [dnn_image_encoder]: In DNN Image Encoder Node preLoadGraphCallback().
[component_container_mt-2] [INFO] [1681433571.312189485] [dnn_image_encoder]: [NitrosNode] Loading application
[component_container_mt-2] [INFO] [1681433571.312412848] [dnn_image_encoder]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_dnn_encoders/share/isaac_ros_dnn_encoders/QNZTXRSQFA.yaml'
[component_container_mt-2] [INFO] [1681433571.320495101] [dnn_image_encoder]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-2] [INFO] [1681433571.320842880] [dnn_image_encoder]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-2] [INFO] [1681433571.320941217] [dnn_image_encoder]: In DNN Image Encoder Node postLoadGraphCallback().
[component_container_mt-2] [INFO] [1681433571.321163651] [dnn_image_encoder]: [NitrosContext] Initializing application...
[component_container_mt-2] [INFO] [1681433571.695553913] [dnn_image_encoder]: [NitrosContext] Running application...
[component_container_mt-2] [INFO] [1681433571.776617086] [triton_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-2] [INFO] [1681433571.776729663] [triton_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-2] [INFO] [1681433571.776754080] [triton_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-2] [INFO] [1681433571.776791232] [triton_node]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-2] [INFO] [1681433571.776804832] [triton_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/tensor_sub", data_format="nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-2] [INFO] [1681433571.776822880] [triton_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-2] [INFO] [1681433571.800834437] [triton_node]: [NitrosNode] Wrote the final top level YAML graph to "/workspaces/isaac_ros-dev/install/isaac_ros_triton/share/isaac_ros_triton/YWFDOSEWPI.yaml"
[component_container_mt-2] [INFO] [1681433571.800951526] [triton_node]: [NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-2] [INFO] [1681433571.800968007] [triton_node]: [NitrosNode] Loading application
[component_container_mt-2] [INFO] [1681433571.800990535] [triton_node]: [NitrosContext] Loading application: '/workspaces/isaac_ros-dev/install/isaac_ros_triton/share/isaac_ros_triton/YWFDOSEWPI.yaml'
[component_container_mt-2] 2023-04-14 12:52:51.803 WARN  gxf/std/yaml_file_loader.cpp@952: Using unregistered parameter 'dummy_rx' in component 'requester'.
[component_container_mt-2] [INFO] [1681433571.803462142] [triton_node]: [NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-2] [INFO] [1681433571.803578208] [triton_node]: [NitrosNode] Calling user's post-load-graph callback
[component_container_mt-2] [INFO] [1681433571.803666720] [triton_node]: In TritonNode postLoadGraphCallback().
[component_container_mt-2] [INFO] [1681433571.803817218] [triton_node]: [NitrosContext] Initializing application...
[component_container_mt-2] WARNING: NvDsTritonServerInit suggest to set model_control_mode:none. otherwise may cause unknow issues.
[component_container_mt-2] I0414 00:52:51.828818 1497463 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x256cd3000' with size 67108864
[component_container_mt-2] I0414 00:52:51.829102 1497463 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
[component_container_mt-2] I0414 00:52:51.848859 1497463 server.cc:559] 
[component_container_mt-2] +------------------+------+
[component_container_mt-2] | Repository Agent | Path |
[component_container_mt-2] +------------------+------+
[component_container_mt-2] +------------------+------+
[component_container_mt-2] 
[component_container_mt-2] I0414 00:52:51.855579 1497463 server.cc:586] 
[component_container_mt-2] +---------+------+--------+
[component_container_mt-2] | Backend | Path | Config |
[component_container_mt-2] +---------+------+--------+
[component_container_mt-2] +---------+------+--------+
[component_container_mt-2] 
[component_container_mt-2] I0414 00:52:51.857960 1497463 server.cc:629] 
[component_container_mt-2] +-------+---------+--------+
[component_container_mt-2] | Model | Version | Status |
[component_container_mt-2] +-------+---------+--------+
[component_container_mt-2] +-------+---------+--------+
[component_container_mt-2] 
[component_container_mt-2] I0414 00:52:51.861923 1497463 tritonserver.cc:2176] 
[component_container_mt-2] +----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[component_container_mt-2] | Option                           | Value                                                                                                                                                                                        |
[component_container_mt-2] +----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[component_container_mt-2] | server_id                        | triton                                                                                                                                                                                       |
[component_container_mt-2] | server_version                   | 2.24.0                                                                                                                                                                                       |
[component_container_mt-2] | server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics trace |
[component_container_mt-2] | model_repository_path[0]         | /workspaces/models                                                                                                                                                                           |
[component_container_mt-2] | model_control_mode               | MODE_EXPLICIT                                                                                                                                                                                |
[component_container_mt-2] | strict_model_config              | 1                                                                                                                                                                                            |
[component_container_mt-2] | rate_limit                       | OFF                                                                                                                                                                                          |
[component_container_mt-2] | pinned_memory_pool_byte_size     | 67108864                                                                                                                                                                                     |
[component_container_mt-2] | cuda_memory_pool_byte_size{0}    | 67108864                                                                                                                                                                                     |
[component_container_mt-2] | response_cache_byte_size         | 0                                                                                                                                                                                            |
[component_container_mt-2] | min_supported_compute_capability | 5.3                                                                                                                                                                                          |
[component_container_mt-2] | strict_readiness                 | 1                                                                                                                                                                                            |
[component_container_mt-2] | exit_timeout                     | 30                                                                                                                                                                                           |
[component_container_mt-2] +----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[component_container_mt-2] 
[component_container_mt-2] [INFO] [1681433571.870891394] [triton_node]: [NitrosContext] Running application...
[component_container_mt-2] I0414 00:52:51.875276 1497463 model_repository_manager.cc:1206] loading: detectnet:1
[component_container_mt-2] I0414 00:52:51.933236 1497463 tensorrt.cc:5427] TRITONBACKEND_Initialize: tensorrt
[component_container_mt-2] I0414 00:52:51.939292 1497463 tensorrt.cc:5437] Triton TRITONBACKEND API version: 1.10
[component_container_mt-2] I0414 00:52:51.939886 1497463 tensorrt.cc:5443] 'tensorrt' TRITONBACKEND API version: 1.10
[component_container_mt-2] I0414 00:52:51.940710 1497463 tensorrt.cc:5486] backend configuration:
[component_container_mt-2] {"cmdline":{"auto-complete-config":"false","min-compute-capability":"5.300000","backend-directory":"/opt/tritonserver/backends","default-max-batch-size":"4"}}
[component_container_mt-2] I0414 00:52:51.941752 1497463 tensorrt.cc:5538] TRITONBACKEND_ModelInitialize: detectnet (version 1)
[component_container_mt-2] I0414 00:52:51.944005 1497463 tensorrt.cc:5587] TRITONBACKEND_ModelInstanceInitialize: detectnet (GPU device 0)
[component_container_mt-2] I0414 00:52:52.763286 1497463 logging.cc:49] Loaded engine size: 21 MiB
[component_container_mt-2] I0414 00:52:52.859469 1497463 logging.cc:49] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +21, now: CPU 0, GPU 21 (MiB)
[component_container_mt-2] I0414 00:52:53.041190 1497463 logging.cc:49] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +579, now: CPU 0, GPU 600 (MiB)
[component_container_mt-2] I0414 00:52:53.088781 1497463 tensorrt.cc:1549] Created instance detectnet on GPU 0 with stream priority 0
[component_container_mt-2] I0414 00:52:53.090651 1497463 model_repository_manager.cc:1352] successfully loaded 'detectnet' version 1
[ERROR] [component_container_mt-2]: process has died [pid 1497463, exit code -11, cmd '/opt/ros/humble/install/lib/rclcpp_components/component_container_mt --ros-args -r __node:=detectnet_container -r __ns:=/detectnet_container'].
[ros2-1] [INFO] [1681433575.534087803] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
[ros2-1] [INFO] [1681433580.993063529] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.
[ros2-1] [INFO] [1681433586.449840448] [rosbag2_storage]: Opened database '/workspaces/isaac_ros-dev/install/isaac_ros_detectnet/share/isaac_ros_detectnet/detectnet_rosbag/detectnet_rosbag_0.db3' for READ_ONLY.

DetectNet freezes and stops publishing

We have been very happy with your Isaac repo, but yesterday I ran into a small problem. After running the object detector for a couple of minutes the publishing freezes and nothing is published to the topic. If you then a ros command like (ros2 topic list, topic echo, etc) it continues again for a couple of minutes and freezes at some point. By calling a ros command you seem to trigger something and make it restart.

Nothing is publishes to the encoder so there is also nothing at the topic /detectnet/detections, but it seems to already stagnate at /tensor_pub. We are testing with a realsense camera and the image topic of the realsense keeps working, so that doesn't seem to be the problem.

Any idea what could be going wrong?