SDK for jetpack 4.4 on Xavier NX?

[stereolabs]

SDK for jetpack 4.4 on Xavier NX?

[obraun-sl]

The boot issue was fixed in JP42/43 but it seems back on JP44. I'll report to NVIDIA to see what they can do.
In the meantime, there is a software workaround. The NX DevKit has a realtek usb hub for 3.0 connection, and it can be reset by software once NX has booted up.

Here is how to do :

• download the zip attached and extract it.
• sh build.sh to build the executable.
• $ sudo ./power_cycle (it's similar to unplug/plug)
  Then lsusb should show f682 and f681.

USBHubReset.zip

[obraun-sl]

Hi,
The next SDK release (3.2) will have the Jetpack 4.4 support (and therefore the NX support).
The release is planned for end of the week or beginning of next week.

[Tetsujinfr]

Ok great, thanks for the quick reply. Looking fwd for the release. Merci.

[Tetsujinfr]

Can I use my Zed Mini on Jetpack 4.4 simply with cheese or v4l2? So far when I plug the camera into the USB port it does not even get recognised as a video device with v4l2? Does the camera works solely with the SDK API?

[obraun-sl]

No the camera is a UVC device, therefore it can work with any v4l2 compatible software. (cheese or guvcview...)
Make sure that the camera is detected under the USB list as :
2b03:f682 (for ZED Mini)

[Tetsujinfr]

I do not see the device on the device list unfortunately. The camera works well on my PC though. The xavier nx has USB 3.1 ports, could that be the problem? Have you tested the Zed mini on a xavier nx devkit already?

[obraun-sl]

It should work with the USB3.1 port as well. Most of our tests were done with the ZED2 but it should behave the same way.
Can you post the complete lsusb output ?

[Tetsujinfr]

here is my lsusb output:

tetsfr@tets:~$ lsusb
Bus 002 Device 002: ID 0bda:0489 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 13d3:3549 IMC Networks
Bus 001 Device 004: ID 04d9:a01c Holtek Semiconductor, Inc.
Bus 001 Device 005: ID 2b03:f681
Bus 001 Device 002: ID 0bda:5489 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Is ID 2b03:f681 supposed to be the ZED Mini?

What library do I need to install on the jetson on top of v4l2?
(when launching cheese it says "No device" found, and no more luck when using VLC Open capture device)

[obraun-sl]

the ZED Mini contains 2 usb devices :

• 2b03:f680/2 : camera device (USB3.0) (1 typeC side only)
• 2b03:f681 : sensors device (USB2.0) ( both type C side)

If you only see the f681 device, try to flip the USB type C cable so that the arrow match the LED side on the camera.

[Tetsujinfr]

So the arrow was already on the LED side of the camera USB-C connector.
I just tried the camera on my Windows PC and it works just fine, and the LED is blue when connecting.
Now on the Xavier NX with Jetpack 4.4, the LED does NOT turn blue, which is likely not good.
Could that be a power draw issue, even if lsusb can read the USB ID 2b03:f681 ?

Did you actually tried/tested the ZED Mini with a Xavier NX DevKit on your side? If yes, what is the outcome? If no, can you please test it and share your findings?

thanks a lot

[Tetsujinfr]

Ok actually I never noticed that the LED is blue only when the camera is connected AND the ZED software are open running, so the LED is controlled via software on the computer side, so basically it is not very informative to know if there is a pb or not.

After looking at this thread, I ran dmesg -Tw, I got the following results:

[Mon Jun 1 15:05:17 2020] usb 1-2.1: new full-speed USB device number 12 using tegra-xusb
[Mon Jun 1 15:05:17 2020] usb 1-2.1: New USB device found, idVendor=2b03, idProduct=f681
[Mon Jun 1 15:05:17 2020] usb 1-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[Mon Jun 1 15:05:17 2020] usb 1-2.1: Product: ZED-M Hid Device
[Mon Jun 1 15:05:17 2020] usb 1-2.1: Manufacturer: STEREOLABS
[Mon Jun 1 15:05:17 2020] usb 1-2.1: SerialNumber: 10541780
[Mon Jun 1 15:05:22 2020] hid-generic 0003:2B03:F681.0009: hidraw0: USB HID v1.11 Device [STEREOLABS ZED-M Hid Device] on usb-3610000.xhci-2.1/input0

So it looks like the camera is kind of recognised on the USB side, but that it is not behaving as a proper UVC device on the Xavier NX DevKit with Jetpack 4.4.

Could that be a firmware issue on your side? Sorry to ask again but did you test the ZED mini on the Xavier NX DevKit?

Thanks

[obraun-sl]

I have a NX prod with a nano carrier board. Here is my lsusb output :

user@sl-dev-nxfoo:/usr/local/zed/tools$ lsusb
Bus 002 Device 019: ID 2b03:f682
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 009: ID 2b03:f681
Bus 001 Device 005: ID 03f0:034a Hewlett-Packard Elite Keyboard
Bus 001 Device 004: ID 1e7d:2dbe ROCCAT
Bus 001 Device 003: ID 2109:2813 VIA Labs, Inc.
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

I have the f682 (camera) and f681 (sensors) available, and I can read the camera.
The only difference I can see is that the NxDevKit has a carrier board slightly different from the Nano, on the USB3 side (Nx DevKit is USB3.1 and Nano is USB3.0).
Maybe this difference cause the issue.
I will try to find a NX devKit to test.

[Tetsujinfr]

Is the Zed subject to issues when plugged on a USB 3.1 port? I mean, I have a set of 2$ super cheap webcams and they work right away on the jetson/PC/usb2/3/3.1 basically on anything (but their quality is horrendous obviously).
But when I can not get the 400$ ZedM to work, I find it strange. Is there an issue with the firmware somehow?? I bought this camera on pupose for the Xavier NX so the situation is a bit frustrating.
I mean if you tell me the SDK supporting jetpack 4.4 is going to fix everything, I ll take that and try it but I did not see it released yet ... help please.

[obraun-sl]

Did you try to unplug/plug again the ZED-M once the NX has boot?

[Tetsujinfr]

Yes I tries to plug it before power on, try to plug and unplug while poweroff. No chance. Again, I do not have any issue with my other cheap webcams I have at hand (which are all USB2 bandwith webcams).

Are you planning on testing with a Xavjer NX devkit?
Is the new SDK for jetpack 4.4. going to help and when?
Is there a Zed dev forum where I can ask the dev community if someone tested the combination Zed-M + Xavier NX devkit successfully? (I have struggled to find forums to talk to people about it hence when I ended up logging this issue here)

[Subliminaltic]

Posted 13 days ago:

Hi,
The next SDK release (3.2) will have the Jetpack 4.4 support (and therefore the NX support).
The release is planned for end of the week or beginning of next week.

My comment:
Any news on ZED SDK release 3.2 that works with Jetpack 4.4 on Xavier NX (and was planned for release last week)?

[jack-flack]

I also have a zed mini & Xavier nx. Have you tried using their zed_explorer program? It works without fuss for me. Changing their sdk makefiles to look for cuda 10.2 may also help you out.

[Tetsujinfr]

Oh thanks for your input here, gives me hope.
So which sdk do you use on the NX? 3.1.2?
Was your ZedM recognized as a std camera (like hot plug then launch cheese and works like any webcam) or was it only working with zed_explorer/Zed_fu/etc? Trying to figure out if the hardware is only available through proprietary software. Thanks a lot

[Tetsujinfr]

Oh and last thing, do you use the Xavier NX devkit or do you use the Xavier NX module + Nano devkit? Thanks

[Subliminaltic]

I also have a zed mini & Xavier nx. Have you tried using their zed_explorer program? It works without fuss for me. Changing their sdk makefiles to look for cuda 10.2 may also help you out.

Thanks for the info, yes zed_explorer works great. None of the other SDK tools work though. On install I got a message stating specifically that the tools would not work with CUDA 10.2, so I made the decision to wait for version 3.2 for Jetpack 4.4.

I have the Xavier NX Dev kit and really look forward to seeing the performance improvement in zed_fu compared to the Nano... I just checked again on the Stereolabs SDK download page and still no new version. I will also buy and try the new ZED2 with the Xavier NX, but I'm not risking that until there's an SDK release built for Jetpack 4.4.

[Tetsujinfr]

Do you see the ZedM camera feed under jetpack 4.4 when using just cheese ?

[obraun-sl]

The ZED SDK 3.2 with Jetpack 4.4 support is available here :
https://www.stereolabs.com/developers/release/

[jack-flack]

Thanks

[Subliminaltic]

Thanks!!

[Tetsujinfr]

Ok great, thanks. So after installing the SDK 3.2 I can use my Zed-M without any issue on my Xavier NXdekKit, brilliant! I can now see the camera with cheese as well. You should have just said that the SDK was necessary to get the camera working, even as a simple video camera, I would just have quietly waited for the SDK release.

Now one new issue though, since I completed the SDK installation my devKit fan is blowing at full speed. Even after rebooting it starts at full speed as soon boot time. Even if I force the fan speed to 0 under /sys/devices/pwm-fan/target_pwm it kicks off again to ful speed after a few seconds.

So I suspect the ZED SDK has some service forcing the fan to run full speed.
I was very happy with the smart fan control of the jetpack install and I am getting quite frustated to hear my new jetfighter turbofan roaring all the time now...

Can you please help fixing this?

ps: if there is a better place to log this issue pls point me to it, I appreciate this issue log here should be marked a closed now that ten SDK 3.2 is out.

[obraun-sl]

@Tetsujinfr In fact, it's not necessary to have the ZED SDK installed to use the camera as a simple UVC device. I am a bit surprised that it's working now, since the USB device was not even detected (no F682) and the SDK should not impact the USB detection.
As I said, the F682 behaves as a standard UVC device, and can be open with any UVC software (cheese, guvcview...).
Maybe it's the nvpmodel set to the maximum (using jetson_clocks) that enables the USB3.0 on the NX. (That's why the fan is turning).

Regarding Max speed, That comes from the maximum performance mode set during the installation (if answered yes). You can remove the jetson clock service:
$ sudo rm /usr/local/zed/tools/jetson_clocks*
$ sudo rm /etc/systemd/system/jetson_clocks*
Then reboot.

[Tetsujinfr]

Your assumption on the nvpmodel is interesting, I ll confirm when I try again experimenting with jetson_clocks deactivated.
Is there a way I can use the max performance mode when using the camera only and returning to managed mode otherwise, without rebooting?

[obraun-sl]

I think it's possible using the jetson_clocks (made by NVIDIA)
just after rebooting, store the current value:
$ /usr/bin/jetson_clocks --store

Then apply max :
$ /usr/bin/jetson_clocks

Use your software

Then restore previous :
$ /usr/bin/jetson_clocks --restore

(Might need to be in sudo)
Taken from here : https://forums.developer.nvidia.com/t/how-do-you-disable-the-jetson-clocks-sh/83973

Another way is to change the nvpmodel to the desired value :
sudo nvpmodel -m <X>
more information here https://www.jetsonhacks.com/2018/10/07/nvpmodel-nvidia-jetson-agx-xavier-developer-kit/ (might need some adjustment for NX)

[Subliminaltic]

Installed ZED SDK 3.2 on the Xavier NX Devkit, no issues.

• All the tools work out-of-the-box.
• The cpp tutorials also compiled and ran with no issues.
  I think you can close this Issue!
  Great job, thanks.

[Tetsujinfr]

Ok , so I have done many tests, with and without the jetson-clocks scripts: the camera works in both cases, so the USB 3.1 works in normal power mgmt mode or in full speed mode either. Great. Now I understand why you trigger the full speed mode, in mormal speed mgmt mode there are some significant slowdowns just when capturing the video feed via opencv, even on the NX, which is a bit surprising but I guess it is what it is.

However, I find out that after reboot, my zed mini is not detected properly and I need to re-plug it so that it gets recognized by the system properly.
That looks similar to this issue thread.
When reading the thread it is not clear to me what the solution to this problem is. You said to use the latest jetpack 4.2 to fix the issue but 1) users have said that did not fix the problem and 2) I am using jetpack 4.4 so I would have thought it would fix the issue if jetpack 4.2 was supposed to fix it on the first place.

Any advice on fixing the "at boot ZedM camera connectivity issue"?
thanks

[Tetsujinfr]

Please note that in normal speed mgmt mode on the NX devkit, ZedFu and ZED_Depth_Viewer have good performance (no slowdown). With Zed_Explorer, very few frames are dropped: 0.07% for WVGA@100fps, 0.2% for 720p@60fps and , 0.1% for 2K@15fps and HD@30fps (I have removed the first cple dropped frames at mode initialization which looks like outliers at startup time).
So I suspect that the perf issues I observed in my previous post were due to poorly optimized python code using opencv.

Still have the USB connection issue at boot, so I need to unplug/plug the camera after each reboot, painful.

[Tetsujinfr]

thanks. so I ran the compilation script, rebooted by devkit, then executed the "power_cycle" program, but unfortunately it throw an error and it did not work:

tetsfr@tets:/media/tetsfr/SSD/download$ ./build.sh
tetsfr@tets:/media/tetsfr/SSD/download$ sudo reboot
tetsfr@tets:/media/tetsfr/SSD/download$ ./power_cycle 
[error]:open device 0bda:5489 fail.
tetsfr@tets:/media/tetsfr/SSD/download$ lsusb
Bus 002 Device 002: ID 0bda:0489 Realtek Semiconductor Corp. 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 13d3:3549 IMC Networks 
Bus 001 Device 005: ID 04d9:a01c Holtek Semiconductor, Inc. wireless multimedia keyboard with trackball [Trust ADURA 17911]
Bus 001 Device 004: ID 2b03:f681  
Bus 001 Device 002: ID 0bda:5489 Realtek Semiconductor Corp. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

2b03:f682 is still missing.

Any idea on what I need to do or change in the source code to get it working?

[Tetsujinfr]

ok that works! I forgot to run the program as root :°)

Would be great if you can get NVdia to fix this in jetpack as you did for 4.2 but I close this issue since all works ok now on my side. The script was useful, thanks.