Comments (12)
Hi, sorry for the super late reply. I did try @mrharicot 's suggestion, and was able to successfully convert the encoder over to TensorRT. But reflection padding is not supported in TensorRT (only zero-padding is included), so I was unable to convert the decoder. Since I'm busy with other components of our project, I didn't pursue this, but I suppose it wouldn't be overly difficult to add support for this in TensorRT and be able to convert the decoder model.
But as quick fix, changing the mode from "bilinear"
to "nearest"
in the call to torch.nn.functional.interpolate()
in test_simple.py (line 127-128) fixed it on the Nano:
disp_resized = torch.nn.functional.interpolate(disp, (original_height, original_width), mode="nearest")
I didn't notice any glaring differences between the disparity images produced from these modes (I tested with "bilinear" on my personal laptop), but my testing was not very extensive. Using pytorch version 1.1.0 and torchvision version 0.3.0, the Nano took about 0.5 seconds per image. When running on a webcam, we were able to achieve roughly 4fps.
Hopefully this helps someone avoid a lot of headache in trying to get this running on the Jetson Nano.
from monodepth2.
To add to michael's answer, you might want to consider converting the model to TensorRT.
https://github.com/onnx/onnx-tensorrt
It will likely fail due to the use of an upsample bilinear, but you could replace them with nearest neighbor upsamples and fine tune the models a bit.
from monodepth2.
Thanks for your interest in this repo, and thanks for the interesting issue!
We have never seen this problem on our machines here, though we are using more 'traditional' GPUs.
There is an issue here with pytorch which seems to be similar. There they are using the Nvidia TX-2, which seems to be similar to the Jetson Nano.Unfortunately the solutions proposed there seem to involve modifying the pytorch source code, which is a bit of a pain.
from monodepth2.
@mdfirman Hi, I'm still working on the suggestion @mrharicot provided. I'm pretty new to this stuff so it may take me awhile, but feel free to close this and I'll get back when I gotten more done. Thanks.
from monodepth2.
Thank you for the suggestions!
@mdfirman I had already seen that issue, but the problem with that is the file in question doesn't seem to be on my machine anywhere (I was running into separate errors with building pytorch from source, so I used the prebuilt wheels provided at the link).
@mrharicot I'll be sure to look into that.
from monodepth2.
@arjunkeerthi thanks, please do report back here so that other people using this repo on a Jetson don't run into the same problem.
from monodepth2.
Super, thank you for the update!
from monodepth2.
Hello!
I have the same problem, I got a jetson nano as well and I cannnot find the
SpatialUpSamplingBilinear.cu as well, is there any solution for it? thx!
from monodepth2.
Hi,
It seems that using nearest neighbor upsampling is the best way to go here.
But as quick fix, changing the mode from
"bilinear"
to"nearest"
in the call totorch.nn.functional.interpolate()
in test_simple.py (line 127-128) fixed it on the Nano.
from monodepth2.
Hi,
thank you for the advice, but could you please show your torch version pls?
when I switch 'bilinear' to 'nearest' I got such an error:
ValueError: align_corners option can only be set with the interpolating modes: linear | bilinear | bicubic | trilinear
thanks!
from monodepth2.
Hi @remindchobits, sorry I forgot to add in my comment above that when switching to nearest neighbor, you need to take out the last argument (align_corners=True) since it can only be used with the modes listed in the error message given.
Edit: I added the full line to my previous comment.
from monodepth2.
Hi, @arjunkeerthi
Thank you! it works :)
from monodepth2.
Related Issues (20)
- onnx
- What are the units in which the results are predicted HOT 2
- A problem when I train my repo code HOT 1
- Can't run the initial training
- Network inference time problem
- The requested array has an inhomogeneous shape after 1 dimensions. HOT 5
- Write split file HOT 1
- obtained some very strange depth maps HOT 4
- The difference in the intrinsic matrix affects the results
- Question about image resolution
- question for the Data Preparation
- Why is smooth_loss divided by 2**scale?
- Questions about the meaning of grid in the F.grid_sample function
- the eval file about 'gt_depths.npz'
- Run on Google Colab,
- Run on Google Colab, but out of System RAM
- The new constraint about pose is not useful?
- RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS HOT 2
- How to obtain reconstructed image and loss for a single demo image
- How to setup already trained computer vision model Ultralytics YOLOv8 with monodepth2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from monodepth2.