Comments (5)
Hi! First of all thanks, I'm glad you're able to use the code.
The depth map cutting is still kinda sketchy and I'm trying to figure out a more serious way to do this. The original idea, including the surface power, is already presented in Semerjian 2014. But in general there are two reasons why I want to do this:
- The first idea is that I do not want to export all generated depth values to the surface reconstruction, mainly to speed up computation times. In theory the SMVS reconstruction should already generate a high quality surfaces in a single depth map. So theoretically we just need to decide which depth map reconstructs which part of the object best and export this to the surface reconstruction (with some amount of overlap).
- The second reason is simply to remove outliers. For complex scenes with many occlusions/disocclusions outliers can happen frequently, so I also want to check if the depth I am exporting is consistent in at least one other depth map.
What happens after L127 is broadly the following:
- We have a point from the current depth map (D1) and it's projection into another depth map (D2)
- I first check if D2 has also reconstructed this part of the surface by checking if the point generated by D1 projects into D2 and coincides with the depth value that has been reconstructed there
- At this point two things can happen:
- The depth of D2 is smaller than expected. In this case I cannot say anything because D2 has probably seen a totally different surface which occluded the surface visible in D1
- The depth of D2 is larger than expected. This is where it gets interesting, because D2 has essentially seen "through" the surface that has been reconstructed in D1. So one of the two reconstructed points must be wrong. It is very hard to tell which one, so what I do is I accumulate how many other depth maps have seen the surface from a similar angle (which is what the surface_power_j_j measures (I need a better name for this)) and agree / disagree with the depth value in D1 - if most of them agree I keep the value in D1, otherwise I remove it. If the other depth map has seen the surface from a way better angle (> 2.0 * surface_power) I immediately remove the depth value in D1 because it is most likely wrong.
Does that help?
from smvs.
Thanks so much for a detailed insight into the depth cutting. I figured it was something similar to what you are saying, but it is good to have additional insight and confirmation. Thanks again!
from smvs.
Do you have any suggestions for outputting the depth maps from smvs as images? I have tried to read in the smvs-cut.mvei and smvs-S1.mvei files but I get weird output that looks incomplete and much lower resolution.
from smvs.
This looks kind of reasonable to me, see my mail. Posting it here for reference:
You’re reconstructing mostly specular materials and there’s not a lot of texture on the van. This is basically the worst case scenario for MVS as it is not really robust for this type of data. Other reasons could be missing neighbors or inaccuracies in the initial SfM.
The reason for why it looks so blocky is that the optimization works on larger patches and not on single pixels. You can optimize on smaller patches using the -o
option.
Note that the algorithm in general is tuned for high quality multi-view reconstruction and not simply generating the most complete depth map for each image. If there is a hole it is because we would rather have no geometry than incorrect geometry. This is usually different from something you would do for regular stereo.
from smvs.
Closing. Feel free to reopen if further questions arise.
from smvs.
Related Issues (20)
- string..h missing HOT 1
- What is the difference between simplify option and MeshSimplifier class? HOT 2
- [Question] HOT 1
- Crash after 8 view selection HOT 5
- Images and camera poses as input HOT 8
- General Protocol for Face Scanning for Surgery Planning (Orthognathic and Rhinoplasty) - Comparison between Tools HOT 1
- General Protocol for Face Scanning for Surgery Planning (Orthognathic and Rhinoplasty) - Comparison between Tools
- Camera Response Function HOT 2
- Is there a way to make it more right height like dmrecon? HOT 2
- Question about parameters for Middlebury benchmark HOT 5
- Models are flipped over X axis HOT 3
- [Nice to have] Optionally save in ASCII PLY format
- Is the result of evaluate_3_band reasonable compared with result of evaluate_3_band_exact HOT 2
- Run smvs on several machines in parallel HOT 2
- Dense reconstruction with Blender cameras HOT 1
- Killed while running HOT 4
- Possible Bug: vector out of range in fill_values_at_pixels HOT 7
- shared_ptr issue, mesh generator is tampering the original depth map content
- Question about image hessian HOT 4
- Compile error - ambiguous reference to linear_at() HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from smvs.