manhofer/line3dpp

Line3D++ - Multi-View Stereo using Line Segments

line3dpp's Issues

how to update orthogonal weigh matrix by cayley transformation

I want to update orthogonal weight matrix by Cayley transformation because weight matrix must preserve orthogonality , first I decompose Wight matrix to U,S,V and update U,V separately by using Cayley transformation what about s , How can I update it ? should update S by typical gradient descent ? after theses three component get update I want to composes them like W= USV in order to have updated W

How to reconstruct 3D line feature

Hi, the program runs normally。 But I only have unordered point cloud information, how can I get line segments？

How to remove outliers and point cloud densities？

Thank you in advance for your contribution.
What parameters can be changed to reduce outliers?
What parameters can be changed to increase the density of the point cloud?
Thank you!

can you give me some dataset about urban building？

gpu || out of memory

i run line3D with the sfm results of colmap, like:

./runLine3Dpp_colmap -i ~/project/Line3Dpp/experiments/jrg-0622-44/colmap-pre/image/pic -m ~/project/Line3Dpp/experiments/jrg-0622-44/colmap-pre/sfm_result

the output is :

[L3D++] @GPU: [0001] --> DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... [0002] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! [0003] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! [0004] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! [0005] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! [0006] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! DataArray::strideGPU(): data is _not_ on GPU! DataArray::dataGPU(): data is _not_ on GPU! [0008] DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory] DataArray::upload(): GPU memory not allocated... DataArray::allocateGPU(): GPU memory could not be allocated...[out of memory]

it seems that the image size is too large(7952x5304) to use gpu successfully. could you give me some help?

Errors to build 'set' in namespace 'std' does not name template

I got lots of errors for 'set', visibility, so on Please let me know how to solve this.
~/VR/Line3Dpp/build$ make
[ 5%] Building CXX object CMakeFiles/line3Dpp.dir/line3D.cc.o[ 5%] Building CXX object CMakeFiles/line3Dpp.dir/line3D.cc.o
In file included from /home/chaiein/VR/Line3Dpp/view.h:43:0,
from /home/chaiein/VR/Line3Dpp/line3D.h:48,
from /home/chaiein/VR/Line3Dpp/line3D.cc:1:
/home/chaiein/VR/Line3Dpp/segment3D.h:194:14: error: ‘set’ in namespace ‘std’ does not name a template type
std::set visibility_;
^
/home/chaiein/VR/Line3Dpp/segment3D.h: In member function ‘void L3DPP::FinalPoint3D::serialize(Archive&, unsigned int)’:
/home/chaiein/VR/Line3Dpp/segment3D.h:201:64: error: ‘visibility_’ was not declared in this scope
ar & boost::serialization::make_nvp("visibility_", visibility_);
^
In file included from /home/chaiein/VR/Line3Dpp/line3D.cc:1:0:
/home/chaiein/VR/Line3Dpp/line3D.h: At global scope:
/home/chaiein/VR/Line3Dpp/line3D.h:380:14: error: ‘set’ in namespace ‘std’ does not name a template type
std::set views_reserved_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:392:31: error: ‘set’ is not a member of ‘std’
std::map<unsigned int,std::set > visual_neighbors_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:392:31: error: ‘set’ is not a member of ‘std’
/home/chaiein/VR/Line3Dpp/line3D.h:392:52: error: template argument 2 is invalid
std::map<unsigned int,std::set > visual_neighbors_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:392:52: error: template argument 4 is invalid
/home/chaiein/VR/Line3Dpp/line3D.h:392:54: error: expected unqualified-id before ‘>’ token
std::map<unsigned int,std::set > visual_neighbors_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:400:31: error: ‘set’ is not a member of ‘std’
std::map<unsigned int,std::set > matched_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:400:31: error: ‘set’ is not a member of ‘std’
/home/chaiein/VR/Line3Dpp/line3D.h:400:52: error: template argument 2 is invalid
std::map<unsigned int,std::set > matched_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:400:52: error: template argument 4 is invalid
/home/chaiein/VR/Line3Dpp/line3D.h:400:54: error: expected unqualified-id before ‘>’ token
std::map<unsigned int,std::set > matched_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:431:35: error: ‘set’ is not a member of ‘std’
std::map<L3DPP::Segment2D,std::setL3DPP::Segment2D > used_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:431:35: error: ‘set’ is not a member of ‘std’
/home/chaiein/VR/Line3Dpp/line3D.h:431:60: error: template argument 2 is invalid
std::map<L3DPP::Segment2D,std::setL3DPP::Segment2D > used_;
^
/home/chaiein/VR/Line3Dpp/line3D.h:431:60: error: template argument 4 is invalid
/home/chaiein/VR/Line3Dpp/line3D.h:431:62: error: expected unqualified-id before ‘>’ token
std::map<L3DPP::Segment2D,std::setL3DPP::Segment2D > used_;
^
/home/chaiein/VR/Line3Dpp/line3D.cc: In member function ‘void L3DPP::Line3D::addImage(unsigned int, cv::Mat&, const Matrix3d&, const Matrix3d&, const Vector3d&, float, const std::cxx11::list&, const std::vector<cv::Vec<float, 4> >&)’:
/home/chaiein/VR/Line3Dpp/line3D.cc:130:12: error: ‘views_reserved’ was not declared in this scope
if(views_reserved.find(camID) != views_reserved_.end())
^
/home/chaiein/VR/Line3Dpp/line3D.cc:145:12: error: ‘views_reserved_’ was not declared in this scope
if(views_reserved_.size() == 1)
^
/home/chaiein/VR/Line3Dpp/line3D.cc:212:9: error: ‘visual_neighbors_’ was not declared in this scope
visual_neighbors_[camID] = std::set();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:212:36: error: ‘set’ is not a member of ‘std’
visual_neighbors_[camID] = std::set();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:212:45: error: expected primary-expression before ‘unsigned’
visual_neighbors_[camID] = std::set();
^
/home/chaiein/VR/Line3Dpp/line3D.cc: In member function ‘void L3DPP::Line3D::matchImages(float, float, unsigned int, float, int, float)’:
/home/chaiein/VR/Line3Dpp/line3D.cc:416:9: error: ‘matched_’ was not declared in this scope
matched_.clear();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:469:20: error: ‘visual_neighbors_’ was not declared in this scope
if(visual_neighbors_[camID].size() == 0)
^
/home/chaiein/VR/Line3Dpp/line3D.cc: In member function ‘void L3DPP::Line3D::findVisualNeighborsFromWPs(unsigned int)’:
/home/chaiein/VR/Line3Dpp/line3D.cc:580:12: error: ‘visual_neighbors_’ was not declared in this scope
if(visual_neighbors_.find(camID) != visual_neighbors_.end())
^
/home/chaiein/VR/Line3Dpp/line3D.cc:615:13: error: ‘set’ is not a member of ‘std’
std::set used_neighbors;
^
/home/chaiein/VR/Line3Dpp/line3D.cc:615:22: error: expected primary-expression before ‘unsigned’
std::set used_neighbors;
^
/home/chaiein/VR/Line3Dpp/line3D.cc:674:43: error: ‘used_neighbors’ was not declared in this scope
while(nit!=neighbors.end() && used_neighbors.size() < num_neighbors_)
^
/home/chaiein/VR/Line3Dpp/line3D.cc:682:21: error: ‘set’ is not a member of ‘std’
std::set::const_iterator u_it = used_neighbors.begin();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:682:30: error: expected primary-expression before ‘unsigned’
std::set::const_iterator u_it = used_neighbors.begin();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:684:27: error: ‘u_it’ was not declared in this scope
for(; u_it!=used_neighbors.end() && valid; ++u_it)
^
/home/chaiein/VR/Line3Dpp/line3D.cc:697:40: error: ‘used_neighbors’ was not declared in this scope
visual_neighbors_[camID] = used_neighbors;
^
/home/chaiein/VR/Line3Dpp/line3D.cc: In member function ‘void L3DPP::Line3D::computeMatches()’:
/home/chaiein/VR/Line3Dpp/line3D.cc:704:31: error: ‘set’ is not a member of ‘std’
std::map<unsigned int,std::set >::const_iterator it = visual_neighbors_.begin();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:704:31: error: ‘set’ is not a member of ‘std’
/home/chaiein/VR/Line3Dpp/line3D.cc:704:52: error: template argument 2 is invalid
std::map<unsigned int,std::set >::const_iterator it = visual_neighbors_.begin();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:704:52: error: template argument 4 is invalid
/home/chaiein/VR/Line3Dpp/line3D.cc:704:54: error: expected unqualified-id before ‘>’ token
std::map<unsigned int,std::set >::const_iterator it = visual_neighbors_.begin();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:705:15: error: ‘it’ was not declared in this scope
for(; it!=visual_neighbors_.end(); ++it)
^
/home/chaiein/VR/Line3Dpp/line3D.cc:705:19: error: ‘visual_neighbors_’ was not declared in this scope
for(; it!=visual_neighbors_.end(); ++it)
^
/home/chaiein/VR/Line3Dpp/line3D.cc:719:13: error: ‘set’ is not a member of ‘std’
std::set::const_iterator n_it = it->second.begin();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:719:22: error: expected primary-expression before ‘unsigned’
std::set::const_iterator n_it = it->second.begin();
^
/home/chaiein/VR/Line3Dpp/line3D.cc:720:19: error: ‘n_it’ was not declared in this scope
for(; n_it!=it->second.end(); ++n_it)
^
/home/chaiein/VR/Line3Dpp/line3D.cc:722:20: error: ‘matched_’ was not declared in this scope
if(matched_[it->first].find(*n_it) == matched_[it->first].end())

I need help

Can any friends give me windows excutables, i can't build it successfully on windows 10, please help me! Thank you very much!

Where is the cuda.h

When I run the main_vsfm.cpp,an error(fatal error: cuda.h: No such file or directory) occurs.How to deal with it?

How to keep the long line segments in the model and eliminate the short line segments ?

hello,
thank you for sharing such a nice code in advance, and it's working well in my computer

I get the 3d-line model by the executable file runLine3Dpp_colmap. Now, I only want to keep the long line segments in the model and eliminate the short line segments. What do I need to do? Can I change some parameters to make adjustments ?

Can vsfm_result.nvm file be replaced with realtime data?

How is the vsfm_result.nvm file is created? What is neccessary changes needed to make use of data online?

STL files show nothing of the result

Hi,@manhofer
I build all the procedures in VS2013.
Finally, three files were generated but when I open the STL with any STL viewer, nothing show up. OBJ file seems work well and i can see the point clouds. I guess the every pair of points stands for one line in 3D space, is that right?
Could you tell me what software you are using for openning the STL file?

any suggestions for reconstructing details

Hi,
Thanks for your excellent work. I tested this project using the openmvg input. Here is my 3D result:

as you can see the red marked parts, there are some details of windows lost. However, I find my window edges in these parts exist(see following). Do you know why this happened or any suggestions for me to reconstruct the details.




Thank you in advance.

compile problem:‘cout’ is not a member of ‘std’

[ 5%] Building CXX object CMakeFiles/line3Dpp.dir/line3D.cc.o
[ 11%] Building CXX object CMakeFiles/line3Dpp.dir/view.cc.o
In file included from /home/jc/Line3Dpp-master/commons.h:26:0,
from /home/jc/Line3Dpp-master/view.h:41,
from /home/jc/Line3Dpp-master/view.cc:1:
/home/jc/Line3Dpp-master/serialization.h: In function ‘void L3DPP::serializeFromFile(std::__cxx11::string, T&, bool)’:
/home/jc/Line3Dpp-master/serialization.h:62:13: error: ‘cout’ is not a member of ‘std’
std::cout << "[L3D++] serializeFromFileArchive(): File '" << file <
^
CMakeFiles/line3Dpp.dir/build.make:75: recipe for target 'CMakeFiles/line3Dpp.dir/view.cc.o' failed
make[2]: *** [CMakeFiles/line3Dpp.dir/view.cc.o] Error 1
CMakeFiles/Makefile2:289: recipe for target 'CMakeFiles/line3Dpp.dir/all' failed
make[1]: *** [CMakeFiles/line3Dpp.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Environment：ubuntu16,boost1.67,opencv3.1.

Assertion failed in remap

./runLine3Dpp_vsfm -m /home/chaiein/VR/Line3Dpp/testdata/vsfm_result.nvm -i /home/chaiein/VR/Line3Dpp/

[L3D++] //////////////////////////////////////////////////////////////////////
[L3D++] Line3D++ - http://www.icg.tugraz.at/ - AerialVisionGroup
[L3D++] (c) 2015, Manuel Hofer
[L3D++] published under the GNU General Public License
[L3D++] //////////////////////////////////////////////////////////////////////
OpenCV Error: Assertion failed (_map1.size().area() > 0) in remap, file /home/chaiein/VR/opencv/modules/imgproc/src/imgwarp.cpp, line 4936
terminate called after throwing an instance of 'cv::Exception'
what(): /home/chaiein/VR/opencv/modules/imgproc/src/imgwarp.cpp:4936: error: (-215) _map1.size().area() > 0 in function remap
Aborted

The SfM pipeline output issue: the output of COLMAP and PIX4D doesn't match with LINE3D++ input

The output file name of COLMAP 3.3 ends up with <.bin>, while you take <.txt> as input. I want to know your COLMAP version, thanks very much!

compile problem

when I change the cmake_build_type from release to debug, errors like "libline3Dpp.so: undefined reference to `ceres::Problem::SetParameterBlockConstant(double*)" appear, how can I solve this?

Link error

Thanks for having made this project has Open Source.

I just tried to compile it and I got to add imgcodecs to the list of required OpenCV libs

## OpenCV
FIND_PACKAGE(OpenCV REQUIRED COMPONENTS core imgproc highgui **imgcodecs**)

Another question, I tried on many scenes and I got very few 3D segments.
Does there is a parameter to tune in order to have more detected segments?

Here the result on the Sceaux Castle Sample scene from OpenMVG
https://github.com/openMVG/ImageDataset_SceauxCastle

Could someone tell me whether this project can use ndk to cross compile for android

I try to use ndk to compile but get some errors and I don't know how to deal with them.

/usr/local/include/eigen3/Eigen/src/Core/util/IndexedViewHelper.h:57:135: Generating code for declaration 'Eigen::lastp1'
clang++: error: unable to execute command: Segmentation fault (core dumped)
clang++: error: clang frontend command failed due to signal (use -v to see invocation)

Thanks in advance !

could not find CeresConfig

Though the cere-solver is successful stil I am not able to solve the above issue. Please help me to solve this problem

How to display the 3d-line model?

hello,
thank you for sharing such a nice code in advance, and it's working well in my computer

but when I got my output files such as .obj file and.txt file
I don't know how to connect points(start-points,end-points) into line segments and visualize these(3d-lines) like pictures in your paper.
Looking forward to your reply.

input and output

May I ask the author, is the input of this line3D++the output from running SFM code, such as an obj file

Cayley transformation degeneracy

When the matrix Q in the Plucker-to-Cayley transformation is symmetric, the transformation is not defined. https://github.com/manhofer/Line3Dpp/blob/master/optimization.cc#L56

For instance, when using 3D endpoints (5, -5, -5) and (-5, -5, -5) the matrix Q is symmetric and s will be filled with NaNs. Do you have any thoughts on how to avoid this?

Obviously you can simply rotate the entire model so that the 3D line does not produce this degeneracy but it is often the case that you take images with front-parallel planes (which would produce lines parallel or nearly parallel to the x-axis) so this degeneracy could happen naturally.

Where is the code combined bundle adjustment with points and lines?

In your paper, you proposed an algorithm combined bundle adjustment in section 3.5. And I read the whole projection, but I can't find the code. So, would you like to tell me where is the code ?

Thanks

Not working well the algorythm on my uav or close range data

Hi;
Thanks for publishing your good program,
I've run it on your test data and got correct 3d line outputs after processing and producing bundler results and then typing:

./runLine3Dpp_bundler -i <image_folder> -b <full_path_to_bundle_file> -f <full_path_to_image List_file>
but there isn't seen any correct 3dLine output after running the program in the same way on my uav or close range data!
I don't know what is wrong...
Thanks for your help,
Best Regards.

Segmentation Fault on When Using Ceres

Hi,
First of all, i want to thank you for this wonderful work. I can run it without problem when I specify

-c 0

for not running optimization. But when I want to do the optimization, I got segmentation error fault. When I do the backtrace, I get this:

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by './runLine3Dpp_vsfm -i ../testdata/ -m ../testdata/vsfm_result.nvm'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f8df05f5bdd in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6

My system is as follows:
Ubuntu 16.04
Boost 1.58
Cuda 8.0
Eigen 3.3.3
Ceres 2.0.0
OpenCV 3.4

Have you ever encounter this error? Thank you for your help.

Compile line3Dpp error

When i compile line3Dpp to produce line3Dpp .lib ,I got a error.

boost version:1.61(when I use boost1.62,the same error occured)