huangjh-pub / di-fusion Goto Github PK

[CVPR'21] [Jittor & Pytorch] DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors

Python 43.12% CMake 0.35% Cuda 45.46% C++ 2.93% C 8.14%

3d-reconstruction deep-learning implicit-representation slam

di-fusion's Introduction

DI-Fusion

This repository contains the implementation of our CVPR 2021 paper: DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors.

Jiahui Huang, Shi-Sheng Huang, Haoxuan Song, Shi-Min Hu

Introduction

DI-Fusion (Deep Implicit Fusion) is a novel online 3D reconstruction system based on RGB-D streams. It simultaneously localizes the camera and builds a local implicit map parametrized by a deep network. Please refer to our technical report and video for more details.

Implementations

We provide two implementations based on Jittor and Pytorch (the latter is also available on the public branch). Please refer to the corresponding folders jittor/ and pytorch/ for specific build and running instructions.

Citation

Please consider citing the following work:

@inproceedings{huang2021difusion,
  title={DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors},
  author={Huang, Jiahui and Huang, Shi-Sheng and Song, Haoxuan and Hu, Shi-Min},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2021}
}

di-fusion's People

Contributors

Stargazers

Watchers

Forkers

jingwenwang95 yuepanedward jh-choi zebrajack hanfeng-wu integritynoble llcc343 longruidong jackzhousz wolfcanli hamzaouajhain guyuezuntinggithub panbanghui liuxinren456852

di-fusion's Issues

Problem while trying to Generate the data

I am in the first step of building the project in order to generate the data, I am having issues with that.

I followed the same instalation instruction of deepSDF and built :

CLI11
EIGEN
PANGOLIN
FLANN

I have cuda 11.7 and Ubuntu 22.04, Any clue what would be the problem ?

"make -j
[ 40%] Building CUDA object CMakeFiles/PreprocessMeshCUDA.dir/Utils.cu.o
[ 40%] Building CUDA object CMakeFiles/PreprocessMeshCUDA.dir/PreprocessMesh.cu.o
[ 60%] Building CXX object CMakeFiles/PreprocessMeshCUDA.dir/ShaderProgram.cpp.o
In file included from /usr/local/include/pangolin/gl/glsl.h:232,
from /home/hamza/Desktop/di-fusion/pytorch/sampler_cuda/ShaderProgram.cpp:3:
/usr/local/include/pangolin/gl/glsl.hpp: In function ‘bool pangolin::IsLinkSuccessPrintLog(GLhandleARB)’:
/usr/local/include/pangolin/gl/glsl.hpp:19:5: error: ‘glGetProgramiv’ was not declared in this scope
19 | glGetProgramiv(prog, GL_LINK_STATUS, &status);
| ^~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp:25:9: error: ‘glGetProgramInfoLog’ was not declared in this scope
25 | glGetProgramInfoLog(prog, PROGRAM_LOG_MAX_LEN, &len, infolog);
| ^~~~~~~~~~~~~~~~~~~
In file included from /usr/local/include/pangolin/gl/glsl.h:232,
from /home/hamza/Desktop/di-fusion/pytorch/sampler_cuda/ShaderProgram.cpp:3:
/usr/local/include/pangolin/gl/glsl.hpp: In function ‘bool pangolin::IsCompileSuccessPrintLog(GLhandleARB, const string&, const string&)’:
/usr/local/include/pangolin/gl/glsl.hpp:51:5: error: ‘glGetShaderiv’ was not declared in this scope
51 | glGetShaderiv(shader, GL_COMPILE_STATUS, &status);
| ^~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp:57:9: error: ‘glGetShaderInfoLog’ was not declared in this scope
57 | glGetShaderInfoLog(shader, SHADER_LOG_MAX_LEN, &len, infolog);
| ^~~~~~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In destructor ‘pangolin::GlSlProgram::~GlSlProgram()’:
/usr/local/include/pangolin/gl/glsl.hpp:88:9: error: ‘glDeleteProgram’ was not declared in this scope; did you mean ‘GlSlProgram’?
88 | glDeleteProgram(prog);
| ^~~~~~~~~~~~~~~
| GlSlProgram
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘bool pangolin::GlSlProgram::AddPreprocessedShader(pangolin::GlSlShaderType, const string&, const string&)’:
/usr/local/include/pangolin/gl/glsl.hpp:108:16: error: ‘glCreateProgram’ was not declared in this scope
108 | prog = glCreateProgram();
| ^~~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp:111:26: error: ‘glCreateShader’ was not declared in this scope; did you mean ‘GlSlVertexShader’?
111 | GLhandleARB shader = glCreateShader(shader_type);
| ^~~~~~~~~~~~~~
| GlSlVertexShader
/usr/local/include/pangolin/gl/glsl.hpp:113:5: error: ‘glShaderSource’ was not declared in this scope
113 | glShaderSource(shader, 1, &source, NULL);
| ^~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp:114:5: error: ‘glCompileShader’ was not declared in this scope; did you mean ‘GlSlComputeShader’?
114 | glCompileShader(shader);
| ^~~~~~~~~~~~~~~
| GlSlComputeShader
/usr/local/include/pangolin/gl/glsl.hpp:117:9: error: ‘glAttachShader’ was not declared in this scope
117 | glAttachShader(prog, shader);
| ^~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::ClearShaders()’:
/usr/local/include/pangolin/gl/glsl.hpp:229:9: error: ‘glDetachShader’ was not declared in this scope
229 | glDetachShader(prog, shaders[i]);
| ^~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp:230:9: error: ‘glDeleteShader’ was not declared in this scope; did you mean ‘GlSlVertexShader’?
230 | glDeleteShader(shaders[i]);
| ^~~~~~~~~~~~~~
| GlSlVertexShader
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘bool pangolin::GlSlProgram::Link()’:
/usr/local/include/pangolin/gl/glsl.hpp:365:5: error: ‘glLinkProgram’ was not declared in this scope; did you mean ‘GlSlProgram’?
365 | glLinkProgram(prog);
| ^~~~~~~~~~~~~
| GlSlProgram
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::Bind()’:
/usr/local/include/pangolin/gl/glsl.hpp:372:5: error: ‘glUseProgram’ was not declared in this scope; did you mean ‘GlSlProgram’?
372 | glUseProgram(prog);
| ^~~~~~~~~~~~
| GlSlProgram
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SaveBind()’:
/usr/local/include/pangolin/gl/glsl.hpp:378:5: error: ‘glUseProgram’ was not declared in this scope; did you mean ‘GlSlProgram’?
378 | glUseProgram(prog);
| ^~~~~~~~~~~~
| GlSlProgram
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::Unbind()’:
/usr/local/include/pangolin/gl/glsl.hpp:383:5: error: ‘glUseProgram’ was not declared in this scope; did you mean ‘GlSlProgram’?
383 | glUseProgram(prev_prog);
| ^~~~~~~~~~~~
| GlSlProgram
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘GLint pangolin::GlSlProgram::GetAttributeHandle(const string&)’:
/usr/local/include/pangolin/gl/glsl.hpp:388:20: error: ‘glGetAttribLocation’ was not declared in this scope
388 | GLint handle = glGetAttribLocation(prog, name.c_str());
| ^~~~~~~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘GLint pangolin::GlSlProgram::GetUniformHandle(const string&)’:
/usr/local/include/pangolin/gl/glsl.hpp:395:20: error: ‘glGetUniformLocation’ was not declared in this scope
395 | GLint handle = glGetUniformLocation(prog, name.c_str());
| ^~~~~~~~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, int)’:
/usr/local/include/pangolin/gl/glsl.hpp:402:5: error: ‘glUniform1i’ was not declared in this scope
402 | glUniform1i( GetUniformHandle(name), x);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, int, int)’:
/usr/local/include/pangolin/gl/glsl.hpp:407:5: error: ‘glUniform2i’ was not declared in this scope
407 | glUniform2i( GetUniformHandle(name), x1, x2);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, int, int, int)’:
/usr/local/include/pangolin/gl/glsl.hpp:412:5: error: ‘glUniform3i’ was not declared in this scope
412 | glUniform3i( GetUniformHandle(name), x1, x2, x3);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, int, int, int, int)’:
/usr/local/include/pangolin/gl/glsl.hpp:417:5: error: ‘glUniform4i’ was not declared in this scope
417 | glUniform4i( GetUniformHandle(name), x1, x2, x3, x4);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, float)’:
/usr/local/include/pangolin/gl/glsl.hpp:422:5: error: ‘glUniform1f’ was not declared in this scope
422 | glUniform1f( GetUniformHandle(name), f);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, float, float)’:
/usr/local/include/pangolin/gl/glsl.hpp:427:5: error: ‘glUniform2f’ was not declared in this scope
427 | glUniform2f( GetUniformHandle(name), f1,f2);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, float, float, float)’:
/usr/local/include/pangolin/gl/glsl.hpp:432:5: error: ‘glUniform3f’ was not declared in this scope
432 | glUniform3f( GetUniformHandle(name), f1,f2,f3);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, float, float, float, float)’:
/usr/local/include/pangolin/gl/glsl.hpp:437:5: error: ‘glUniform4f’ was not declared in this scope
437 | glUniform4f( GetUniformHandle(name), f1,f2,f3,f4);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, double)’:
/usr/local/include/pangolin/gl/glsl.hpp:442:5: error: ‘glUniform1d’ was not declared in this scope
442 | glUniform1d( GetUniformHandle(name), f);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, double, double)’:
/usr/local/include/pangolin/gl/glsl.hpp:447:5: error: ‘glUniform2d’ was not declared in this scope
447 | glUniform2d( GetUniformHandle(name), f1,f2);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, double, double, double)’:
/usr/local/include/pangolin/gl/glsl.hpp:452:5: error: ‘glUniform3d’ was not declared in this scope
452 | glUniform3d( GetUniformHandle(name), f1,f2,f3);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, double, double, double, double)’:
/usr/local/include/pangolin/gl/glsl.hpp:457:5: error: ‘glUniform4d’ was not declared in this scope
457 | glUniform4d( GetUniformHandle(name), f1,f2,f3,f4);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, pangolin::Colour)’:
/usr/local/include/pangolin/gl/glsl.hpp:462:5: error: ‘glUniform4f’ was not declared in this scope
462 | glUniform4f( GetUniformHandle(name), c.r, c.g, c.b, c.a);
| ^~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetUniform(const string&, const pangolin::OpenGlMatrix&)’:
/usr/local/include/pangolin/gl/glsl.hpp:472:5: error: ‘glUniformMatrix4fv’ was not declared in this scope
472 | glUniformMatrix4fv( GetUniformHandle(name), 1, GL_FALSE, m);
| ^~~~~~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::BindPangolinDefaultAttribLocationsAndLink()’:
/usr/local/include/pangolin/gl/glsl.hpp:529:5: error: ‘glBindAttribLocation’ was not declared in this scope
529 | glBindAttribLocation(prog, DEFAULT_LOCATION_POSITION, DEFAULT_NAME_POSITION);
| ^~~~~~~~~~~~~~~~~~~~
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘GLint pangolin::GlSlProgram::GetProgramResourceIndex(const string&)’:
/usr/local/include/pangolin/gl/glsl.hpp:539:12: error: ‘glGetProgramResourceIndex’ was not declared in this scope; did you mean ‘GetProgramResourceIndex’?
539 | return glGetProgramResourceIndex(prog, GL_SHADER_STORAGE_BLOCK, name.c_str());
| ^~~~~~~~~~~~~~~~~~~~~~~~~
| GetProgramResourceIndex
/usr/local/include/pangolin/gl/glsl.hpp: In member function ‘void pangolin::GlSlProgram::SetShaderStorageBlock(const string&, const int&)’:
/usr/local/include/pangolin/gl/glsl.hpp:544:5: error: ‘glShaderStorageBlockBinding’ was not declared in this scope
544 | glShaderStorageBlockBinding(prog, GetProgramResourceIndex(name), bindingIndex);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/PreprocessMeshCUDA.dir/build.make:90: CMakeFiles/PreprocessMeshCUDA.dir/ShaderProgram.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/usr/local/include/pangolin/gl/gl.hpp(348): error: identifier "glCopyImageSubDataNV" is undefined

/usr/local/include/pangolin/gl/gl.hpp(522): error: identifier "glDeleteRenderbuffersEXT" is undefined

/usr/local/include/pangolin/gl/gl.hpp(527): error: identifier "glGenRenderbuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(528): error: identifier "glBindRenderbuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(529): error: identifier "glRenderbufferStorage" is undefined

/usr/local/include/pangolin/gl/gl.hpp(536): error: identifier "glDeleteRenderbuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(585): error: identifier "glDeleteFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(592): error: identifier "glGenFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(600): error: identifier "glGenFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(609): error: identifier "glGenFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(619): error: identifier "glGenFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(630): error: identifier "glGenFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(641): error: identifier "glBindFramebuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(642): error: identifier "glDrawBuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(648): error: identifier "glDeleteFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(650): error: identifier "glGenFramebuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(655): error: identifier "glDrawBuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(656): error: identifier "glBindFramebuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(664): error: identifier "glBindFramebuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(665): error: identifier "glFramebufferTexture2D" is undefined

/usr/local/include/pangolin/gl/gl.hpp(676): error: identifier "glBindFramebuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(678): error: identifier "glFramebufferRenderbuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(734): error: identifier "glDeleteBuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(751): error: identifier "glGenBuffers" is undefined

/usr/local/include/pangolin/gl/gl.hpp(759): error: identifier "glBufferData" is undefined

/usr/local/include/pangolin/gl/gl.hpp(765): error: identifier "glBindBuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(770): error: identifier "glBindBuffer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(780): error: identifier "glBufferSubData" is undefined

/usr/local/include/pangolin/gl/gl.hpp(787): error: identifier "glGetBufferSubData" is undefined

/usr/local/include/pangolin/gl/gl.hpp(862): error: identifier "glGetBufferSubData" is undefined

/usr/local/include/pangolin/gl/gl.hpp(863): error: identifier "glBufferData" is undefined

/usr/local/include/pangolin/gl/gl.hpp(864): error: identifier "glBufferSubData" is undefined

/usr/local/include/pangolin/gl/gl.hpp(969): error: identifier "glGenVertexArrays" is undefined

/usr/local/include/pangolin/gl/gl.hpp(974): error: identifier "glDeleteVertexArrays" is undefined

/usr/local/include/pangolin/gl/gl.hpp(979): error: identifier "glBindVertexArray" is undefined

/usr/local/include/pangolin/gl/gl.hpp(985): error: identifier "glBindVertexArray" is undefined

/usr/local/include/pangolin/gl/gl.hpp(993): error: identifier "glVertexAttribPointer" is undefined

/usr/local/include/pangolin/gl/gl.hpp(995): error: identifier "glEnableVertexAttribArray" is undefined

/usr/local/include/pangolin/gl/gl.hpp(348): error: identifier "glCopyImageSubDataNV" is undefined