Comments (6)
from splatapult.
This is likely due to the rendering order of the VAOs. Gaussian splats require depth sorting in order to render correctly. If the splats from two different VAOs overlap/interpenetrate, they will likely exhibit errors like the ones shown in the images you provided (I assume you set the background color to red)
You are already making changes to the source code, so I would suggest making another modification. Specifically, combining the data from both ply files into one. Here's how you "might" achieve this.
diff --git a/src/gaussiancloud.cpp b/src/gaussiancloud.cpp
index 654d7f7..363c6b9 100644
--- a/src/gaussiancloud.cpp
+++ b/src/gaussiancloud.cpp
@@ -92,34 +92,38 @@ bool GaussianCloud::ImportPly(const std::string& plyFilename)
}
}
- gaussianVec.resize(ply.GetVertexCount());
+ std::vector<Gaussian> gVec;
+ gVec.resize(ply.GetVertexCount());
int i = 0;
ply.ForEachVertex([this, &i, &props](const uint8_t* data, size_t size)
{
- gaussianVec[i].position[0] = props.x.Get<float>(data);
- gaussianVec[i].position[1] = props.y.Get<float>(data);
- gaussianVec[i].position[2] = props.z.Get<float>(data);
+ gVec[i].position[0] = props.x.Get<float>(data);
+ gVec[i].position[1] = props.y.Get<float>(data);
+ gVec[i].position[2] = props.z.Get<float>(data);
for (int j = 0; j < 3; j++)
{
- gaussianVec[i].f_dc[j] = props.f_dc[j].Get<float>(data);
+ gVec[i].f_dc[j] = props.f_dc[j].Get<float>(data);
}
for (int j = 0; j < 45; j++)
{
- gaussianVec[i].f_rest[j] = props.f_rest[j].Get<float>(data);
+ gVec[i].f_rest[j] = props.f_rest[j].Get<float>(data);
}
- gaussianVec[i].opacity = props.opacity.Get<float>(data);
+ gVec[i].opacity = props.opacity.Get<float>(data);
for (int j = 0; j < 3; j++)
{
- gaussianVec[i].scale[j] = props.scale[j].Get<float>(data);
+ gVec[i].scale[j] = props.scale[j].Get<float>(data);
}
for (int j = 0; j < 4; j++)
{
- gaussianVec[i].rot[j] = props.rot[j].Get<float>(data);
+ gVec[i].rot[j] = props.rot[j].Get<float>(data);
}
i++;
});
+ // concatinate gVec on the end of gaussianVec.
+ gaussianVec.insert(gaussianVec.end(), gVec.begin(), gVec.end());
+
return true;
}
After this change you can load two ply files into a single GaussianCloud instance.
gaussianCloud->ImportPly(ply1);
gaussianCloud->ImportPly(ply2);
then pass that gaussianCloud to splatRenderer->Init(). This should "in theory" render both ply files together as a single VAO, which will eliminate the sorting errors.
Hope this helps.
from splatapult.
Thank you very much for your answer. Unfortunately, two weeks ago, I had already completed the plan you just proposed. Enter the folder path containing multiple Gaussian folders. Then, multiple ply layers are formed into one ply layer, and VAO is called for display. The rendering was successful without any issues.
In the past few days, I believe that editing and other operations may be necessary for future single Gaussian layers.
Gaussian space requires depth sorting to render correctly. I completely agree that if rendering multiple VAOs, it may be necessary to perform depth sorting on all VAOs, but I think this is still a bit difficult. I will continue to consider how to implement him, and I would be very happy if you have any good suggestions.
from splatapult.
Apologies for the delay. Perfect rendering of interpenetrating .ply files without full sorting isn't possible. But if the splats aren't fully interpenetrating, but are partitioned into a grid or some other pattern. You could sort by each centroid and render in back to front order. Also, if you're willing to live with some incorrect results along the edges of two grid cells, you can just turn off the depth test, this would get rid of those "red gaps". There would likely be some flickering as the sort order changed.. but it would be "mostly" correct.
Here's were you can disable the depth test.
diff --git a/src/app.cpp b/src/app.cpp
index f2d5108..cc43efb 100644
--- a/src/app.cpp
+++ b/src/app.cpp
@@ -144,7 +144,7 @@ static void Clear(glm::ivec2 windowSize, bool setViewport = true)
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// NOTE: if depth buffer has less then 24 bits, it can mess up splat rendering.
- glEnable(GL_DEPTH_TEST);
+ //glEnable(GL_DEPTH_TEST);
#ifndef __ANDROID__
// AJT: ANDROID: TODO: implement this in fragment shader, for OpenGLES I guess.
from splatapult.
Before your reply, I also tried to turn off deep testing (or glEnable (GL-DEPTH-TEST) myself;
GlDepthFunc (GL-ALWAYS);). There has indeed been some improvement, but I am well aware that it is not very reasonable.
I saw your reply today. What you mean is to take the center coordinates of each tile and calculate the distance from the camera position. Then render in reverse order of distance. I have tried and there is some relief, but it is not as effective as removing the depth test. Because when the camera is above the seam of two tiles, the effect is still not very good.
struct RendererDistancePair {
std::shared_ptr<SplatRenderer> renderer;
float distance;
bool operator<(const RendererDistancePair& other) const {
return distance > other.distance;
}
};
std::vector<RendererDistancePair> pairs;
for (int i = 0; i < vecSplatRenderer.size(); i++) {
std::vector<glm::vec4> posVec = vecSplatRenderer[i]->posVec;
glm::vec3 center = calculateCenter(posVec);
glm::vec4 camPosition = glm::inverse(cameraMat) * glm::vec4(center, 1.0f);
float distanceInViewSpace = glm::distance(glm::vec3(camPosition), objectCenter);
pairs.push_back({ vecSplatRenderer[i], distanceInViewSpace });
}
std::sort(pairs.begin(), pairs.end());
for (const auto& pair : pairs) {
std::shared_ptr<SplatRenderer> renderer = pair.renderer;
renderer->Sort(cameraMat, projMat, viewport, nearFar);
renderer->Render(cameraMat, projMat, viewport, nearFar);
}
from splatapult.
Why does the super-splat project on Github add multiple Gaussian files that render exactly right. This one is based on WebGL. Is it also just rendering a VAO?
from splatapult.
Related Issues (12)
- Camera is moving too fast when fps is low HOT 2
- Compilation errors on Linux. HOT 5
- Error on loading files HOT 4
- pre sort by depth sould be "float depth = p.z / p.w"? HOT 3
- How to change the height of the camera? HOT 3
- Question about render result on different PC HOT 19
- VR control does not working when using other VR headmount HOT 6
- VR creator with limited amount of CLI experience HOT 1
- Quest 3: VR head not working HOT 2
- Some questions. Thank you HOT 2
- Qestion about quantizedZs value in presort_compute.glsl HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from splatapult.