How to add RGB information to mesh?

[victorprad]

How to add RGB information to mesh?

[vinjn]

Mission complete! RGBD of my dog

[vinjn]

typedef ITMVoxel_s_rgb ITMVoxel; sounds like the first step.

[olafkaehler]

That first step of yours will at least make sure the colour information is integrated into the 3D volume. However, extracting it from there into the mesh data structure and exporting RGB along with the geometry into an STL file are not currently supported.

Others have asked for this feature as well, and since it's not too complicated to implement, they've even started working on it. If you want to try yourself and need some hints, please let me know. I'll see whether we can get some sort of common implementation into the repository at some point.

[vinjn]

Thx for the reply, and yes, I want to try myself.

[vinjn]

To extract color into the mesh data structure

template<bool hasColor, class TVoxel, class TIndex> struct SimpleVoxelColorReader;

template<class TVoxel, class TIndex>
struct SimpleVoxelColorReader<false, TVoxel, TIndex> {
    _CPU_AND_GPU_CODE_ static Vector3f calc(const CONSTPTR(TVoxel) *voxelData, const CONSTPTR(typename TIndex::IndexData) *voxelIndex,
    const THREADPTR(Vector3i) & point)
    {
        return Vector3f(0.0f, 0.0f, 0.0f);
    }
};

template<class TVoxel, class TIndex>
struct SimpleVoxelColorReader<true, TVoxel, TIndex> {
    _CPU_AND_GPU_CODE_ static Vector3f calc(const CONSTPTR(TVoxel) *voxelData, const CONSTPTR(typename TIndex::IndexData) *voxelIndex,
    const THREADPTR(Vector3i) & point)
    {
        typename TIndex::IndexCache cache;
        bool isFound;
        TVoxel resn = readVoxel(voxelData, voxelIndex, point, isFound, cache);

        // TODO: optimize
        if (isFound)
            return Vector3f(resn.clr.x / 255.0f, resn.clr.y / 255.0f, resn.clr.z / 255.0f);
        else
            return Vector3f(0.0f, 0.0f, 0.0f);
    }
};

[rajputasif]

Amazing

Would you like to share the code changes for rgb-d fusiin?

Thanks

[carlren]

Actually, could you make a PR and we might be able to merge that back to
the code base.

Thanks and best wishes,

Carl

Carl Yuheng Ren
Postdoc Research Assistant
Oxford Torr Vision Group
Department of Engineering Science
University of Oxford, UK
E-mail: [email protected] or [email protected] [email protected]
Cell Phone:[phone]
Website: http://www.carlyuheng.com

---

任禹衡
牛津大学工程系博后研究员
牛津大学工程学博士
哈尔滨工业大学计算机科学与技术学士
电话：[phone]
邮箱：[email protected] 或 [email protected]
网站: http://www.carlyuheng.com

On Fri, Sep 18, 2015 at 5:31 PM, Asif Rajput [email protected]
wrote:

Amazing

Would you like to share the code changes for rgb-d fusiin?

Thanks

—
Reply to this email directly or view it on GitHub
#21 (comment)
.

[vinjn]

Sure, I will make a PR after cleaning the code.

[vinjn]

@carlren @rajputasif Can you have a look at the PR?

[MarkZurkberg]

Hi, I have tried your work and got an RDB mesh. I wonder if I can expand this function, well, I mean I want to get the whole live reconstruction scene with RGB information, not only the mesh generated by pressing 'w'. Could you please give some advice? @vinjn

[rajputasif]

Hi @vinjn excellent work, for some reason I am able to view Colored raycasting in viewer but the mesh is all black... Strange

[rajputasif]

@MarkZurkberg I just checked this version https://github.com/victorprad/InfiniTAM/tree/c8ba1872746a4f490568613e331069a9166a855f and I can see the live reconstruction with RGB information (Which I think is your goal) I might be wrong.... As I was out of touch for so long

[MarkZurkberg]

Thanks! I will check this branch later. By the way, I want to add some 3D models like .obj files into the reconstruction scene, do you have some good advice? @rajputasif

[MarkZurkberg]

It works good, but when I want to import the .obj file by glm( a library used to load models ) it doesn't show any color information. So I check the .obj file using txt tools, I find this output .obj file kind of special because there are 6 numbers after each line of information for vertex. Of course, the last three numbers are used for RGB information, right? I still want to write a program to load models with color information, but I have no idea how to deal this kind of .obj file, and direction would be appreciated. Thanks! @vinjn @carlren @olafkaehler

[vinjn]

meshlab supports this kind of obj. (x,y,z,r,g,b)

[MarkZurkberg]

Yeah! Meshlab works good, But if I want to write a program myself, I have no idea how to deal with this kind of format. Could you please give me some directions? Thanks! @vinjn

[ganlumomo]

@rajputasif @MarkZurkberg @vinjn I just checked out the master branch and tried the demo with Teddy frames. However, I did not see color on the reconstructed scene even I turned on 'color' using UI. Do you know what's the reason? Thanks!

[sgolodetz]

@ganlumomo Did you change the ITMVoxel typedef as mentioned above? The default voxel type doesn't store colour information.

[TotoLulu94]

@vinjn @MarkZurkberg
This issues is a bit old but I hope you both can still help me.
I have a 3D point clouds modelling an object, could you indicate me the direction about how to use Meshlab ?
For now, I've write the points into an .obj file :
v x y z R G B
is that the right format ? Do I need to write in the file anything else ? Then, where did you add codes in order to include the 3D model into the reconstruction scene ? Thank you.

[nonlinear1]

I test this branch "https://github.com/victorprad/InfiniTAM/tree/c8ba1872746a4f490568613e331069a9166a855f" but I didn't
see color on the the reconstructed scene and no "color" option in the UI, Why? Do this verison need some special configuraion?
Thanks a lot

[vinjn]

My PR #22 was rejected :( and I am not working on this project anymore.

You can still use my old fork which has color info -> https://github.com/jing-vision/InfiniTAM

[nonlinear1]

Thank you very much

[nonlinear1]

@vinjn Thank you for help. I download your color version InfiniTAM　and I compiled it successfully.
But I cann't find the color image and color opiton in UI. Could you give me a suggestion? I think maybe
your version InfiniTAM need some configuraiton?

[nonlinear1]

I know why! see the following codes:
if (uiEngine->freeviewActive)
{
sprintf(str, "n - next frame \t b - all frames \t e/esc - exit \t f - follow camera \t c - colours (currently %s) \t t - turn fusion %s", uiEngine->colourModes[uiEngine->currentColourMode].name, uiEngine->intergrationActive ? "off" : "on");
}
else
{
sprintf(str, "n - next frame \t b - all frames \t e/esc - exit \t f - free viewpoint \t t - turn fusion %s", uiEngine->intergrationActive ? "off" : "on");
}
must be freeview!!!

[nonlinear1]

@vinjn
请问你有微信或者qq吗？希望能交流一下，我要把这个颜色信息像深度信息一样进行融合一下。

[vinjn]

-> vinjnz

[zhaozhongch]

Well, it's 2020 now. I also want to make rgb in Infinitam works.
Inifinitam v3 master branch doesn't integrate rgb yet. The commit in the previous discussion seems is in v1 or v2. I tried to apply its change v3. Looks work on demo dataset

However, I run tum dataset and the color looks bad

Without color the grayscale reconstruction looks fine
Any idea why the code just works on the demo dataset? Is the calibration file's problem?
The calibration file is (following discussion #30 )

640 480
591.123473 590.076012
331.038659 234.047543

640 480
517.306408 516.469215
318.643040 255.313989

1 0 0 0
0 1 0 0
0 0 1 0

affine 0.0002 0.0

I know it may not easy to answer since many people won't test tum rgbd dataset but just in case someone knows it...