ajhamdi/mvtorch

a Pytorch library for multi-view 3D understanding and generation

mvtorch's Issues

multi-view stereo example

multi-view stereo example is necessary since its one of the oldest tasks on multi-view ( I recommend taking this modern example from CVPR 2022 and try to do it in MVtorch , its using Pytorch3d so it should be straightforward ) . please watch this video to know what they are doing in this MVS paper.

creating posed datasets

creating posed multi-view datasets using MVtorch rendering , 3D dataset , and saving camera parameters and rendered images.

nerf dataset loader

[OPTIONAL] Structure-from-motion

[EXAMPLE] Structure-from-motion is also one of the most important multi-view applications, typically using colormap. this ICCV21 paper is a good starting point ( maybe we can do something with MVtorch there when camera poses are available)

Some Issues regarding Vointcloud

2 Issues:

1. np.int in function getitem in class ShapeNetPart in data.py does not work(from line 963 to line 989). Should all change to np.int32 or np.int64.
2. The Github page for Vointcloud does not give any instructions about the function lifting_net while it is none in segmentation.py. None type lifting_net doesn't work for lifting_method mlp.

visualizer : offline and interactive

Benchmarks

we need to create a benchmarking table as we discussed earlier that compares speed, cost, and performance of different submodules , and also reproduce results of other works ( eg. classification with MVCNN, segmentation , nerfs , etc ) and report the performance of those others works using MVtorch ( please see again the Pytorch3d arxive paper )

nerf example on mvtorch

Allow others to add to the MVtorch library

Please write down instructions to anyone who wants to add and contribute to the library ( see pytorch3D ) so they can fork and pull requests and then we would need to approve and merge to the main branch

ViewSelector : accepting camera_to_world matrices as `views` in addition to (azimuth,elevation, distance)

see https://ksimek.github.io/2012/08/22/extrinsic/ , https://pytorch3d.org/docs/cameras , https://trimsh.org/trimesh.scene.cameras.html ,

new dataloader for shapenet parts ( from CurveNet paper code )