hydrogenion/vipc

PyTorch implementation for View-Guided Point Cloud Completion

ViPC: View-Guided Point Cloud Completion

PyTorch implementation of ViPC (View-Guided Point Cloud Completion)

Introduction

This work is accepted by CVPR 2021. We present a view-guided solution for the task of point cloud completion. Unlike most existing methods directly inferring the missing points using shape priors, we address this task by introducing ViPC (view-guided point cloud completion) that takes the missing crucial global structure information from an extra single-view image. Besides, we build a large-scale dataset for the point cloud completion task on the ShapeNet dataset. This dataset simulates point cloud defects caused by various kinds of occlusions.You can also check out paper for a deeper introduction.

Citation

if you find our work useful in your research, please consider citing:

@inproceedings{zhang2021vipc,
  title={View-Guided Point Cloud Completion},
  author={Zhang, Xuancheng and Feng, Yutong and Li, Siqi and Zou, Changqing and Wan, Hai and Zhao, Xibin and Guo, Yandong and Gao, Yue},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={15890--15899},
  year={2021}
}

Usage

Installation

Install PyTorch over 1.4. The code has been tested with Python 3.7, PyTorch 1.7.0 and CUDA 10.1 on Ubuntu 20.04 and GTX 1080Ti. Note that RTX graphics card with Tensor Core may not be able to run part of the code.

Data Prepare

First, please download the ShapeNetViPC-Dataset (143GB, code: ar8l). Then run cat ShapeNetViPC-Dataset.tar.gz* | tar zx, you will get ShapeNetViPC-Dataset contains three floders: ShapeNetViPC-Partial, ShapeNetViPC-GT and ShapeNetViPC-View.

For each object, the dataset include partial point cloud (ShapeNetViPC-Patial), complete point cloud (ShapeNetViPC-GT) and corresponding images (ShapeNetViPC-View) from 24 different views. You can find the detail of 24 cameras view in /ShapeNetViPC-View/category/object_name/rendering/rendering_metadata.txt.

unzip train_test_list.zip, you will get train and test list. Use the code in utils/dataloader.py to load the dataset.

The remaining code will be open source soon!!!

License

Our code is released under MIT License (see LICENSE file for details).

vipc's Issues

视图对于模型的作用

你好，ViPC文章中研究了什么样的输入视图可以更好地辅助补全，我们注意到dataloader中似乎对不完整点云和图像进行了对齐和校准，如果要研究视图对于模型的作用，这部分的代码是不是应该注释掉？如果不注释这部分代码，输入模型的不完整点云似乎也随视图在旋转变化。

Meaning of parameters in rendering/camera_calibration.txt file for each shape

Hi, thanks for open sourcing your work! I would like to get the intrinsic camera matrix for each rendered view and I have noticed there is a rendering/camera_calibration.txt file for each shape. In this file, there are three parameters, but I can't understand their meaning. Can you help me to decode such values and build the 3x3 calibration matrix?

code cann't run directly.

code has some wrong. I have fixed some problems. Welcome to discuss with me.

How do you pretrain the modality transfer network?

the loss function is CDloss of complete point cloud and reconstrusted point cloud?

Evaluation Code

Could you please upload your evaluation code?

ModuleNotFoundError: No module named 'point_gpu'

Some questions about the dataset

Hello, I downloaded the dataset according to the instructions, but encountered an error when training certain categories. After checking the dataset, I found that the file was indeed missing. I tried to download the dataset again, but the error still persists. Can you help me take a look? This type of error is not limited to just one category, and I am conducting experiments to investigate it.

数据集下载问题

你好，请问为什么我的数据集下载解压有些文件夹是空的

could training batch size=1 be changed?

From the code file train.py, the batch of training is 1, can it be modified? eg: batch size=32

How can I pass from 3500 points of the pointclouds provided in your dataset to the 2048 used in the paper?

Questions about the metric

Sorry to bother you. What does the [email protected] mean in your paper, do you mean F-score*10? And what is the value of fscore's threshold?

怎么把点云投影和视图相对应？

我们尝试过把点云进行投影，但是始终和ViPC数据集里面提供的视图对应不上。
你们能不能告诉我们怎么把点云投影到平面和视图相对应？
非常感谢！

Meaning of parameters in rendering/rendering_metadata.txt

Thank you for publishing your data set!
I want to know the meaning of each column of data in rendering/metadata.txt. Each line of metadata has five values, but how can I convert them into projection matrix? Thank you very much!

Could you please tell me the detail of how to used ViPC dataset to train PCN model?

Thank you for publishing your dataset!
We trained the Point Completion Network (PCN) using the vehicle categories in your dataset, and our chamfer distance only reached 9.6 * 10e-3 (almost twice your experimental results), however, your experiment can reach 4.840 * 10e-3. Our experimental setup is the same as yours, with 2048 points input and 2048 points output.
Could you please point out our problem and tell me the detail of how to used ViPC dataset to train PCN model.
Thanks a lot!

About experimental metrics

Hello, congratulations, you are doing a great job. And I have a question about the experimental indicators, in the table 1 of the part 'Experimental Results and Analysis', is the indicator 'Mean Chamfer Distance per point' the result after *1000？

I cannot download the dataset from beidu

I'm opening this issue since Beidu doesn't allow the download of the dataset from Italy, I tried other way for example with a Download manager, but there are a lot of issues and the speed is too low to download it. It would be possible to have the dataset in another way?

How do you process ViPC dataset and how long does it takes when training on PCN?

Could you please tell me how do you process the ViPC dataset when training on PCN and how long it takes to train PCN with the whole ViPC dataset?
Thanks a lot!

Could you please provide the pre-trained model?

Could you offer the pretrained models?

@Hydrogenion Hello, I found you offer the pretrained model of the ModalityTransfer network, but not the whole ViPC model. So I wonder could you offer the pretrained models for your whole networks?

Dataset download issue

I saw a comment saying that You will uploading a dataset using another cloud. When can you upload it?