imoonlab/pvrnet

PVRNet: Point-View Relation Neural Network for 3D Shape Recognition (AAAI 2019)

PVRNet

PVRNet: Point-View Relation Neural Network for 3D Shape Recognition (AAAI 2019)

Created by Haoxuan You, Yifan Feng, Xibin Zhao, Changqing Zou, Rongrong Ji, Yue Gao from Tsinghua University.

Introduction

This work will appear in AAAI 2019. We propose a point-view relation neural network called PVRNet for 3D shape recognition and retrieval. You can chekc our paper for more details.

In this repository, our code and data are released for training our PVRNet on ModelNet40 dataset.

Citation

If you find our work useful in your research, please cite our paper:

@article{you2018pvrnet,
title={PVRNet: Point-View Relation Neural Network for 3D Shape Recognition},
author={You, Haoxuan and Feng, Yifan and Zhao, Xibin and Zou, Changqing and Ji, Rongrong and Gao, Yue},
journal={AAAI 2019},
year={2018}
}

Configuration

Code is tested under the environment of Pytorch 0.4.1, Python 3.6 and CUDA 9.0 on Ubuntu 16.04.

Data: point cloud data and multi-view(12-view) data from ModelNet40 dataset.

Pretrained Model: multi-view part(MVCNN), point cloud part(DGCNN) and PVRNet

Usage

• Download data and pretrained ckpt from above links. Create dir for data as well as result, and place them under corresponding dirs(./data/ and ./result/ckpt/).

  mkdir -p data result/ckpt

• Train PVRNet. This would use pretrained MVCNN model and DGCNN model saved in ./result/ckpt:

  python train_pvrnet.py

• If validate the performance of PVRNet with our pretrained model:

  python val_pvrnet.py

  If validate the performance of pretrained MVCNN and DGCNN models:

  python val_mvcnn.py
  python val_pc.py
  

• If you want to train new model for MVCNN and DGCNN:

  python train_mvcnn.py
  python train_pc.py
  

License

Our code is released under MIT License (see LICENSE file for details).

pvrnet's Issues

why no update the first layer in AlexNet

Hi, excellent work. I found thisline confusing. I am wondering why you put this line here, should it be better to update all the parameters in alexnet?

0

TypeError: can't convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Think you for your great work,After training,I meet this problem.can you tell me how to solve this problem?
TypeError: can't convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

torch.cuda.is_available() is False.

Hello,Think your for your work,when I tried to run your code ,I met this problem.can you tell me how to solve this problem?
raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location='cpu' to map your storages to the CPU.

The accuracy of the pretrained model

Hi, I validate the performance of the network with the pretrained model(PVNet2-ALEXNET-v94-ckpt.pth). However, I only get 92.55% accuracy. Have you tried to get the results with a higher version of pytorch？
Here is my environment: pytorch1.6.0 cuda10.1 NVIDIA TITAN Xp.

How to do the retrieval experiment on the modelnet40 dataset?

Think you for your work,How to do the retrieval experiment on the modelnet40 dataset?looking forward your reply.

Can you send me code about Visualization of relation score module?

Can you send me code about Visualization of relation score module?looking for your reply.

config.pv_net.train.iter_train == False

Sorry to bother you!
Should this function be TRUE or FALSE?
In the paper PVRNet, in the first 10 epochs,the feature extraction model is fixed and only finetune the other part.

And if 'config.pv_net.train.iter_train == True:',I only got a result of 92.0%+,but not 93.6%
if I change this line to False, the result can be 93.1% but still not 93.6%
Did I make something wrong in using your code?

Thank you for your reply!

what is the aim of 3D shape recognition

Hello, I would like to ask what is the task of 3D shape recognition? Or what is its application background and prospects? Thank you

the dimensional transformation of the input point cloud

Hello, your work is excellent, I have a question to be solved, the input point cloud dimension should be BXNXdim, why not see the dimension transformation in your code, but in the defined functions like [gather_neighbor] and [get_neighbor_feature], There is batch_size = x.size()[0] ;num_dim = x.size()[1] ; num_point = x.size()[2], the input is BXdimXN?why? I am very confused.I am looking forward to your answer.