gostinshell/pari-conv

Official Implementation of "The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant Learning via Pose-aware Convolution" (CVPR 2022)

The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant Learning via Pose-aware Convolution (CVPR, 2022)"

The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant Learning via Pose-aware Convolution
by Ronghan Chen, Yang Cong
in CVPR, 2022. Arxiv

If you have any question about the code or the paper, don't hesitate and open an issuse!😉

Dependencies

Python=3.7, CUDA=10.0, PyTorch=1.4.0, torch_geometric=1.6.0, torch_cluster=1.5.4, torch_sparse=0.6.1, torch_scatter=2.0.4, tensorboardX, scikit-learn, numpy, termcolor

Data

First, please download the ModelNet40 dataset from here(1.6G), and place it at dataset/modelnet40_normal_resampled.

Point Cloud Classification on ModelNet40

To train a model under SO(3) or z rotations:

sh scripts/PaRINet_so3.sh 
sh scripts/PaRINet_rot_z.sh 

Best model will be saved at log/PaRINet_***/best. And you can evaluate them by running:

sh scripts/test_PaRINet_rot_z.sh
sh scripts/test_PaRINet_so3.sh

To visualize the training process, please run:

tensorboard --logdir log

Cite this work

If you find this work useful, please citing the paper:

@inproceedings{chen2022devil,
  title={The Devil is in the Pose: Ambiguity-free 3D Rotation-invariant Learning via Pose-aware Convolution},
  author={Chen, Ronghan and Cong, Yang},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={7472--7481},
  year={2022}
}

Acknowledgement

• The code framework is borrowed from 3D-CODED
• The code for classification architecture is borrowed from DGCNN

TODO

Code on ScanObjectNN and ShapeNetPart.

pari-conv's Issues

The difference of implementation

Hi. Thanks for sharing codes. Your work is excellent and significant for inspiring others.

However, I tried to reproduce the performance of your method, Pari-Conv, and only got 92.0% twice on MoldelNet40.
And the codes of Pari-conv seem to be different from that provided in the paper. The dynamic kernel shall be a kCC matrix and operated with matrix multiplication. But in the model.py, it's k*C and used with Hadamard product.
Is it the reason for low performance？ Please check Lines 107-130 in model.py.

ShapeNetPart code

Hi @GostInShell !
When will you update the code on ShapeNetPart?
I want to use your PaIR to conduct experiments on 3D point cloud segmentation.

Question about results

Hi @GostInShell ,

The evaluation code calculates two kinds of accuracy: (1) accuracy (which tensorboard shows as test_acc) (2) balanced accuracy (which tensorboard shows as test_avg_acc)

In Table1 of main paper, which accuracy did you report?

Thanks,
Supriya

Error while training

Hi @GostInShell ,
Thank you for sharing the code.
I am getting following error during training on last batch (4920 /4921) of first epoch
Could you take a look at the code and help me resolve the error?
Appreciate your help

para_num is: 1.849128 MB
CosineAnnealingLR

[0: 4920/4921] train loss:  3.0886149406433105 Ellapsed Time: 0.07949230167600844h ETL: 23.775468789989244h

Traceback (most recent call last):
  File "train.py", line 44, in <module>
    main()
  File "train.py", line 23, in main
    trainer.train_epoch()
  File "PaRI-Conv/trainer.py", line 192, in train_epoch
    self.train_iteration()
  File "PaRI-Conv/trainer.py", line 175, in train_iteration
    pred = self.network(self.data)
  File "/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "PaRI-Conv/Networks/model.py", line 194, in forward
    x = F.leaky_relu(self.bn6(self.linear1(x)), negative_slope=0.2)
  File "python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "python3.7/site-packages/torch/nn/modules/batchnorm.py", line 179, in forward
    self.eps,
  File "python3.7/site-packages/torch/nn/functional.py", line 2280, in batch_norm
    _verify_batch_size(input.size())
  File "python3.7/site-packages/torch/nn/functional.py", line 2248, in _verify_batch_size
    raise ValueError("Expected more than 1 value per channel when training, got input size {}".format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512])

Thank you,
Supriya