lijx10/so-net

SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018

so-net's Issues

how to run?

Dataset for shape retrieval

Hi. Where can I get dataset for shape retrieval?

Is there any already-trained model available? (3D retrieval)

Hi, thanks for your great work and patient guidance.

I want to know if there is a already-trained model for 3D retrieval available for a test demo? Because I have few computing source in school lab to train a model from zero.

No offence to your work~I will be appreciate if you provided one.

shrec16/test.py error

line 6 of shrec16/test.py

line 24 of shrec16/test.py

Could you please upload related code? Thank you very much.

The error in ‘Application - Part Segmentation’

My environment is ubuntu16.04，cuda9.2，cudnn7.6.4,pytorch1.1.0,faiss and visdom installed.

I used the shapenet dataset in your Google Drive.Its location is '/media/rmv/data1/CPR/shapenetcore_partanno_segmentation_benchmark_v0_normal/',and I changed the 'dataroot' in the opinion of part-seg folder.

Then I compiled customized cuda code.The process is showed below.

Then I began the training,but I found the error''No module named 'util'''.To slove this,I add the 'SO-Net-master' folder path to the python path.

At last it showd the error in the error.txt below,and I cannot figure out how to solve it.

error.txt

Very low mIoU on part set

Hello,

I tried the part-seg task using the default settings as in part-seg folder. The test mIoU is only around 0.16 while test accuracy is above 90%. Any suggestion on such weird performance?

Thanks

test file for autoencoder

Hi,
I would like to try autoencoder on my dataset. I could not find the test file for the autoencoder. Only provided test file is in shrec16. If possible, could you share the test file for autoencoder and part-seg?

Thanks in advance
edit: I saw the test part in the train file for the autoencoder, so I have to use the same for train and test right?

How to run on multi-gpu？

changing dataset path

I am getting this error message when I try executing train.py in the modelnet folder Please tell me a solution.

Supplementary material

Hi Jiaxin,
Thank you very much for your great work! I'm quite interested in how to initialize the SOM. The CVPR2018 paper mentioned it's in the supplementary material. Could you tell me where I can find the supplementary material?

masked max in encoder

gather_index is the max_id, then you use it to multiply mask_row_max. I believe these nodes without neighbor will choose point 0. So what' the difference with just using the max_val and mask the nodes with no neighbor with channels all zero?

The function of index_max.

Thanks for your work!
I have a few questions about the index_max function. I am not familiar with cuda. The index_max seems is used to max pooling for all point features where the points allocated in one SOM cluster.
Do I understand right? Why do you implement it with cuda? Does it speed up a lot?
Appreciated if you can reply.

Low validation mIoU despite high validation accuracy on part seg

I am getting the same error as mention in a previous issue: #34

On the part segmentation, the validation mIoU is only reaching 0.16, although the validation accuracy is about 90%. The was run with the default parameters in the code and using the prepared version of ShapeNet found here: https://drive.google.com/open?id=184MbflF_RbDX9MyML3hid7OxsYJ8oQQ7

I've attached a plot that shows the train and validation losses and accuracy, and the validation mIoU, at every epoch during training.

Retrieval result unclear

Sorry to take your time again and again (sorry for being stupid TAT)

I sucessfully got the retrieval result. It turns out to be only 55 files (weird). And in each file, there are 1000 lines, which match with SHREC16 requirement. But the sorted ids are strange, all of them are double digits, e.g.

file 000000
000000 0.000000
000048 0.065430
000054 0.100580
000052 0.107867
000036 0.122333
000050 0.124427
000044 0.126996
000054 0.128756
000040 0.146189
000053 0.147754
000010 0.148218
000033 0.149304
000052 0.155951
000031 0.157031
000030 0.157686
000050 0.161438
000014 0.163000
000038 0.166047
000007 0.166474
000053 0.166583
000011 0.169018
000042 0.170448
000052 0.171148
000041 0.171711
000046 0.173249
000003 0.174657
000035 0.175464
000009 0.177142
000009 0.177213
000018 0.179162
000042 0.179293
000016 0.180040
000050 0.180699
000040 0.181859
000034 0.183287
000021 0.184157
000014 0.186218
000005 0.189774
000044 0.191259
000005 0.192165
......

I m working on this problem now, if you have any idea please let me know. I will update if I find anything.

Thanks for being patient :)

modelnet/options.py

We don't have modelnet40-normal_numpy file! In the Pointnet++,the name of the file is modelnet40-normal_resampled，it is different with yours!

dataset download issue

When I click the link 'modelnet10/40'

How to train the model on SHREC2016

Thanks for your work,
I'm running the Shape Retrieval task but I get the following error:

and I get this error (autoencoder/train)

why your prepared dataset can not be download?

How to test the model on SHREC2016 retrieval challenge?

Hello, what a great work you have done. But I still have some questions about testing the model and get the result on SHREC2016 retrieval task. I have run test.py and get those .txt files, also I have downloaded the evaluation files from the website, then what should I do next? Can I directly run the node js file as described in its ReadMe file as "./evaluate.js path/to/dir/with/results/methodName/"? And the path is the directory where I save the folders for all the models and it's retrieval results right?
Looking forward to your reply!

How to get the input of the autoencoder?

Hello:
I really want to know how to get the input(e.g node,node_knn_I) of the autoencoder when I use the original CAD model.
Any help would be appreciated, thanks.

About the real-time nature of the network

Hello!

I read your paper and I think your work is really great!

I have a question, your input in the paper is 5000 x 3, but in reality the number of point clouds is unknown. Can this network be used to identify obstacles in real time? Can you easily reveal the real-time nature of this network?
Looking forward to your reply!

**Segmentation Fault** in autoencoder

Hi, thanks for sharing the source code. I have encounted some problems while implementing auto-encoder script.
The environmetal setting is as follows:
---Pytorch 0.4.1
---faiss is installed by anaconda

When I use the model40 dataset, there exist several problems, like
AttributeError: 'Namespace' object has no attribute 'rot_horizontal'
After setting this to False, then the script can run and meets new problem:

------------ Options -------------
activation: relu
batch_size: 8
bn_momentum: 0.1
bn_momentum_decay: 0.6
bn_momentum_decay_step: None
checkpoints_dir: ./checkpoints
classes: 40
dataroot: ../data/modelnet40-normal_numpy/
dataset: modelnet
device: cuda:0
display_id: 200
display_winsize: 256
dropout: 0.5
feature_num: 1024
gpu_id: 0
input_pc_num: 1024
k: 3
lr: 0.001
nThreads: 8
name: train
node_num: 64
normalization: batch
output_conv_pc_num: 1024
output_fc_pc_num: 256
output_pc_num: 1280
random_pc_dropout_lower_limit: 1
rot_horizontal: True
rot_perturbation: True
som_k: 9
som_k_type: avg
surface_normal: True
translation_perturbation: True
-------------- End ----------------
#training point clouds = 9843
Segmentation Fault

Is there anything I forget to modify, I hope you can help me to solve this problem, thanks.

About SOM init_weighting_matrix

why self.init_weighting_matrix = torch.FloatTensor(self.node_num, self.rows, self.cols)?

I think self.init_weighting_matrix = torch.FloatTensor(dim, self.rows, self.cols)

SO-Net/util/som.py

Line 41 in 8afd487

self.init_weighting_matrix = torch.FloatTensor(self.node_num, self.rows, self.cols) # node_numxrowsxcols

question about decoder with conv?why the less params

the branch of conv in decoder.the first conv's prams size is 102433*1024 ,lager than the decoder with fc,why u use the conv branch ?why did u directly use fc?

Has anyone successfully reproduced the code for the sorting task for this article on modelnet40??

Has anyone successfully reproduced the code for the sorting task for this article on modelnet40??

Question about point cloud transformation

Hi, I'm very new to deal with point cloud. So maybe my question is not quite related to your repository.
I want to ask if I have a point cloud(Nx3), a rotation matrix(3x3) and translation matrix(1x3). How could I get the point cloud after the transformation(rotation and translation→4X4)?
For rotation, I guess just multiply with the matrix? And how about translation?
Your help will be highly appreciated.

SOM training

Hi, thanks for your jobs!
I have one question about SOM. In /data/build_som/save_som.ipynb your provide an instruction to train the som, but it seems that the som for each pc was trained individually. I am not familiar with som, shouldn't it be a whole class to train together more reasonable?

how to train som with my own data?

I have seen some codes in /data/build_som/util/, it seems som.py is the file for training som. But I'm not sure about how to use it. Can you offer some instructions like training som with modelnet?

Category.txt file?

Hi, where can I find the category.txt file mentioned in https://github.com/lijx10/SO-Net/blob/master/data/modelnet_shrec_loader.py#L73

can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Thanks for your great work, but when I run part-seg code "python train.py", there is an error below, I can't figure it out. I would appreciate it if you could give some advice. Thank you again.

Traceback (most recent call last):
File "train.py", line 73, in
visualizer.plot_current_errors(epoch, float(epoch_iter) / dataset_size, opt, errors)
File "/data/zdw/code/Reproduce/SO-Net-master/util/visualizer.py", line 78, in plot_current_errors
print(np.array(self.plot_data['Y']))
File "/data/zdw/env/pytorchre/lib/python3.8/site-packages/torch/_tensor.py", line 757, in array
return self.numpy()
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

ModelNet10 dataset issue

Hi, Jiaxin, Thanks for your work and sharing your code. You use ModelNet10 dataset in your experiment, may you share the ModelNet10 dataset? It seems that the dataset released by Princeton could not be used directly. And I could not find it under PointNet2 repository.

Compile ‘index_max’ fail.

Thanks for your working and new uploads ~

When compiling in file 'index_max_ext', I fail like below,

I cannot find why I fail. Do I have to use this new extension? I remember last time I run this code it does not need to be done.

Thanks and regards.

Data augmentation not work?

Hello all,

It seems that data augmentation for translation and rotation did not work in SO-Net, even worse, enabling rot_horizontal, rot_pertubation, translation_pertubation in options.py hurt the final classification performance.

We all know that many models use data augmentation to increase robustness to point cloud rotation and translation. I have not tried the performance of SO-Net under the rotating point cloud yet. But I believe the absence of rotation augmentation hurts.

Any discussions could be helpful.

Error in SO-NET code

Kindly resolve this issue.

I'm running the classification task but I get the following error:
system info:
python 3.7
pytorch 1.2
cuda 10.2
faiss toolkit for cuda 10
------------ Options -------------
activation: relu
batch_size: 8
bn_momentum: 0.1
bn_momentum_decay: 0.6
bn_momentum_decay_step: None
checkpoints_dir: ./checkpoints
classes: 40
dataroot: /home/raman/Reconstruction/modelnet40-normal_numpy/
dataset: modelnet
device: cpu
display_id: 200
display_winsize: 256
dropout: 0.7
feature_num: 1024
gpu_id: 0
input_pc_num: 5000
k: 3
lr: 0.001
nThreads: 8
name: train
node_num: 64
normalization: batch
pretrain: None
pretrain_lr_ratio: 1
random_pc_dropout_lower_limit: 1
rot_horizontal: False
rot_perturbation: False
som_k: 9
som_k_type: avg
surface_normal: True
translation_perturbation: False
-------------- End ----------------
Traceback (most recent call last):
File "train.py", line 20, in 
from models.classifier import Model
File "../models/classifier.py", line 11, in 
from . import networks
File "../models/networks.py", line 17, in 
import index_max
ImportError: /home/raman/anaconda3/lib/python3.7/site-packages/index_max-0.0.0-py3.7-linux-x86_64.egg/index_max.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at19UndefinedTensorImpl10_singleton

index_max builds but fails at runtime

I'm running the classification task but I get the following error:

system info:
python 3.7
pytorch 1.2
cuda 10.2
faiss toolkit for cuda 10

------------ Options -------------
activation: relu
batch_size: 8
bn_momentum: 0.1
bn_momentum_decay: 0.6
bn_momentum_decay_step: None
checkpoints_dir: ./checkpoints
classes: 40
dataroot: /home/raman/Reconstruction/modelnet40-normal_numpy/
dataset: modelnet
device: cpu
display_id: 200
display_winsize: 256
dropout: 0.7
feature_num: 1024
gpu_id: 0
input_pc_num: 5000
k: 3
lr: 0.001
nThreads: 8
name: train
node_num: 64
normalization: batch
pretrain: None
pretrain_lr_ratio: 1
random_pc_dropout_lower_limit: 1
rot_horizontal: False
rot_perturbation: False
som_k: 9
som_k_type: avg
surface_normal: True
translation_perturbation: False
-------------- End ----------------
Traceback (most recent call last):
File "train.py", line 20, in
from models.classifier import Model
File "../models/classifier.py", line 11, in
from . import networks
File "../models/networks.py", line 17, in
import index_max
ImportError: /home/raman/anaconda3/lib/python3.7/site-packages/index_max-0.0.0-py3.7-linux-x86_64.egg/index_max.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at19UndefinedTensorImpl10_singleton

How about the Training Time?

About 3 hours with a Titan 1080.

how to install the index_max?

I follow the prompts to install index_max and the following error occurs

If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().b
ackend() use tensor.device().
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.35.32215\bin\HostX86\x64\cl.exe' failed with exit code 2

environment：windows10， pytoch3.9

OSError：library nvvm not found

When i run the program ,I met the question:OSError：library nvvm not found.What I can do?
thank you .

Error in "python setup.py install"

What should I do in such a situation?

/home1/pan-internship-1/anaconda3/envs/pytorch/lib/python3.6/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  setuptools.SetuptoolsDeprecationWarning,
/home1/pan-internship-1/anaconda3/envs/pytorch/lib/python3.6/site-packages/setuptools/command/easy_install.py:159: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing index_max.egg-info/PKG-INFO
writing dependency_links to index_max.egg-info/dependency_links.txt
writing top-level names to index_max.egg-info/top_level.txt
reading manifest file 'index_max.egg-info/SOURCES.txt'
writing manifest file 'index_max.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'index_max' extension
Emitting ninja build file /home1/pan-internship-1/3D/SO-Net/models/index_max_ext/build/temp.linux-x86_64-3.6/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
1.10.2.git.kitware.jobserver-1
g++ -pthread -shared -L/home1/pan-internship-1/anaconda3/envs/pytorch/lib -Wl,-rpath=/home1/pan-internship-1/anaconda3/envs/pytorch/lib,--no-as-needed /home1/pan-internship-1/3D/SO-Net/models/index_max_ext/build/temp.linux-x86_64-3.6/index_max.o /home1/pan-internship-1/3D/SO-Net/models/index_max_ext/build/temp.linux-x86_64-3.6/index_max_cuda.o -L/home1/pan-internship-1/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib -L/home1/pan-internship-1/cuda/cuda_10.1/lib64 -L/home1/pan-internship-1/anaconda3/envs/pytorch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -lpython3.6m -o build/lib.linux-x86_64-3.6/index_max.cpython-36m-x86_64-linux-gnu.so
g++: error: /home1/pan-internship-1/3D/SO-Net/models/index_max_ext/build/temp.linux-x86_64-3.6/index_max.o: No such file or directory
g++: error: /home1/pan-internship-1/3D/SO-Net/models/index_max_ext/build/temp.linux-x86_64-3.6/index_max_cuda.o: No such file or directory
error: command 'g++' failed with exit status 1```

Sampling code non-uniform?

I might be mistaken, but from what I can tell the sampling code appears to be non-uniform for each triangle. Results for a single triangle shown below. See here for what I think is the issue.

For fine-detailed meshes I doubt it will make a big difference, but for any meshes dominated by large surfaces I feel things could get hairy.

About idx2multi

SO-Net/util/som.py

Line 80 in 8afd487

return (i // self.cols, i % self.cols)

It should be (i // self.rows, i % self.cols)