peizhishi/msvnet

A novel learning-based feature recognition method using multiple sectional view representation

MsvNet

*Update (15/10/2020): Please check out our recent work published on IEEE TII (paper, code), which yields better results than the MsvNet.

Created by Peizhi Shi at University of Huddersfield

Acknowledgements: We would like to thank Zhibo Zhang for providing the dataset and source code of FeatureNet on Github.

You will NOT utilise the code for military, nuclear, missile, animal slaughter, meat production, weaponry end uses or conduct any other activities involving the code where human/animal life or property may be at stake.

Introduction

The MsvNet is a novel learning-based feature recognition method using multiple sectional view representation. At the time of its release, the MsvNet achieves the state-of-the-art single feature recognition results by using only a few training samples, and outperforms the state-of-the-art learning-based multi-feature recognition method in terms of recognition performances.

This repository provides the source codes of the MsvNet for both single and multi-feature recognition, a reimplemented version of the FeatureNet for multi-feature recognition, and a benchmark dataset which contains 1000 3D models with multiple features.

If this project is useful to you, please consider citing our paper:

@article{shi2020novel,
  title={A novel learning-based feature recognition method using multiple sectional view representation},
  author={Shi, Peizhi and Qi, Qunfen and Qin, Yuchu and Scott, Paul J and Jiang, Xiangqian},
  journal={Journal of Intelligent Manufacturing},
  volume={31},
  number={5},
  pages={1291--1309},
  year={2020},
  publisher={Springer}
}

This is a peer-reviewed paper, which is available online.

Experimental configuration

1. CUDA (10.0.130)
2. cupy-cuda100 (6.2.0)
3. numpy (1.17.4)
4. Pillow (6.2.1)
5. python (3.6.8)
6. pyvista (0.22.4)
7. scikit-image (0.16.2)
8. scipy (1.3.3)
9. selectivesearch (0.4)
10. tensorflow-estimator (1.14.0)
11. tensorflow-gpu (1.14.0)
12. torch (1.1.0)
13. torchvision (0.3.0)

All the experiments mentioned in our paper are conducted on Ubuntu 18.04 under the above experimental configurations. If you run the code on the Windows or under different configurations, slightly different results might be achieved.

Single feature recognition

1. Get the MsvNet source code by cloning the repository: git clone https://github.com/PeizhiShi/MsvNet.git.
2. Download the FeatureNet dataset, and convert them into voxel models via binvox. The filename format is label_index.binvox. Then put all the *.binvox files in a same folder data/64/. 64 refers to the resolution of the voxel models. This folder is supposed to contain 24,000 *.binvox files. Please note there are some unlabelled/mislabelled files in category 8 (rectangular_blind_slot) and 12 (triangular_blind_step). Before moving these files in the same folder, please correct these filenames.
3. Run python single_train.py to train the neural network. Please note that data augmentation is employed in this experiment. Thus, the training accuracy is lower than the val/test accuracy.

Benchmark dataset for multi-feature recognition

1. Get the MsvNet source code by cloning the repository: git clone https://github.com/PeizhiShi/MsvNet.git.
2. Download the benchmark multi-feature dataset, and put them in the folder data/.
3. Run python visualize.py to visualize a 3D model in this dataset.

Multi-feature recognition

1. Get the MsvNet source code by cloning the repository: git clone https://github.com/PeizhiShi/MsvNet.git.
2. Download the benchmark multi-feature dataset, and put them in the folder data/.
3. Download the pretrained optimal MsvNet and FeatureNet models, and put them int the folder models/. These models are trained under the optimal settings (instead of near-optimal settings) mentioned in our paper, which could produce the multi-feature recognition results reported in the paper.
4. Run python multi_test.py to test the performances of the MsvNet and FeatureNet for multi-feature recognition. Please note that the multi-feature recognition part of the FeatureNet is only a reimplemented version. Watershed algorithm with the default setting is employed. Detailed information about the FeatureNet can be found from their original paper.

If you have any questions about the code, please feel free to contact me ([email protected]).

msvnet's Issues

运行single_train.py报错

作者您好，我在运行single_train.py后报错，暂时不知道如何解决，您能帮我看一下什么问题吗
运行后打印输出:
`
Downloading: "https://download.pytorch.org/models/vgg11-bbd30ac9.pth" to C:\Users\41475/.cache\torch\checkpoints\vgg11-bbd30ac9.pth
100.0%
Stage 1: fine tuning the neural network ...

Epoch: 0
Stage 1: fine tuning the neural network ...
接着报错如下：
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
ForkingPickler(file, protocol).dump(obj)

BrokenPipeError: [Errno 32] Broken pipe
`

Checkpoint file.

Hello,
First of all, MSVnet is impressive and thanks for sharing the algorithm.

I was using the code for a while and classifications are pretty perfect with the featurenet dataset. But i would like to use my dataset and would like to evaluate the performance for my application.
Since the checkpoint file saving part it not available in the source code, if u could provide me the respective part also it would be gratefully. (i tired my own way of custom checkpoint file creation and parameter saving, but failed)

Thanks in Advance :) .

How did you created .csv file for a specific model, what values it contains

We are working on multi feature recognition and using MsvNet for that.

I have downloaded dataset 'data' from here

This dataset contains directories as 'set1', 'set2', ....., 'set10' . Each set has models with multiple features with STL and binvox formats.

Each set contains csv file for each model in sets.

trul = get_lvec(vis.get_label('data/set'+str(sidx)+'/'+str(idx)+'.csv',0)[:,6]).astype(int)
print('True labels:\t\t',trul)

These snippet from 'lins.multi.py' from your project reads csv file and returns array of 0 & 1 in length of 24. 1 is for recognized feature.

After testing some models I conclude that these true labels from csv are accurate. Great work!

Can you guys elaborate,

1. How to create that csv file?
2. What values does csv contains? (I found that last value of each line in the csv is index of recognized feature)
3. How should I improve predicted labels towards accuracy?

Thanks in advance.