ltl7155/npc

NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks

Overview of NPC :

Parameters:

In our paper, the value ranges of the three hyperparameters α, β and k are {0.7,0.8,0.9,1.0}, {0.6,0.7,0.8,0.9} and {1,4,7}, respectively.

Derive LRP path.

first, please download some pretrained weight, data and seed case from drive.google.com by our scripts.

More data will be released in future.

• download MNIST's model's weight (4.7 Mb), MNIST dataset (53 Mb), one seed-case (36Mb)
  

cd data && python download.py --download_mnist_example && cd ..

RQ1 Mask Experiment.

We can run mask_critical_units.py to get inconsistency rate after masking neurons in CDP and NCDP.

python mask_critical_units.py --paths_path ./data/LRP_path/mnist_convmnist_lrp_path_threshold0.8_train.pkl --data_train --dataset mnist --arc convmnist

python mask_critical_units.py --paths_path ./data/LRP_path/cifar10_convcifar10_lrp_path_threshold0.7_train.pkl --data_train --dataset cifar10 --arc convcifar10

python mask_critical_units.py --paths_path ./data/LRP_path/cifar10_vgg_lrp_path_threshold0.9_train.pkl --data_train --dataset cifar10 --arc vgg

python mask_critical_units.py --paths_path ./data/LRP_path/SVHN_alexnet_lrp_path_threshold0.7_train.pkl --data_train --dataset SVHN --arc alexnet

python mask_critical_units.py --paths_path ./data/LRP_path/imagenet_vgg16_bn_lrp_path_threshold0.7_train.pkl --data_train --dataset imagenet --arc vgg16_bn

RQ2 Cluster.

We can run cluster_three_level_mask.py to cluster theses paths and derive the abstract path.

python cluster_three_level_mask.py --paths_path ./data/LRP_path/mnist_convmnist_lrp_path_threshold0.8_train.pkl --arc convmnist --b_cluster --dataset mnist --gpu 1 --n_clusters 4 --threshold 0.8 --grids 5 --data_train

python cluster_three_level_mask.py --paths_path LRP_path/cifar10_convcifar10_lrp_path_threshold0.7_train.pkl --arc convcifar10 --b_cluster --dataset cifar10 --gpu 1 --n_clusters 7 --threshold 0.9 --grids 5 --data_train

python cluster_three_level_mask.py --paths_path LRP_path/cifar10_vgg_lrp_path_threshold0.9_train.pkl --arc vgg --b_cluster --dataset cifar10 --gpu 1 --n_clusters 7 --threshold 0.9 --grids 5 --data_train

python cluster_three_level_mask.py --paths_path LRP_path/SVHN_alexnet_lrp_path_threshold0.7_train.pkl --arc alexnet --b_cluster --dataset SVHN --gpu 1 --n_clusters 4 --threshold 0.6 --grids 5 --data_train

python cluster_three_level_mask.py --paths_path LRP_path/imagenet_vgg16_bn_lrp_path_threshold0.7_train.pkl --arc vgg16_bn --b_cluster --dataset imagenet --gpu 1 --n_clusters 4 --threshold 0.7 --grids 5 --data_train

RQ3 SNPC,LSA,DSA,ANPC

In calc_NPC.py, we inplemented the SNPC. And in calc_sadl, we can run new_run_torch.py to get the LSA,DSA,ANPC metric.

• ANPC
  batch_size=64 python new_run_torch.py -nma
  
• SNPC
  batch_size=64 python calc_SNPC.py
  
• LSA(LSC) DSA(DSC)
  batch_size=64 python new_run_torch.py -lsa -dsa --last_layer
  

Citation

@article{xie2022npc,
  title={NPC: N euron P ath C overage via Characterizing Decision Logic of Deep Neural Networks},
  author={Xie, Xiaofei and Li, Tianlin and Wang, Jian and Ma, Lei and Guo, Qing and Juefei-Xu, Felix and Liu, Yang},
  journal={ACM Transactions on Software Engineering and Methodology},
  year={2022},
  publisher={ACM New York, NY}
}