Comments (1)
Hello. Thank you for your excellent paper, which I read with great interest.
I conducted experiments to reproduce the performance described in the paper.
As a result, I found that the performance difference was significantly beyond the range of variation. (GPU used: RTX TITAN)
Training with MVTecAD and testing with VisA resulted in the following performance: 2-shot: 0.820 (AUROC), 0.837 (AUPRO), paper: 0.858(AUROC), 0.877(AUPRO) 4-shot: 0.832 (AUROC), 0.849 (AUPRO), paper: 0.877(AUROC), 0.902(AUPRO) 8-shot: 0.827 (AUROC), 0.848(AUPRO), paper: 0.887(AUROC), 0.904(AUPRO)
In addition, I did not change the default settings.
There might be an issue with the command I used, so I am attaching it as well. (I trained with all classes of MVTecAD and tested by VisA, The shot was directly modified in main.py.)
train command: python main.py --normal_json_path D://InCTRL-main//data//mvtec_json//AD_json//bottle_normal.json D://InCTRL-main//data//mvtec_json//AD_json//cable_normal.json D://InCTRL-main//data//mvtec_json//AD_json//capsule_normal.json D://InCTRL-main//data//mvtec_json//AD_json//carpet_normal.json D://InCTRL-main//data//mvtec_json//AD_json//grid_normal.json D://InCTRL-main//data//mvtec_json//AD_json//hazelnut_normal.json D://InCTRL-main//data//mvtec_json//AD_json//leather_normal.json D://InCTRL-main//data//mvtec_json//AD_json//metal_nut_normal.json D://InCTRL-main//data//mvtec_json//AD_json//pill_normal.json D://InCTRL-main//data//mvtec_json//AD_json//screw_normal.json D://InCTRL-main//data//mvtec_json//AD_json//tile_normal.json D://InCTRL-main//data//mvtec_json//AD_json//toothbrush_normal.json D://InCTRL-main//data//mvtec_json//AD_json//transistor_normal.json D://InCTRL-main//data//mvtec_json//AD_json//wood_normal.json D://InCTRL-main//data//mvtec_json//AD_json//zipper_normal.json --outlier_json_path D://InCTRL-main//data//mvtec_json//AD_json//bottle_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//cable_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//capsule_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//carpet_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//grid_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//hazelnut_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//leather_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//metal_nut_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//pill_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//screw_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//tile_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//toothbrush_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//transistor_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//wood_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//zipper_outlier.json --val_normal_json_path D://InCTRL-main//data//mvtec_json//AD_json//bottle_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//cable_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//capsule_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//carpet_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//grid_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//hazelnut_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//leather_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//metal_nut_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//pill_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//screw_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//tile_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//toothbrush_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//transistor_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//wood_val_normal.json D://InCTRL-main//data//mvtec_json//AD_json//zipper_val_normal.json --val_outlier_json_path D://InCTRL-main//data//mvtec_json//AD_json//bottle_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//cable_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//capsule_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//carpet_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//grid_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//hazelnut_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//leather_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//metal_nut_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//pill_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//screw_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//tile_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//toothbrush_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//transistor_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//wood_val_outlier.json D://InCTRL-main//data//mvtec_json//AD_json//zipper_val_outlier.json
test command(repeated for each class): python test.py --val_normal_json_path D://InCTRL-main//data//visa_json//AD_json//macaroni2_val_normal.json --val_outlier_json_path D://InCTRL-main//data//visa_json//AD_json//macaroni2_val_outlier.json --category macaroni2 --few_shot_dir D:/InCTRL-main/data/visa_inference/visa/8/
Is there an issue with the way I used the command?
Hi, I meet the same problem. Have you found the reason?
from inctrl.
Related Issues (20)
- Can you provide code for visual inspection HOT 3
- Guidance on Training and Testing with Custom Dataset Similar to MVTec Format
- could you share how to visualize the segment result when inference? HOT 1
- Hi, you can use **_torch.save()_** to generate .pt file for your own few-shot samples. HOT 2
- The performance of using 4-shot or 8-shot on the Visa dataset is similar to that of 2-shot HOT 2
- Can you provide the test set about Visa, ELPV, SDD, AITEX? HOT 3
- the training process HOT 1
- lower performance on 2-shot visa-candle with the default setting (e.g., pre-trained model and few-shot prompts)
- Is the model trained on the full dataset of MVTec available?
- Few-shot Normal Images for Inference. HOT 5
- License HOT 1
- 在测试过程中,可以不传入--few_shot_dir这个参数吗?
- Is it possible to not pass in the --few_shot_dir parameter during testing? HOT 1
- how to test one img? HOT 2
- json generation fails HOT 6
- Permission denied error because of CUDA?
- Few-shot Normal Samples for Inference - MVTec HOT 2
- INT8
- About few-shot normal images
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from inctrl.