Comments (8)
I’ll finish the ABS docs soon. Hope that will help you.
from trojanzoo.
Sorry for not replying in time. In hospital for a few days. Will take a look tonight.
From provided commands and logs, I find several suspicious points, that might be helpful.
-
You are using old version trojanzoo.
--pretrain
is replaced by--pretrained
in new version to keep consistent with torchvision. And new version trojanzoo supports python 3.10 only. You may use the last release version that supports 3.9 if you don’t want to change.I’ve reimplemented abs in recent version and tested. It should be working fine. I’ll later add a log file.
-
In defense log, the clean acc listed in attack and defense summary is 76, which is strange. It should be consistent with training result 96.
from trojanzoo.
Here's the result.
I just updated the print sentences to make it prettier.
Note the current implementation should be consistent with original author's code.
While TrojanZoo paper's ABS result is actually my own implementation after reading the paper (Because the original authors didn't release their codes at that time). That implementation is more similar to Neural Cleanse and therefore its performance is good. But it should NOT be.
From the result, we can observe that class 0 (target class) has a 30% ASR watermark, while class 2 has a 40%.
That 40% backdoor does exist in the poisoned model, but Neural Cleanse can't detect that interestingly.
from trojanzoo.
Hi, I hope you're fine now.
Thanks a lot for your time. I'm investigating the problem and will tell you the result.
from trojanzoo.
- In defense log, the clean acc listed in attack and defense summary is 76, which is strange. It should be consistent with training result 96.
I realized that the defense code first prints the summary, then the detect
method of the defense class loads the trojaned model.
from trojanzoo.
I realized that the defense code first prints the summary, then the detect method of the defense class loads the trojaned model.
In your case, you call --pretrained
, so the model will load clean pretrained weights in model.__init__()
. And clean_acc
of attack
and defense
should be set as this value as well. After you call defense.detect()
, It will load poisoned model weights.
I've moved the model loading from detect()
to __init__()
And btw, you don't need to set --pretrained
in command line
trojanzoo/trojanvision/defenses/abstract.py
Lines 58 to 61 in 2698c22
from trojanzoo.
Now it's OK for me, but just FYI, I ran it without --pretrain and faced a similar issue. I think that in this case, it uses a randomly initialized model at first and prints its summary.
Anyway, thank you for investigating the issue.
from trojanzoo.
I'll close this issue if there is no further question.
from trojanzoo.
Related Issues (20)
- BackdoorAttack class has no argument for source_class HOT 1
- Low effective loading in get_class_subset function HOT 1
- Install newest version fail HOT 1
- Using a custom model HOT 4
- RuntimeError: Dataset not found or corrupted. You can use download=True to download it HOT 10
- Clean label attack accuracy is wrong HOT 5
- In new push model path is not working HOT 1
- badnet folder information HOT 1
- [Error] When I test Neural Cleanse i got a error HOT 2
- Is it possible to apply methods to graph? HOT 6
- Input aware dynamic backdoor error HOT 5
- trojanvision.datasets.ImageFolder HOT 1
- Possible bug: target_class not changed when computing ASR for reversed triggers HOT 2
- problem about saving the intermediate results and config problem HOT 6
- strange mark saved HOT 2
- Hyperparameters for training Resnet18 on CIFAR10? HOT 1
- STRIP implementation doesn't match original codebase HOT 1
- Attack saving and loading is not working HOT 2
- Comp version of networks HOT 2
- Unable to Access Triggered Dataset in BadNet Attack HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trojanzoo.