Request for AFLChurn jobs

Successful integration of regressed bugs for AFLChurn

If you want to add new fuzz targets, remove --depth 1 in Dockerfile of the new target projects.

Regressed:

Non-regressed:

Others:

Install

Prerequisites

Docker
python >= 3.8.0

Install requirements

make install-dependencies
source .venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
make install-dependencies
deactivate

Update submodules

oss-fuzz is a submodule

git pull
git submodule update --init
git submodule update --remote --merge

Verification

sudo make presubmit

Fuzzers to run

afl
aflchurn

Run fuzzers

example:

sudo make run-aflchurn-file_magic_fuzzer

For more details in fuzzbench guide.

Remove history images/containers/caches if you want

CAUTION: This will remove all images created by fuzzbench. You can skip this step if you don't want to.

sudo docker rm $(sudo docker ps -qa --no-trunc --filter "status=exited")
sudo docker rmi -f $(sudo docker images | grep -e gcr -e none | sed 's/  */ /g' | cut -d" " -f3 | sort | uniq)
sudo docker builder prune

Get bug logs and time to bug

Change crash_plotdata_filestore accordingly in file experiment-config.yaml. crash_plotdata_filestore is the folder including experiment results.

crash test cases: e.g., $crash_plotdata_filestore/openssl_x509-afl/trial-753301/corpus/crashes/id*
plot_data: e.g., $crash_plotdata_filestore/openssl_x509-afl/trial-753301/corpus/plot_data

First, remove file docker/generated.mk if experiment-config.yaml is changed

If one changes crash_plotdata_filestore in experiment-config.yaml, remove docker/generated.mk to enable the change of crash_plotdata_filestore. The docker/generated.mk will be auto-generated after being removed.

Then, get bug logs with afl_debug

Get bug logs

sudo make churn-debug-afl_debug-[subject]

Then, get time2bug

Calculate time to bug

./time2bug.sh

Run AFLgo

Currently, aflgo can only run on some subjects, which are

libgit2_objects_fuzzer
libhtp_fuzz_htp

example:

sudo make run-aflgo-libgit2_objects_fuzzer

FuzzBench: Fuzzer Benchmarking As a Service

FuzzBench is a free service that evaluates fuzzers on a wide variety of real-world benchmarks, at Google scale. The goal of FuzzBench is to make it painless to rigorously evaluate fuzzing research and make fuzzing research easier for the community to adopt. We invite members of the research community to contribute their fuzzers and give us feedback on improving our evaluation techniques.

FuzzBench provides:

An easy API for integrating fuzzers.
Benchmarks from real-world projects. FuzzBench can use any OSS-Fuzz project as a benchmark.
A reporting library that produces reports with graphs and statistical tests to help you understand the significance of results.

To participate, submit your fuzzer to run on the FuzzBench platform by following our simple guide. After your integration is accepted, we will run a large-scale experiment using your fuzzer and generate a report comparing your fuzzer to others. See a sample report.

Overview

Sample Report

You can view our sample report here and our periodically generated reports here. The sample report is generated using 10 fuzzers against 24 real-world benchmarks, with 20 trials each and over a duration of 24 hours. The raw data in compressed CSV format can be found at the end of the report.

When analyzing reports, we recommend:

Checking the strengths and weaknesses of a fuzzer against various benchmarks.
Looking at aggregate results to understand the overall significance of the result.

Please provide feedback on any inaccuracies and potential improvements (such as integration changes, new benchmarks, etc.) by opening a GitHub issue here.

Documentation

Read our detailed documentation to learn how to use FuzzBench.

Contacts

Join our mailing list for discussions and announcements, or send us a private email at [email protected].

afosscontact / aflchurnbench Goto Github PK

aflchurnbench's Introduction