softsec-kaist/codealchemist

CodeAlchemist: Semantics-Aware Code Generation to Find Vulnerabilities in JavaScript Engines (NDSS '19)

CodeAlchemist

CodeAlchemist is a JavaScript engine fuzzer that improves classic grammar-based JS engine fuzzers by a novel test case generation algorithm, called semantics-aware assembly. The details of the algorithm is in our paper, "CodeAlchemist: Semantics-Aware Code Generation to Find Vulnerabilities in JavaScript Engines", which appeared in NDSS 2019. This is a stable version of CodeAlchemist and it currently supports ChakraCore, V8, SpiderMonkey, and JavaScriptCore.

Installation

CodeAlchemist currently works on only Linux and we tested on Ubuntu 18.04.

1. Install dependencies (build-essential, nodejs, npm, esprima, dotnet)

$ sudo apt update
$ sudo apt install build-essential
$ sudo apt install nodejs npm
$ npm i [email protected]

Installation for dotnet depends on OS version, so please refer this link.

2. Clone and build CodeAlchemist

$ git clone https://github.com/SoftSec-KAIST/CodeAlchemist
$ cd CodeAlchemist
$ make

Usage

1. Prepare to start

Prepare JS seed files, a configuration file, and the requirements in the configuration. Please refer conf/README.md for writing the configuration file.

2. Preprocess JS seed files

$ dotnet bin/Main.dll rewrite <conf ABSPATH>
$ dotnet bin/Main.dll instrument <conf ABSPATH>

3. Run fuzzing process

$ dotnet bin/Main.dll fuzz <conf ABSPATH>

There are four optional parameters for our JS code generation algorithm.

• iMax (default: 8): The maximum number of iterations of the generation algorithm.
• pBlk (default: 16): The probability of reinventing block statements.
• iBlk (default: 3): The maximum number of iteration for generating a block statement.
• dMax (default: 3): The maximum nesting level for a reassembling block statement.

You can specify parameters with following commands.

$ dotnet bin/Main.dll fuzz <conf ABSPATH> --iMax 8 --pBlk 16 --iBlk 3 --dMax 3

CVEs (Credits)

If you find bugs and get CVEs by running CodeAlchemist, please let us know by sending a PR for ./docs/CVE.md.

• JavaScriptCore: CVE-2018-4464, CVE-2018-4437, CVE-2018-4378, CVE-2018-4372

Authors

This research project has been conducted by SoftSec Lab at KAIST.

• HyungSeok Han
• DongHyeon Oh
• Sang Kil Cha

Citation

If you plan to use CodeAlchemist in your own research. Please consider citing our paper:

@INPROCEEDINGS{han:ndss:2019,
  author = {HyungSeok Han and DongHyeon Oh and Sang Kil Cha},
  title = {{CodeAlchemist}: Semantics-Aware Code Generation to Find Vulnerabilities in JavaScript Engines},
  booktitle = ndss,
  year = 2019
}

codealchemist's Issues

I met some problem when I used CodeAlchemist to fuzz Chakra

When I entered "sudo dotnet bin/Main.dll rewrite , there were something errors here. May I have your help to solve it please?

Are you willing to make the dataset public? And some questions about paper.

Firstly, thanks for your work, it's amazing.

And as the title says, It is helpful if you can make the dataset public. I mean, the 63,692 js seed files you mentioned in your paper.

Thank you in advance!

Need more diverse seeds

HI. When i use CodeAlchemist, it tells me "Need more diverse seeds", how many seeds is enough atl least?

IndexOutOfRangeException happen when run fuzz

when run fuzz a error happen. below is the error info.

Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at Fuzzer.Selector.pickBrick[a,b,c,d,e,f](Random rnd, Tuple`2[] pool_0, Tuple`2[] pool_1, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Selector.fs:line 72
   at Fuzzer.Fuzzer.genStmt@10[a](Tuple`2 sPool, Random rnd, StringBuilder sb, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 11
   at Fuzzer.Fuzzer.generate@16[a](Int32 iBlk, Tuple`2 sPool, Tuple`2[] gPool, Random rnd, Int32 pBlk, StringBuilder sb, Int32 i, Int32 d, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 22
   at [email protected](StringBuilder sb, Int32 i, Int32 d, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 16
   at Microsoft.FSharp.Core.FSharpFunc`2.InvokeFast[V,W,X](FSharpFunc`2 func, T arg1, TResult arg2, V arg3, W arg4)
   at [email protected](Unit unitVar) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 53
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvoke[T,TResult](AsyncActivation`1 ctxt, TResult result1, FSharpFunc`2 part2)
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit()
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation)
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout)
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
   at Fuzzer.Fuzzer.fuzz(Conf conf, FSharpList`1 bricks) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 63
   at Main.Run.main(String[] argv) in /home/test/Desktop/CodeAlchemist/src/Main/Main.fs:line 74
Aborted (core dumped)
test@test:~/Desktop/CodeAlchemist$ dotnet bin/Main.dll fuzz   /home/test/Desktop/CodeAlchemist/conf/MOZ.conf

Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at Fuzzer.Selector.pickBrick[a,b,c,d,e,f](Random rnd, Tuple`2[] pool_0, Tuple`2[] pool_1, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Selector.fs:line 72
   at Fuzzer.Fuzzer.genStmt@10[a](Tuple`2 sPool, Random rnd, StringBuilder sb, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 11
   at Fuzzer.Fuzzer.generate@16[a](Int32 iBlk, Tuple`2 sPool, Tuple`2[] gPool, Random rnd, Int32 pBlk, StringBuilder sb, Int32 i, Int32 d, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 22
   at [email protected](StringBuilder sb, Int32 i, Int32 d, ValueTuple`5 ctx) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 16
   at Microsoft.FSharp.Core.FSharpFunc`2.InvokeFast[V,W,X](FSharpFunc`2 func, T arg1, TResult arg2, V arg3, W arg4)
   at [email protected](Unit unitVar) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 53
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvoke[T,TResult](AsyncActivation`1 ctxt, TResult result1, FSharpFunc`2 part2)
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit()
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation)
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout)
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
   at Fuzzer.Fuzzer.fuzz(Conf conf, FSharpList`1 bricks) in /home/test/Desktop/CodeAlchemist/src/Fuzzer/Fuzzer.fs:line 63
   at Main.Run.main(String[] argv) in /home/test/Desktop/CodeAlchemist/src/Main/Main.fs:line 74
Aborted (core dumped)

my conf file:

  "engine_path": "/home/test/Desktop/gecko-dev/js",
  "argv": [],
  "env": { },
  "seed_path": "/home/test/Desktop/path/seed",
  "preproc_dir": "/home/test/Desktop/path/preproc",
  "tmp_dir": "/home/test/Desktop/path/tmp",
  "bug_dir": "/home/test/Desktop/path/bug",

Exception when run the command dotnet bin/Main.dll rewrite ~/CodeAlchemist/conf/Chakra.conf

Hi, when I run the following command:
$dotnet bin/Main.dll rewrite ~/CodeAlchemist/conf/Chakra.conf
it reports the following error. Do you know what is wrong here? Thank you.
Unhandled Exception: System.ComponentModel.Win32Exception: No such file or directory
at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start()
at Common.ConfModule.getBuiltIns(String binPath, String[] argv) in /home/s2e/CodeAlchemist/src/Common/Conf.fs:line 62
at Common.ConfModule.load(String fname, Int32 pBlk, Int32 iBlk, Int32 iMax, Int32 dMax) in /home/s2e/CodeAlchemist/src/Common/Conf.fs:line 70
at Main.Run.main(String[] argv) in /home/s2e/CodeAlchemist/src/Main/Main.fs:line 84
Aborted (core dumped)

And the conf file is like this.
{
"engine": "Chakra",
"timeout": 30,
"engine_path": "/ChakraCore/out/Release/ch",
"argv": [],
"env": { },
"seed_path": "/ChakraTest/seed",
"preproc_dir": "/ChakraTest/pre",
"tmp_dir": "/ChakraTest/tmp",
"bug_dir": "~/ChakraTest/bug",
"filters": [
"load", "assert", "eval", "shouldBeFalse", "shouldNotThrow", "shouldBe",
"shouldBeNull", "shouldBeUndefined", "shouldThrow", "shouldBeTrue",
"shouldBeEqualToString", "crash" , "$ERROR", "Test", "tryItOut", "Function",
"read", "readbuffer", "readline", "console"
],
"jobs": 2
}

How to build chakracore

I build witth options "./build.sh --sanitize=address --static"
Is it right?

help

I want to test some other softwares, what can I do to make CodeAlchemist support them? like:
1）https://github.com/Moddable-OpenSource/moddable
2）https://github.com/jerryscript-project/jerryscript
3） https://github.com/pcmacdon/jsish

Error: Need more diverse seeds

Hello!

Your paper makes it a little unclear as to what the intended corpus of seeds must look like. I took a guess and used the ECMAScript conformance test suite but got the error:

[ERROR] Need more diverse seeds

Can you give some suggestions of what a valid suite of seed javascript files should look like?