Comments (4)
Hi! It's certainly possible to hit OOM if the samples are large and the corpus keeps growing. However, I'm curious if you can share more about your setup (either here or privately), because I used the grammar engine to fuzz reasonably complex targets (such as v8 javascript engine for example) and didn't encounter such errors in practice (IIRC the fuzzing workers had 8 or 16GB RAM each, not sure anymore).
Jackalope has some mechanisms to decrease RAM usage such as -keep_samples_in_memory=0
, however this might not give you much with the grammar mode. The interesting_trees
you mentioned keeps all samples in the corpus in the tree form. This is so that these trees can be combined with the currently fuzzed sample to generate new interesting samples. In the interest of saving memory, you could simply comment out this line
However, there are several other, easier things you can do to to reduce memory usage in grammar fuzzing mode
- reduce the value of MAX_DEPTH
Jackalope/mutators/grammar/grammar.h
Line 14 in bd49b2c
- Reduce the value of MINIMIZATION_LIMIT in . This describes when the minimizer stops, e.g. the value of 1500 means that the minimizer will stop when there is less then 1500 nodes in the tree, even if the sample could be minimized further. If significantly reducing this value does not make samples in the corpus smaller, then it could mean that the minimizer is not working properly.
Having said all of the above, if there is a memory leak somewhere, of course this would be considered a bug.
from jackalope.
Thank you for your reply! I combined the Jackalope grammar mutation strategy
with AFL
(alias as afl-jackalope
)for better fuzzing binaries with source code. So I'm sorry for not being able to provide OOM setups as I'm not using Jackalope directly.
For Jackalope itself, simply removing interesting_trees.push_back(context->tree)
may not reduce memory consumption, because context
is still held by other data structures, such as sample_queue
.
For afl-jackalope, if interesting_trees
's size reaches a certain value, then I just make interesting_trees
simply delete
half of nodes to prevent from OOM, now it works quite well.
I read the source code of jackalope carefully, although there are quite a few structures that never release memory, their life cycles are quite correct. So there is no hint that there is any memory issue inside.
I have solved the problem about OOM. However, I still have some questions about the corpus.
- Can jackalope run without inputs?
- Is there any way to convert from
common JS
toserialized tree
(jackalope input format)? - Would the input of jackalope have an impact on its mutation process? And if so, what is the approximate impact?
Please excuse my rough English. Best wishes for you! :)
from jackalope.
Thanks for reviewing the code :)
"Can jackalope run without inputs?"
Not sure what you mean, did you mean with inputs. Because in the grammar fuzzing mode, it's actually expected that Jackalope will start without inputs; and the initial inputs will be generated using the provided grammar. See more info in this thread #26
"Is there any way to convert from common JS to serialized tree (jackalope input format)?"
As noted in the other thread, there is no guaranteed unique way to parse a sample generated by a context-free grammar back into its grammar representation. We could maybe do a "best effort", but this is not implemented in Jackalope currently.
"Would the input of jackalope have an impact on its mutation process? And if so, what is the approximate impact?"
Sorry, not sure what you mean here - can you give me an example?
from jackalope.
Thanks for your kind reply. The last question means whether the corpus is important for the fuzzing process or coverage results. For example, in AST fuzzer, good corpus inputs can significant improve the performance of fuzzer, while junk corpus inputs make littile contribution to the performance of entire fuzzing process.
Now I can probably deduce the answer to the last question with your kind answer and great code implementation.
Thanks for your artwork and patient reply. I have no other questions.
Best wishes.
from jackalope.
Related Issues (20)
- PROGRAM ABORT : No interesting input files HOT 1
- The value of FUZZ_CHILD_CTRL_IN / OUT is too low. HOT 1
- Question on Grammar-based mutation HOT 2
- Update README instructions for building on macOS HOT 13
- How to solve create process failed, GLE=2 error HOT 9
- sir fuzzer is running but errors are not identifying..
- Investigating performance HOT 25
- Wrong crashes count? HOT 1
- How can I add new samples to an existing session? HOT 2
- Help with running as server HOT 9
- Handling Wide strings HOT 7
- WARNING: Error delivering sample, retrying with a clean target
- Implementing network fuzzing HOT 2
- Program abort due to `Process hanged before reaching the target method` HOT 1
- Adding extensions list HOT 2
- "process dead" issue that is not occured by WinAFL or other Fuzzer HOT 6
- Question: Would Jackalope be able to track code coverage for a child process spawned by the target? HOT 2
- PROGRAM ABORT : Process exited before reaching the target method - due to partial memory read from process HOT 3
- Fuzzer quits without saving crash HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jackalope.