assist-project/dtls-fuzzer

Protocol state machine learner and fuzzer for DTLS servers and clients

dtls-fuzzer's Issues

Alternative means of detecting SUT termination for Scandium/JSSE

The test programs for Scandium/JSSE are not the SUT themselves, but instead act as 'SUT launchers'. Hence, detecting when the SUT process terminates cannot be based the termination of the process running the test program (the test launcher stays active during the learning run). Instead, detection for these implementations relies on network errors signaled via ICMP packets.

Unfortunately, in later versions of TLS-Attacker network errors due to ICMP packets are no longer visible (see relevant issue). Hence, we need a different mechanism of detecting when the SUT is down.

install.sh not working on macOS

running install.sh returns COMPILATION WARNING: system modules path not set in conjunction with -source 11
Then
COMPILATION ERROR : (with 100 errors 'does not exists)like:
/Users/giorgiocorna/dtls-fuzzer/src/main/java/se/uu/it/dtlsfuzzer/sut/io/ClientHelloRenegotiationInput.java:[3,33] package javax.xml.bind.annotation does not exist

This can't let me install dtls-fuzzer.
All requirements (Java11, Maven...) are satisfied.

Regression tests using long-running CRON jobs

Due to time constraints, our CI tests only compare the first five hypotheses that are generated during learning. It would help to have some testing workflows that execute infrequently and periodically (e.g., once every week), which check models generated deeper in the learning process (e.g., after 20 rounds). There are guides on how this could be done using GitHub actions, for example, here.

Reparsing leads to duplication of certain arguments

DTLS-Fuzzer supports placeholder variables, adjusted via -D, which can be used to form 'configurable' arguments. One common example is the 'fuzzer.dir' and 'sut.port' placeholder variable, used throughout DTLS-Fuzzer's argument files. Below is how they are used to form the 'command' argument for Eclipse TinyDTLS servers.

-cmd "${fuzzer.dir}/suts/ctinydtls/tests/dtls-server -p ${sut.port}"

Placeholder variables take default values as configured in 'src/main/resources/dtls-fuzzer.properties', and can be set via '-D' to custom values. To support the latter case, DTLS-Fuzzer reparses the arguments according to the custom variable setting. Unfortunately, this reparsing can result in duplication of certain arguments (due to improper reset of argument-holding instances prior to reparsing).

For example, it can result in:

Equivalence Algorithms: [RANDOM_WP_METHOD, WP_METHOD, RANDOM_WP_METHOD, WP_METHOD]

OpenSSL 3.0.0

Hi, I've been interested in this fuzzer. I tried to use it to fuzz the new Openssl 3.0.0, while in the end there were only 2 hyps.

Is it need some changes to support openssl 3.0.0?

setup_sut.sh should not perform `sudo make install` operations

The setup_sut.sh performs sudo make install in order to deploy SUTs and SUT dependencies. This has the potential to break things. We should tweak the setup_sut.sh to install dependencies locally, and optionally, have some convenient way of adding the dependencies/SUTs to the PATH.

-responseWait in learn_openssl_server_psk causes hello-msg/handshake failure

When attempting the very first hello-msg or handshake in the Readme,
the presence of the -responseWait argument to openssl causes a complete failure:

c@intel12400 ~/dtls-fuzzer (remove-response-wait)> LD_LIBRARY_PATH=suts/openssl-1.1.1b/ java -jar target/dtls-fuzzer.jar @
args/openssl/learn_openssl_server_psk -test examples/tests/servers/psk
05:16:31 [main] INFO : Main - Processing command state-fuzzer-server
05:16:31 [main] INFO : Main - Running test runner
05:16:31 [main] INFO : ProcessHandler - Command to launch SUT: /home/c/dtls-fuzzer/suts/openssl-1.1.1b/apps/openssl s_server -psk 1234 -key /home/c/dtls-fuzzer/experiments/keystore/rsa2048_key.pem -cert /home/c/dtls-fuzzer/experiments/keystore/rsa2048_cert.pem -CAfile /home/c/dtls-fuzzer/experiments/keystore/rsa2048_cert.pem -accept 27791 -dtls1_2 -responseWait 5000 -mtu 5000
05:16:31 [main] INFO : TestRunner - Test: PSK_CLIENT_HELLO PSK_CLIENT_HELLO PSK_CLIENT_KEY_EXCHANGE CHANGE_CIPHER_SPEC FINISHED APPLICATION
1 times outputs: TIMEOUT SOCKET_CLOSED SOCKET_CLOSED SOCKET_CLOSED SOCKET_CLOSED SOCKET_CLOSED

Removing the -responseWait argument from the last line of args/openssl/learn_openssl_server_psk allows the initial handshake to proceed:

c@intel12400 ~/dtls-fuzzer (remove-response-wait)> LD_LIBRARY_PATH=suts/openssl-1.1.1b/ java -jar target/dtls-fuzzer.jar @args/openssl/learn_openssl_server_psk -test examples/tests/servers/psk
05:17:11 [main] INFO : Main - Processing command state-fuzzer-server
05:17:11 [main] INFO : Main - Running test runner
05:17:11 [main] INFO : ProcessHandler - Command to launch SUT: /home/c/dtls-fuzzer/suts/openssl-1.1.1b/apps/openssl s_server -psk 1234 -key /home/c/dtls-fuzzer/experiments/keystore/rsa2048_key.pem -cert /home/c/dtls-fuzzer/experiments/keystore/rsa2048_cert.pem -CAfile /home/c/dtls-fuzzer/experiments/keystore/rsa2048_cert.pem -accept 27831 -dtls1_2 -mtu 5000
05:17:12 [main] INFO : TestRunner - Test: PSK_CLIENT_HELLO PSK_CLIENT_HELLO PSK_CLIENT_KEY_EXCHANGE CHANGE_CIPHER_SPEC FINISHED APPLICATION
1 times outputs: HELLO_VERIFY_REQUEST SERVER_HELLO|SERVER_HELLO_DONE TIMEOUT TIMEOUT CHANGE_CIPHER_SPEC|FINISHED APPLICATION

I discovered openssl is not accepting the -responseWait flag by testing the full openssl command from the shell.

I am not sure of the root-root cause (I am sure there is a good reason you are passing -responseWait to openssl)

TLS

I saw your old project of fuzzing TLS implementations on USENIX 15. So I was wondering if dtls-fuzzer support tls fuzzing. If so, how?

Well-formatted source code

We should ensure DTLS-Fuzzer's code is formatted according to standards.

Invalid error for -equivalenceAlgorithms parameter

When i try to run the command LD_LIBRARY_PATH=suts/openssl-1.1.1b/ java -jar target/dtls-fuzzer.jar @args/openssl/learn_openssl_server_psk -test examples/tests/servers/psk i am getting error as Invalid error for -equivalenceAlgorithms parameter. Please correct my error.

Remove '>' from bash code blocks.

In the readme every shell line has a '> ' before the command.
ie:

> sudo apt-get install openjdk-8-jdk

The greater sign at the start if the line is not needed for this command to execute under bash.
But it does cause issues for other shell users.

For bash, normally, greater than indicates stdout should be redirected. reference link
But typically '>' is followed by a file name or other syntax.
But followed by a space character, it seems bash ignores this stdout redirect.

But it causes problems for users of other shells (like the fish shell):

The greater than signs do add a nice visual highlight, but might not be necessary as the README page on github seems to be pretty clear that the block is a "click-copy-paste".
See this screenshot inside the border.

If the greater sign was removed, would users be less clear about the purpose of the code block??
I suspect probably not for most.

As another point in support of removing the '> ' chars, please see this openjdk building page:
This open jdk build directions don't contain '>' chars

I suggest considering removing the '> ' character from the start of the shell commands in the README

If removing the pain for non-bash shell users is deemed useful, then I could submit a PR.