pid1/test-sec-flags

Automated testing of various security-oriented compilation flags for Arch Linux packages.

Hello!

• My personal website is available here.
• For consulting services, please see Cygnus Strategy.

test-sec-flags's Issues

Only first five tests compile

I ran the following:
git clone https://github.com/pid1/test-sec-flags.git --recursive
cd test-sec-flags
make setup
./test-xz
(which seemed to run fine, but I interrupted it so I could just run everything with:)
make

The last command compiled and ran the first 5 tests of unixbench, but then terminated with:

make[2]:``Leaving directory '/home/mostlyh/test-sec-flags/unixbench/UnixBench' make[1]: Leaving directory '/home/mostlyh/test-sec-flags/unixbench/UnixBench' Compilation finished, running test 6 make: *** [Makefile:10: test-unixbench] Error 1

running on Arch, updated a few days ago, i7-4930K

Consider using Phoronix Test Suite

It might be a good idea to use PTS to measure the overall impact of changes after rebuilding the whole system.

Automated Reporting

We should make it as easy as possible for people to contribute their results. Perhaps we could have the script concat the result files and automatically paste them to a pb instance?

test_ffmpeg seems to stuck on compile (?)

The test_ffmpeg seems to stuck on compilation. CPU shows 100% and processes (8) are RUNNING.

License: GPL version 3 or later  
Creating config.mak, config.h, and doc/config.texi...  
config.h is unchanged  
config.asm is unchanged  
libavutil/avconfig.h is unchanged  
CC  libavdevice/alsa.o  
CC  libavdevice/alldevices.o  
CC  libavdevice/alsa_dec.o  
CC  libavdevice/alsa_enc.o  
CC  libavdevice/avdevice.o  
CC  libavdevice/dv1394.o  
CC  libavdevice/fbdev_common.o  
CC  libavdevice/fbdev_dec.o

See top screenshot:

measure the overhead of UBSan bounds checking

It would be neat to measure the overhead of -fsanitize=bounds,object-size -fsanitize-undefined-trap-on-error. It's going to be way too expensive to enable on Arch, but it's useful information nonetheless. There are some other hardening-friendly UBSan subsets that might also be worth measuring.

xz compilation flags are seemingly not applied

Probably due to a misunderstanding of the makefiles, checksec is only reporting partial relro for test 5 (and presumably the other tests) after compilation.

ffmpeg tests are broken

The flags for ffmpeg are being set at all.

Flag Combinations to Test

We should decide which flag combinations we want to include in our testing. Ideas already raised have included -fPIE by itself, -fstack-protector-strong by itself, and these in combination. Please add these to FLAGS.rst, and use this issue for discussion.