joyliu-q/sastall

Static application security testing toolkit

CIS188 Final Project SAST Tools

sastall's Issues

Caching is not working

I need to actually figure out how caching works

Make individual example runs not override each other

This is actually really simple to do. Just replace the file name from sastall.sarif to a specific one to the project (e.g. flaskproject-sastall.sarif).

I just haven't done it yet because of finals season.

Progress Tracking

Opening this issue to use it to track the progress of SASTAll.

The vision

We want a tool that is able to simplify the process of running 3 different SAST tools.

There are 2 possible solutions that would follow SASTAll's vision.

1. Publish a Github actions that is able to run 3 different SAST tools at once
2. TODO: Before, I was thinking creating a template with Helm, but that would involve K8s. I was ALSO thinking about making a docker image that just runs these tools as CLIs manually, but some of them already have pre-existing Github actions and that just seemed like doing something for the sake of doing something. If we want to do something cool with aggregation, we could host the aggregated result in the form of a SARIF file at another endpoint different from Github Securities tab, which would allow me to attempt to do something outside of CI/CD through this project.

Aggregation & Whether it's needed

Either way, the results have to be aggregated by some means. Or do they?

Before starting on this project, I thought that aggregation was one of the biggest wins/advantages of SASTAll. Running multiple tools does not change the rate of false positives (~50%), but if we can take all of the results in SARIF format and do something interesting with it, running multiple tools could actually be worth it.

However, after looking into how Github actually works, I was greatly disillusioned. Apparently for these tools, they already have CIs built into place and everything just shows up on the Github security tab. It looks great, too: there are even little tags you can filter different issues by. So, is there even a purpose to take all of the SARIF files generated, parse it, combine results, and display it somewhere?

The only advantage to doing aggregation is because currently Github just takes the issues found and throws it onto the security tab. There may be some redundancy. However, redundancy might be good because you see "Wow, all 3 of these tools did not like this particular line. Maybe I should look into this!"

But that's okay! Because here's a potential idea: do something with the Code Scanning API

TODO

• Make an MVP of a publishable github actions that—at the very minimum—runs these SAST tools.
• Store the sarif files somewhere
• ???

Actions.yml supporting multiple jobs or concurrent steps

Late into this project, I realized custom GH actions through actions.yml doesn't seem to support having multiple jobs inside it. The closest thing I found is composite action, which I will try to do.

Running the different tools concurrently is integral to the project. Even with concurrency, CodeQL is already bottlenecking, and running them one-by-one will reduce the benefits of this project to only convenience.

If the only benefit is just convenience, I would not even use this myself because DIY would be more customizable.

Anyways gotta try to find a workaround