Giter Site home page Giter Site logo

purpleteam-labs / purpleteam Goto Github PK

View Code? Open in Web Editor NEW
119.0 4.0 15.0 2.31 MB

CLI component of OWASP PurpleTeam

Home Page: https://owasp.org/www-project-purpleteam

License: Other

JavaScript 90.41% Shell 0.03% Roff 9.56%
security-regression-testing application-security cloud-security build-tool ci purpleteam cli web-security devsecops devsecops-pipeline

purpleteam's People

Contributors

binarymist avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

purpleteam's Issues

Prepare purpleteam CLI for deployment to NPM using `np`

Add releases using github but in the changelog format. Using np

After selecting the version to release, np runs the full release and publish to npm.

np installed globally.

np doesn't allow one to add the latest tag to a pre-release, so next seems the only option. Do this, then:

npm dist-tag add [email protected] latest

Check the files array in package.json to make sure no secrets can be published. Double check what will be published


From project root dir: np (release and publish to npm)

After selecting the version to release, np runs:

Todo: detail what np runs.

  • logger
  • mocksse
  • purpleteam (CLI)

Once published to npm npm install and check the file contents.
Check the Github release Assets archive.


From project root dir: np --no-publish

After selecting the version to release, np runs:

✔ Git
✔ Installing dependencies using npm
✔ Running tests using npm
✔ Bumping version using npm
✔ Pushing tags
✔ Creating release draft on GitHub

  • orchestrator
  • iac-sut
  • lambda
  • app-scanner
  • tls-checker (not yet released)
  • server-scanner (not yet released)

Check the Github release Assets archive.


From project root dir: np --no-publish --no-tests

  • doc
  • s2-containers

Check the Github release Assets archive.

Time appears incorrect in outcomes files

Actual Behavior:

hour produced by NowAsFileName returns 05 when it's 17.

Issue exists in all testers that use the NowAsFileName function.

Expected Behavior:

Should return time as according to local

Steps to Reproduce:

During a Test Run this is executed.

Additional Details:

The Date Get methods return values based on the local time, problem is the timezone is set in the container image (in our case to UTC). If we change it, then we change it for everyone that uses the images.

The folloing placed into the docker-compose file fixes it for New Zealand, but then it's wrong for everyone else:

    environment:
      - TZ=Pacific/Auckland

We've decided to leave it as UTC.

Provide ability for Build User to pass flag to disable colour in tester reporting

The enhancement may already be reported! Please search for the enhancement before creating one.

Current Behavior:

colorsEnabled is set to true in app-scanner.

Proposed Behavior:

Provide ability for buildUserConfig Job file to pass through flag for testers to check whether colour should be enabled or dissabled in tester reporting

How It Would Benefit You:

I wouldn't have to see colour symbols in cli logs that are writen to file

Convert our usage of child_process spawn to fork in app.parallel.js

CLI Integrations

Current Behavior:

We have a process that runs anywhere. Do we need integrations with the likes of Github, Gitlab, others?

Proposed Behavior:

Integrations

How It Would Benefit You:

Good question?

That's what this issue is about answering.


Resources that may be helpful:

Integration test installing and running purpleteam CLI

As part of ci-cd for the CLI, we should test that purpleteam CLI installs and runs correctly as if it was done so from NPM. We can do this using verdaccio:

Current Behavior:

No integration test for the CLI

Proposed Behavior:

Add integration tests that:

  • Test the options detailed on the CLI README
    • Installs
    • Configuratioins
    • Runs

Basically we want to be testing things we don't usually test in development... like the actual ilnstallation and verifying that the CLI runs as a Build User would use it.

How It Would Benefit You:

It would provide confidence that the community can install and run purpleteam without errors

How graphql ready are we? What to do to support if not?

The enhancement may already be reported! Please search for the enhancement before creating one.

Current Behavior:

Not sure. Investigate what we need to do to support GraphQL if we don't already

Proposed Behavior:

Implement GraphQL support if it doesn't exist.

  • Should we do this?
  • Do we need to do this?

How It Would Benefit You:

Not sure, who actually wants this? Anyone?

Implement SSL/TLS Scanner

Upgrade aws-sdk from v2 to v3

Tweak Copyright notices

Actual Behavior:

All the source files that have a Copyright notice at the top refer to purpleteam as the solution.

Also the last two lines of the Copyright are:

// You should have received a copy of the GNU Affero General Public License
// along with purpleteam. If not, see <https://www.gnu.org/licenses/>.

Expected Behavior:

  • The casing should match that as defined in the definitions
  • The last two lines of the Copyright should be:
    // You should have received a copy of the GNU Affero General Public License
    // along with this PurpleTeam project. If not, see <https://www.gnu.org/licenses/>.
    

Additional Details:

This affects most/all public PurpleTeam projects.

Create API SUT and test

local returns 'app tests are running', cloud returns 'initialising job'

Actual Behavior:

In local env the very first response is 'app tests are running'
In cloud env the very first reponse is 'initialising job'

I think this may be because the cloud returns the status faster than local but unsure.

Expected Behavior:

Both local and cloud to be the same

Steps to Reproduce:

Run a test

Create Tester reset for "Tester failure:" occurrances

If the orchestrator.js testTeamAttack returns initTesterResponsesForCli due to failedTesterInitialisations.length || !startTesters, for example if a Tester failure: occurred, for example due to:

  • No tlsScanner resource object being provided in the Job file
    • Then the App Tester will be in the state of Tester initialised.
  • Or perhaps a timeout while waiting for s2 containers to be up (easy to simulate)
    • Then the Tls Tester will be in the state of Tester initialised.

This means subsequent calls to orchestrator.js testTeamAttack will return the same internals.initTesterResponsesForCli as last time, and Testers will never change state.

  • What we need is a reset routine for each Tester

The first place the reset should be invoked from would be within the first option of the current internals.initTesterResponsesForCli = failedTesterInitialisations.length || !startTesters ternary in testTeamAttack of orchestrator.js.
internals.initTesterResponsesForCli = null which is currently in orchestrator.js processTesterFeedbackMessageForCli should be in something like a resetTesters routine of the orchestrator.
The resetTesters routine would also call each model's testerFinished(true)

  • reset should probably attempt stopping any s2 containers as well as setting status.

  • Add tests, even if manual

Unclear instructions / failing start

Actual Behavior:

Wanted to try out purpleteam (guide at https://purpleteam-labs.com/doc/local/set-up/)

Cloned https://github.com/purpleteam-labs/purpleteam-orchestrator
npm run dc-build
results in

npm run dc-build                                                    

> [email protected] dc-build /Users/my-user/Develop/purpleteam-orchestrator
> npm run dc-orchestrator-testers build


> [email protected] dc-orchestrator-testers /Users/my-user/Develop/purpleteam-orchestrator
> APP_SCANNER_GROUP_ID=$(id -g) APP_SCANNER_USER_ID=$(id -u) ORCHESTRATOR_GROUP_ID=$(id -g) ORCHESTRATOR_USER_ID=$(id -u) docker-compose -f ./compose/orchestrator-testers-compose.yml "build"

WARNING: The HOST_DIR variable is not set. Defaulting to a blank string.
ERROR: build path /Users/my-user/develop/purpleteam-app-scanner either does not exist, is not accessible, or is not a valid URL.

Mh, okay. Can't see that in instructions, but okay. Cloned https://github.com/purpleteam-labs/purpleteam-app-scanner.
npm run dc-build
now results in

npm run dc-up

> [email protected] dc-up /Users/my-user/Develop/purpleteam-orchestrator
> npm run dc-orchestrator-testers up


> [email protected] dc-orchestrator-testers /Users/my-user/Develop/purpleteam-orchestrator
> APP_SCANNER_GROUP_ID=$(id -g) APP_SCANNER_USER_ID=$(id -u) ORCHESTRATOR_GROUP_ID=$(id -g) ORCHESTRATOR_USER_ID=$(id -u) docker-compose -f ./compose/orchestrator-testers-compose.yml "up"

WARNING: The HOST_DIR variable is not set. Defaulting to a blank string.
Creating network "compose_pt-net" with the default driver
Pulling redis (redis:alpine)...
alpine: Pulling from library/redis
540db60ca938: Pull complete
29712d301e8c: Pull complete
8173c12df40f: Pull complete
a77b7ddf4978: Pull complete
3f34a000c6b3: Pull complete
275dfaedaf41: Pull complete
Digest: sha256:f8f0e809a4281714c33edf86f6da6cc2d4058c8549e44d8c83303c28b3123072
Status: Downloaded newer image for redis:alpine
Creating compose_redis_1 ... done
Creating pt-app-scanner-cont  ... error
Creating pt-orchestrator-cont ... 

Creating pt-orchestrator-cont ... error

ERROR: for pt-orchestrator-cont  Cannot create container for service orchestrator: invalid mount config for type "bind": field Source must not be empty

ERROR: for app-scanner  Cannot create container for service app-scanner: invalid mount config for type "bind": field Source must not be empty

ERROR: for orchestrator  Cannot create container for service orchestrator: invalid mount config for type "bind": field Source must not be empty
ERROR: Encountered errors while bringing up the project.

...

Expected Behavior:

Either something starts without error or guide has clear instructions.

...

Steps to Reproduce:

Seen above.

...

Environment:

  • Specific purpleteam project and version?
    0.1.1 (i guess)
  • Running locally or in the cloud?
    local
  • Details of your specific System Under Test (SUT)?
    didn't come there
  • Which purpleteam components are running on which Operating System and version?
    0.1.1
    MacOS 11.4 + Docker 20.10.2
    Ubuntu Server 20.04 + Docker 20.10.6
  • Anything else that will set the stage for us understanding your context?
    node v14.17.0

Additional Details:

Checklist

  • I have read the documentation. (I hope I did not oversee/misunderstand something)

.env file rework

Rework how we handle environment variables in most of our services and CLI, use a .env and dotenv/config for local and provide a .env.example for source control. We use .env files in a few places, but will take this a bit further,

https://medium.com/the-node-js-collection/making-your-node-js-work-everywhere-with-environment-variables-2da8cdf6e786

  • setup .env files and dotenv/config in as many repos as makes sense. We didn't use dotnev anywhere
  • Add .env.example files
  • What happens if dotenv cant find a .env? Then it breaks
  • Document the parse error (due to incompatible TERM env var) in pt cli and detail what to do commit

Doc changes here (https://github.com/purpleteam-labs/purpleteam) and probably quite a few other places.

dotenv doesn't overload by default environment variables that are already set and by the look of it, can't do it using the pre-load technique (node -r dotenv/config server.js)

Add option to disable CLI screen

Actual Behavior:

Running (spawning, etc) purpleteam within another process, for example the NPM install locally option causes blessed errors due to the fact that the purpleteam CLI is not running directly within a terminal.

npm start for example.

Example output follows:

image

Expected Behavior:

No errors

Steps to Reproduce:

Run the CLI from within another process. It must get to the stage where the blessed screen is actually drawn though. So the orchestrator has to be running.

Because the CLI errors when creating the screen, the cli log also fails to be written.

Additional Details:

  1. I think we need to change the transport that ptLogger is inited with to probably Console
  2. Work out a way to swap the dashboard (blessed)
  3. Release changes

Before starting work

  • link purpleteam dep purpleteam-logger
  • link purpleteam-build-test-cli dep purpleteam
  • Using npm list -g --depth 0 before and after linking to see the differences

Before commit/push/PR:

  • Review purpleteam README
  • Spell check README
  • Review source code changes
  • Fix tests
  • Test all of the steps in the README

After commit/push/PR:

  • Publish release
  • Check release on github and npm (README, etc)
  • Tell the Slacks and twitter that new release is out
  • unlink

Re-work orchestrator.js

  • Re-work orchestrator.js
  • Re-work orchestrator.js variables like testerModels should be in internals or even better a member of the class

While doing this:

  • We may as well try and see what happens if we turn the CLI POST retry on again (default 2 retries), will this effect the cloud? Adds complexity for little gain. Currently there isn't really a need, and the time-outs would have to be tweaked quite a bit. Added comments to apiDecoratingAdapter.js
  • Currently the CLI still subscribes to updates even if Testers were not started, this shouldn't happen
    a3dc4fe and daaec32 addresses this

Tickets to keep an eye on

Handle fatal failures in app.parallel better

  • We need to look at turning fatal failures into non fatal via retries and other means. Just search for throws and Todo's in app.parallel.js
    • If all the s2 Service Discovery Service Instances don't come up within the timeOut (70000 at time of writing) app.parallel getS2ContainerHostNamesWithPorts rejects with new Error 'Timed out while waiting for S2 Service Discovery Service Instances to be available.'. This is currently fatal (app-scanner container hangs), Can we reduce the timeOut to well under the Api Gateway timeout and use our redis publisher to publish a message. Then retry getS2ContainerHostNamesWithPorts up to two more times? This would satisfy the Api Gateway 30 second timeout, provide some feedback to the purpleteam CLI, and almost certainly be successful. The first try on a cold EC2 instance takes longer. Also the s2 Service Discovery Service Instances that are up will remain up after retrys, making subsequent retrys faster. Also if an exception is thrown, which in this case it is, the s2 Service Discovery Service Instances will currently never be brought down. This is also a problem that needs addressing. If the AMI resources is not sufficient for the number of containers requested, then the timeOut will consistenly be hit, there is a comment in getS2ContainerHostNamesWithPorts of app.parallel around this.
      Need to also work out what happens if the Lambda timeouts are hit. Test this by reducing the timeout. Then handle this in the lambdas and in the app-scanner, publishing feedback to the purpleteam CLI
  • Problems when we kill the CLI before tests are finished due to pollTesterMessages (that hasn't been called on last message) not cleaning up the testerWatcher
  • Starting a test too soon after a previous one, we'll need better error handling and notification to the build user

Improve orchestrator Tester model error handling

Orchestrator Tester models startTester, initTester and plan:

  • Need to retry if requests to Testers fail. Swap Wreck for got as Wreck doesn't support retry
  • Provide error handling for failed requests.
    • plan already has some, but it'll probably need changing
    • initTester should be easy to handle as we can just pass back a "Tester failure:"
    • startTester may be a little more work,
      • startTester needs to set isFinished to true and return a "Tester failure:" message, which would also need to be augmented into the combinedInitTesterResponses that the orchestrator returns to the CLI
      • The following could be done, but may not need to be:
      • #warmUpTestSessionMessageChannels should probably also check that it's only acting on Tester models where testerFinished() is true
      • There is also no point in the CLI asking for Tester updates from the orchestrator

May as well swap axios and http-proxy-agent for got and hpagent in app.emissary.js of the app-scanner.

Add tests, even if manual

Rename slave/slaves

slave -> emissary
slaves -> emissaries
Slave -> Emissary
Slaves -> Emissaries

proj commited apply -> search again merged pushed change-set
doc - commit
CLI - commit
app-scanner - commit
lambda - commit
s2-containers - commit
orchestrator - commit
iac-sut - - - - -
logger - - - - -
tls-checker - - - - -
server-scanner - - - - -
  • System tested
  • Check badges

Get API scanning ready

Implement Server scanner

We're going to be using Nikto. The majority of the code will be in https://github.com/purpleteam-labs/purpleteam-server-scanner.

This is mostly green fields work. We have the app-scanner and tls-scanner fully implemented to use as a reference for what Testers look like.

The diagram here shows where Testers fit into the architecture.

As with the TLS Tester, this will require on-going maintenance, is the work worth what the Server Tester provides? If not, decide what to do.

Only trust secure TLS certificates in selenium webdriver

This covers Zap self-signed certs and self-signed certs that SUTs may present.

Currently all self-signed certs presented to the webdriver are accepted. Ideally where we want to get to is to not trust any insecure certificates, that is passing false in the following calls in webDriverFactory.js in the app-scanner:

chromeOptions.setAcceptInsecureCerts(true)
firefoxOptions.setAcceptInsecureCerts(true)

In order to do this the following two items need to be done.

Zap

Each Zap instance uses it's own root certificate.

Currently if we have:

chromeOptions.setAcceptInsecureCerts(false)
firefoxOptions.setAcceptInsecureCerts(false)

With the Zap self-signed cert, this will block the selenium browser currently. So what we need to do is:

  1. Get the dynamicaly created Root CA from each Zap instance. The Zap API method coreOtherRootcert should do the trick
  2. create our own browser profile (chrome and firefox) with the zaproxy cert added
  3. Tell selenium to launch the browser based on that profile

These steps should be performed within the routines addressed by the following calls in the Cucumber world:

async initialiseBrowser() {
  await this.sut.initialiseBrowser(this.zap.getPropertiesForBrowser(), this.selenium);
}

SUT

Step 2 and 3 of the above will be similar, but we will need to work out where the best place to source the self-signed customer SUT certificate from. A couple of thoughts:

  • Passed in the Build User config (Job). This seems a bit smelly
  • Provided as an environment variable. This would work for both local and cloud

Discussions

I have enabled the Discussions feature. It would be a good place to ask purpleteam-related questions and discuss it in general.

Please be kind and constructive.

This issue can be used for discussing adding new categories and other meta things.

zaproxy needs it's deps (request and friends) updated

This is another you could dive into @ricekot. If and when you do, let me know and I'll dig out as much info as I have. I've done this once before. My fork is here: https://github.com/binarymist/zap-api-nodejs

purpleteam-app-scanner interfaces with Zap via the zap-api-nodejs. The app-scanner is currently pointing at a commit (https://github.com/purpleteam-labs/purpleteam-app-scanner/blob/main/package.json#L88)

Basically we need to update the HTTP library (move away from request as it's now depricated): https://github.com/binarymist/zap-api-nodejs/blob/master/package.json#L28-L29

I'd probably use got again. I moved from request to got in the CLI. Most of the changes were in the apiDecoratingAdapter.js: a705ae8

The zap-api-nodejs is all generated JavaScript, so you need to play a little with Java, not a big deal though. When you get there give me a yell. This not only affects purpleteam-labs but also Zap obviously and all of it's consumers via the Node API @psiinon would also be happy 😄

Update: (2021-02-25) See issue in zaproxy

Creat and deploy website for public documentation

Current Behavior:

Currently public doc:

Proposed Behavior:

public docs need to have their own website

How It Would Benefit You:

Users wil be able to find information about purpleteam, it'll be user friendly

Additional details

@binarymist has been doing research on this for a couple of years and has a bunch of ideas, requirements and a shortlist of possible platforms to use. Ping @binarymist when starting this. Some of the notes taken have a degree of sensitivity.

As a stop-gap measure we've hosted https://github.com/purpleteam-labs/purpleteam-doc as doc.purpleteam-labs.com
Details here: https://webmasters.stackexchange.com/questions/105899/cloudflare-dns-how-to-setup-a-github-repository-to-a-custom-sub-domain

Update Dependencies

SUT

System

Project

  • purpleteam-logger
    • test via purpleteam-orchestraiter
    • test via purpleteam-app-scanner
    • publish to NPM and update orchestrator and app-scanner deps
    • test locally via cloud SUT
  • mocksse
  • purpleteam
  • purpleteam-lambda
    • test local
    • test local with cloud sut
    • test cloud
  • purpleteam-orchestrator
    • Dockerfile node version
    • gitlab-ci node version
    • devDependencies
      • code, lab
      • eslint and friends
      • npm-check
      • pre-commit (no update)
      • rewire, sinon
    • dependencies
      • hapi-json-api
      • ajv and frineds
      • app-module-path (no update)
      • convict
      • diff
      • hapi 18->20
      • hapi-good-winston
      • good (is depricatedd, hapi needs to be upgraded at same time)
      • inert (hapi needs to be upgraded at same time)
      • joi
      • purpleteam-logger
      • redis
      • susie (no update)
      • wreck
    • test local
    • test local with cloud sut
    • test cloud
  • purpleteam-app-scanner
    • Dockerfile node version
    • gitlab-ci node version
    • devDependencies
      • lab
      • eslint and friends
      • npm-check
      • pre-commit (no update)
      • rewire
      • sinon
    • dependencies
      • code
      • app-module-path (no update)
      • aws-sdk
      • axios
      • convict
      • cucumber
      • good (is depricatedd, hapi needs to be upgraded at same time)
      • hapi 18->20
      • hapi-good-winston
      • http-proxy-agent (no update)
      • joi
      • purpleteam-logger
      • redis
      • selenium-webdriver (no update)
      • zaproxy (on latest commit)
        • zaproxy needs it's deps (request and friends) updated. Issue added.
    • test local
    • test local with cloud sut
    • test cloud
  • purpleteam-s2-containers

@hapi/good (API here) is deprecated. It doesn't actually do anything other than log process metrics (memory, uptime, load). All other logging is done by the generic hapi logger (server.log, request.log) or purpleteam-logger. good can stay until it no longer works, then we can just remove it along with hapi-good-winston.

  • Replace instances of JSON.parse with Bourne.parse

Message on #hapi Slack from the BDFL on 2020-07-05:
eran 08:05
@channel hapi v18 will is being soft-deprecated. This means it will only get critical security fixed between now and the end of the year. No other bug fixes or node version updates will be released. It was supposed to be deprecated months ago but was postponed due to the COVID-19 situation. You will start seeing a deprecation message on npm now, and then it will no longer receive any support or security fixed at the end of the year. There is no reason not to upgrade to hapi v19. It is a trivial upgrade from v18
hapijs/hapi#4111

NPM tricks

Publish

Publish and create release notes

What other Authentication techniques does purpleteam need to support?

Current Behavior:

We have username and password SUT login.

Proposed Behavior:

How It Would Benefit You:

Build Users would be able to provide authentication details to purpleteam other than username and password so that purpleteam can authenticate with the SUT

Notes:

  • Zaproxy had a google doc somewhere that listed all it's authentication techniques. Finding where this is documented would be a good place to start
  • Possibly put a survey out to find out what Devs and their Teams need

Resources

Scratch

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.