neuralegion/bright-cli

Command Line Interface (CLI) tool for NeuraLegion's solutions.

bright-cli's Issues

Rename agent to repeater

Add support on repeater part to Web Sockets

The engine supports scanning Web Sockets, but the repeater was built without this in mind, hence:

Error executing request: "GET wss://example.com/"
Cause: Invalid protocol: wss:

We need to be sure to add support for Web Sockets to the repeater by implementing a new RequestExecutor. You can use ws or any other libraries. Maybe, it requires some changes from the engine side.

"yargonaut" breaks execution of "y18n"

/home/projects/nexploit-cli/node_modules/yargs/build/lib/yargs.js:1132
                throw err;
                ^

TypeError: Cannot read property 'indexOf' of undefined
    at Y18N.__n (/home/projects/nexploit-cli/node_modules/y18n/index.js:158:12)
    at Object.nonOptionCount (/home/projects/nexploit-cli/node_modules/yargs/build/lib/validation.js:31:32)
    at Object.runValidation [as _runValidation] (/home/projects/nexploit-cli/node_modules/yargs/build/lib/yargs.js:1154:20)
    at Object.parseArgs [as _parseArgs] (/home/projects/nexploit-cli/node_modules/yargs/build/lib/yargs.js:1124:26)
    at Object.get [as argv] (/home/projects/nexploit-cli/node_modules/yargs/build/lib/yargs.js:986:25)
    at Object.<anonymous> (/home/projects/nexploit-cli/dist/index.js:2:12990)
    at __webpack_require__ (/home/projects/nexploit-cli/dist/index.js:2:176)
    at /home/projects/nexploit-cli/dist/index.js:2:1255
    at Object.<anonymous> (/home/projects/nexploit-cli/dist/index.js:2:1301)
    at Module._compile (internal/modules/cjs/loader.js:1236:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1257:10)
    at Module.load (internal/modules/cjs/loader.js:1085:32)
    at Function.Module._load (internal/modules/cjs/loader.js:950:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47

Blocks queue

target	direct	repeater
www.neuralegion.com	5.4 req/s	0.3 req/s
owasp benchamrk	104.6 req/s	1.0 req/s

We need to figure out how to manage a faster event bus to cater to more requests.
Old crystal based repeater managed ~60Rps when I tested.

Add new tests

Add support of business_constraint_bypass, broken_saml_auth, proto_pollution, nosql and id_enumeration tests

Add ability to set a timeout for polling

As discussed with @rielas, @ArtLinkov and @bararchy in Slack

Readme contains invalid examples

Rework agent command to accept env variables

Add support of agents

As discussed with @rielas in Slack

es2016 doesn't support Object.fromEntries

Starting/stopping the service

Notify the user on disconnected repeater

Additional, replace all mentions "agent" to "repeater".

Add step-by-step verification to CLI setup

Implement “Wizard” as step-by-step guide to make it fool-proof for users to install our CLI and validate the connectivity of the Repeater functionality.

Implement on-prem agent to simplify integration

The repeater has been deprecated. The development has moved to this repository.

Points of implementation:

• Make sure to use AMQ over HTTPS.
• Allow for --proxy host:port settings, allowing the user to configure a proxy for the AMQoverHTTP traffic.
• When starting do a simple sanity check (amq.nexploit.app reachable, target reachable, api key and ID manage to authorize, and alert about any issues.)
• Allow one agent to cater for multiple scans, decreasing the need for multiple agents.

Prevent Following Redirect

Fix spec type casting

Startup Scripts Generation

We should generate and configure a Startup Script to keep repeater process alive at every system restart.

As discussed with @ArtLinkov , we should introduce --daemon option to configure the startup script on the fly. To allow the user to re-configure the repeater, we should stop the existing process, change the script, and restart a process on nexploit-cli repeater command.

For starters, we can support the following service managers:

• rcd (bsd)
• systemd (linux)
• launched (mac)
• sc (windows)

Create a single executable out

Build an executable file for the following platforms:

• mac
• linux
• windows

NPE: Cannot read property 'number' of undefined

$ nexploit-cli scan:polling --interval 30s --timeout 10m --token *** --breakpoint high_issue ***

Starting polling...
Error during "scan:polling": Cannot read property 'number' of undefined
Error: Process completed with exit code 1.

Add support of "open_buckets" test

Add support for new tests

Comprehensive scan tests:

• improper_asset_management
• server_side_js_injection
• exposed_couch_db_apis
• email_injection
• http_response_splitting
• insecure_tls_configuration

Business tests:

• mass_assignment

"scan:stop" command finishes with 404 error

$ nexploit-cli scan:stop --api-key=<TOKEN> <SCAN_ID>
Error during "scan:run" run: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot POST /api/v1/scans/{scan_id}/stop</pre>
</body>
</html>

Reported by @rielas in Slack

Auto-build container

@NeuraLegion/devops we now use this repo as the basis for the "on-prem agent" solution.
Let's publish it automaticlly to dockerhub neuralegion/repeater instead of the old one.

We can fix docs on the way.

"configure" without GUI

Currently, when we run the configure option, the only way to execute the tests is via a web interface at localhost(port-forwarding), but when there is no such option (i.e server access via terminal only) we cannot perform this critical diagnostic.

The required flow of testing should be similar to the current configure command, just in text form.

$ nexploit-cli configure --nogui
Welcome to the NexPloit Network Testing wizard!

Note: To run the test, you will require a 'Repeater ID' and an 'Repeater Token' with the correct scopes. 
If you are running the configuration as part of a POC, both of these should have been sent to you via your sales contact.

Please enter your Repeater ID: <ID>
Please enter your Repeater API Token: <TOKEN>

--

Starting EXTERNAL communication diagnostics:

Validating that the connection to amq.nexploit.app at port 5672 is open… [Success|Failed]
Validating that the connection to nexploit.app at port 443 is open… [Success|Failed]
Verifying provided Token and Repeater ID… [Success|Failed]

EXTERNAL communication diagnostics completed.

--

Next step is to validate the connection to your INTERNAL (local) target application(s).

Please enter the target URLs to test (separated by commas): <URL1>, <URL2>, <URL3>

--

Starting INTERNAL communication diagnostics:

Trying to reach <URL1>... [Success|Failed]
Trying to reach <URL2>... [Success|Failed]
Trying to reach <URL3>... [Success|Failed]

EXTERNAL communication diagnostics completed.
1 out of 3 URLs could not be reached.

--

Communication diagnostics done.

Possible errors:

Test	Text	Error Message
TCP test	Validating that the connection to amq.nexploit.app at port 5672 is open	Connection to amq.nexploit.app:5672 is blocked, please verify that the machine on which the Repeater is installed can reach the remote server. Possible reasons for communication failure: Outbound communication to the host is blocked by a Firewall or network settings
HTTPS client test	Validating that the connection to nexploit.app at port 443 is open	Connection to nexploit.app:443 is blocked, please verify that the machine on which the Repeater is installed can reach the remote server. Possible reasons for communication failure: Outbound communication to the host is blocked by a Firewall or network settings
Auth details validation (API call)	Verifying provided Token and Repeater ID	Invalid Token or Repeater ID, please make sure you are using the correct details provided to you. If you need further assistance, please reach out to your NeuraLegion technical support contact.
Access to local target	Trying to reach <url>	Error: Connection to {{url}} is blocked, please verify that the machine on which the Repeater is installed can reach the target server. Possible reasons for communication failure: Outbound communication to the host is blocked by a Firewall or network settings

Invalid exec args

There is a bug in nexploit-cli msi 7.5.2, on windows, when getting to the run scan it has a validation error, and can't scan, saying the target can't be reached

Remove `type` and `protocol` options from Run command

Bypass header validation

Error executing request: "GET https://example.com/? HTTP/1.1"
Cause: Invalid character in header content ["Referer"]
Error executing request: "GET https://example.com/? HTTP/1.1"
Cause: Invalid character in header content ["Referer"]
Error executing request: "GET https://example.com/#/ HTTP/1.1"
Cause: Invalid character in header content ["Referer"]

https://tools.ietf.org/html/rfc7230#section-3.2.6
Node.js won’t accept any other characters which violate spec above. It’s the core functionality of http module.

Ensure the message is delivered before closing CLI

Support both Unicast and Multicast

Migrate from tslint to eslint

Better error handling on disconnect

Improve UI/UX for the daemon functionality

Rename --remove to --remove-daemon, ensure backward compatibility

Also, add some sort of validation message:
--daemon - A Repeater daemon process was initiated successfully (SERVICE: ${SERVICE_NAME})
--remove-daemon - The Repeater daemon process (SERVICE: ${SERVICE_NAME}) was stopped and deleted successfully

Add option to feed findings to a local Syslog/Siem systems

• Add tag to CLI which enables feeding "found issues" directly to TCP/UDP syslog server
• Also add TLS option for TCP
• The issue data should be converted into 1 of 2 formats: CEF (https://www.npmjs.com/package/cef) and standard syslog When running the command nexploit-cli logfeed you will have more options like:
  • --syslog-host: address to syslog server
  • --port: port of syslog host, default to syslog default
  • --protocol: UDP or TCP/TLS, should default to UDP
  • --format: cef or format-string (i.e {DATE} {SEVERITY} {DETIALS} {LINK}), default to cef
  • --severity-map: cef has severity levels from 0 to 10, we should be able to allow mapping our severity against specified cef numbers, defaults are low=6 medium=8 high=10
• all the relevant auth parameters
• The CLI needs to actively pull new findings from nexploit.app via api
• The back-end should tag which issues have been pulled by the feeder already and the CLI should skip them when polling

Implement command to output the issues

+-------+----------+--------------------------------+
| Issue | Severity | Link                           |
+-------+----------+--------------------------------+
| XSS   | High     | https://nexploit.app/dsfsdfsfd |
+-------+----------+--------------------------------+
| SQLi  | High     | https://nexploit.app/dsfsdfsfd |
+-------+----------+--------------------------------+
| Cors  | Low      | https://nexploit.app/dsfsdfsfd |
+-------+----------+--------------------------------+

Allow to run a scan via agent

Example usage: nexploit-cli scan --repeater --target https://my-site.com/

DNS query takes a too long time

Increase UV_THREADPOOL_SIZE

MSI Windows installation package

Relates to #75

Add support of "open_database" and "html_injection" tests

Wrong executable path

pkg sets the process.execPath in run-time to the full path of the executable file. We should skip somehow setting programPath

Request path contains unescaped characters

To reproduce you can send URL which ends with \u200E.

Add the ability to select tests to run

Related to https://github.com/NeuraLegion/nexploit-service/issues/377

Malformed URI sequence

NAME

CLI bug in executing request

DESCRIPTION

While using the Repeater, the scan gets stuck and the CLI reports and error (attached to thread)

Error executing request: "GET https://dev3.netwrix.com/operators/get-chat-status?lang=ftp://...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\%SystemDrive%\\boot.ini&lang= HTTP/1.1"
Cause: Malformed URI sequence: https://dev3.netwrix.com/operators/get-chat-status?lang=ftp://...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\%SystemDrive%\\boot.ini&lang=
events.js:174
      throw er; // Unhandled 'error' event
      ^
Error: read ECONNRESET
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:111:27)
Emitted 'error' event at:
    at Connection.emit (events.js:198:13)
    at Connection.C.onSocketError (C:\Program Files\NexPloitCLI\node_modules\amqplib\lib\connection.js:353:10)
    at TLSSocket.emit (events.js:203:15)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)

AFFECTED USERS

Repeater users

REQUESTED BY

Dev first beta user

Reported by @ArtLinkov in Slack

Add support for scripts

Intro

In some cases the request data is being manipulated in some specific way before it is sent to the server as part of a "signature" effort, this means that unless we allow the user to specify how exactly this signature works our requests to the target will not be accepted at all.

As an example, a customer might have a JSON in the body that has several integer parameters that before the sending of the request, their values will be collected and summed (or any other mathematical shenanigans), and the result will be added to a separate "signature" parameter.

We need to provide a way for the user to specify such custom pre-processing steps and apply them to our requests dynamically.

Real world Examples:

• https://www.wolfe.id.au/2012/10/20/what-is-hmac-authentication-and-why-is-it-useful/

Usage Definition

Loading scripts to Repeater

• A repeater should be able to receive a list of scripts and load them into memory for execution, either:

  • From the BE - will be updated at initialization or during health check

  • From local files, the user has to specify JSON string that contains script list, which is initially empty and consists of zero or more host and path pairs. Example: {"*.example.com": "./hmac.js"}'

    nexploit-cli repeater --scripts '{"*.example.com": "./hmac.js"}'

    To load a global script, the user should perform the command in such a way:

    nexploit-cli repeater --scripts '{"*": "./hmac.js"}'

    • local scripts should be shown in the UI as “read only” with just name data

Custom script requirements

• Every custom script must include a function with the hardcoded name: handle

  • Which accepts the following arguments from each request automatically:

  interface RequestOptions {
    method: string;
    url: string;
    headers: Record<string, string | string[]>;
    body: string;
  }

  • And must return RequestOptions with which to build the request (in the expected order)

  export const handle = (options) => options;

  Example:

  const { createHmac } = require('crypto');
  
  const handle = ({ method, url, headers, body }) => {
    const version = 'v1';
    const secret = 'someSecret';
    const timestamp = Date.now();
    const signature = createHmac('sha256', secret)
      .update(`${version}:${timestamp}:${body}`)
      .digest('base64');
  
    return {
      url,
      method,
      body,
      headers: {
        ...headers,
        'x-request-timestamp': timestamp,
        'x-request-signature': signature
      }
    };
  };
  
  exports.handle = handle;

Fix loading CommonJs modules

Describe docs for v3

Add a url for the chrome extension

https://github.com/NeuraLegion/nexploit-cli/blob/%2315_docs/describe_docs_for_v3/README.md#arguments

Originally posted by @ArtemDerevnjuk in #16 (comment)

Prevent re-connecting on fatal errors

now "core" and "exploratory" aren't accepted as values of module

Add missing test types

Invalid links in the README.md

for example cli commands