snyk/broker

A broker system between a public service and a private service

Getting started with the Snyk CLI

Introduction to Snyk and the Snyk CLI

Snyk is a developer-first, cloud-native security tool to scan and monitor your software development projects for security vulnerabilities. Snyk scans multiple content types for security issues:

• Snyk Open Source: Find and automatically fix open-source vulnerabilities
• Snyk Code: Find and fix vulnerabilities in your application code in real time
• Snyk Container: Find and fix vulnerabilities in container images and Kubernetes applications
• Snyk Infrastructure as Code: Find and fix insecure configurations in Terraform and Kubernetes code

Learn more about what Snyk can do and sign up for a free account.

The Snyk CLI brings the functionality of Snyk into your development workflow. You can run the CLI locally from the command line or in an IDE. You can also run the CLI in your CI/CD pipeline. The following shows an example of Snyk CLI test command output.

Snyk CLI test command output

Snyk CLI scanning supports many languages and tools. For detailed information, see the following:

• Supported languages and frameworks for Open Source and Code
• Supported operating system distributions for Container
• Supported IaC Lanuages and cloud providers

This page explains how to install, authenticate, and start scanning using the CLI. Snyk also has an onboarding wizard to guide you through these steps. For a demonstration, view Starting with Snyk: an overview of the CLI onboarding flow.

Install the Snyk CLI and authenticate your machine

To use the CLI, you must install it and authenticate your machine. See Install or update the Snyk CLI and Authenticate the CLI with your account. You can refer to the release notes for a summary of changes in each release. Before scanning your code, review the Code execution warning for Snyk CLI.

Note: Before you can use the CLI for Open Source scanning, you must install your package manager. The needed third-party tools, such as Gradle or Maven, must be in the PATH.

You can use the CLI in your IDE or CI/CD environment. For details, see Install as part of a Snyk integration.

Test your installation

After authenticating, you can test your installation. For a quick test, run snyk --help.

Alternatively, you can perform a quick test on a public npm package, for example snyk test ionic.

Look at the test command report in your terminal. The report shows the vulnerabilities Snyk found in the package. For each issue found, Snyk reports the severity of the issue, provides a link to a detailed description, reports the path through which the vulnerable module got into your system, and provides guidance on how to fix the problem.

Scan your development Project

Note: Before using the Snyk CLI to test your Open Source Project for vulnerabilities, with limited exceptions, you must build your Project. For details, see Open Source Projects that must be built before testing.

In addition, depending on the language of your open-source Project, you may need to set up your language environment before using the Snyk CLI. For details, refer to Supported languages, frameworks, and feature availability overview.

After you have installed the CLI and authenticated your machine, to scan an open-source Project, use cd /my/project/ to change the current directory toafolder containing a supported package manifest file, such as package.json, pom.xml, or composer.lock. Then run snyk test. All vulnerabilities identified are listed, including their path and fix guidance.

To scan your source code run snyk code test.

You can scan a Docker image by its tag running, for example: snyk container test ubuntu:18.04.

To scan a Kubernetes (K8s) file run the following:
snyk iac test /path/to/kubernetes_file.yaml

For details about using the Snyk CLI to scan each content type, see the following:

• Snyk CLI for Snyk Open Source and the CLI help for the test and monitor commands
• Snyk CLI for Snyk Code and the Snyk Code CLI help
• Snyk CLI for Snyk Container, including Docker scanning, and the Snyk Container CLI help
• Snyk CLI for Snyk IaC, including Terraform and Kubernetes (K8s) Projects, and the Snyk IAC CLI help

Monitor your Open Source or Container Project

Snyk can monitor your Open Source or Container integrated SCM Project periodically and alert you to new vulnerabilities. To set up your Project to be monitored, run snyk monitor or snyk container monitor.

This creates a snapshot of your current dependencies so Snyk can regularly scan your code. Snyk can then alert you about newly disclosed vulnerabilities as they are introduced or when a previously unavailable patch or upgrade path is created. The following code shows an example of the output of the snyk monitor command.

> snyk monitor
Monitoring /project (project-name)...

Explore this snapshot at
https://app.snyk.io/org/my-org/project/29361c2c-9005-4692
-8df4-88f1c040fa7c/history/e1c994b3-de5d-482b-9281-eab4236c851e

Notifications about newly disclosed issues related to these
dependencies will be emailed to you.

You can log in to your Snyk account and navigate to the Projects page to find the latest snapshot and scan results:

Snyk monitor snapshot and scan results

For more information, see Monitor your Projects at regular intervals.

Running out of tests

Snyk allows unlimited tests for public repositories. If you are on the Free plan, you have a limited number of tests per month. Paid plans have unlimited tests on private and public repositories. If you are on the Free plan and notice that your test count is quickly being used, even with public repositories, you can remedy this by telling Snyk the public url of the repository that is being scanned by the Snyk CLI. This ensures that Snyk does not count a public repository towards the test limits.

If you run out of tests on an open-source Project, follow these steps:

• Run snyk monitor.
• Open the Snyk UI and navigate to the settings of the Project.
• Enter the URL of your open-source repository in Git remote URL.

Additional information about the Snyk CLI

Run snyk help or see the CLI commands and options summary.

See the course Introduction to the Snyk CLI for a quick video training session.

Snyk also provides a cheat sheet (blog post) and a video tutorial.

In particular, see the information about the following options that you may find useful:

• --severity-threshold=low|medium|high|critical: Report only vulnerabilities of the specified level or higher.
• --json: Print results in JSON format.
• --all-projects: Auto-detect all Projects in the working directory.

For detailed information about the CLI, see the CLI docs.

Getting support for the Snyk CLI

Submit a ticket to Snyk support whenever you need help with the Snyk CLI or Snyk in general. Note that Snyk support does not actively monitor GitHub Issues on any Snyk development project.

Contributing to the Snyk CLI

The Snyk CLI project is open-source, but Snyk does not encourage outside contributors.

You may look into design decisions for the Snyk CLI.

The Snyk CLI repository is a monorepo that also covers other projects and tools, such as @snyk/protect, also available at npm package for snyk-protect command.

Security

For any security issues or concerns, see the SECURITY.md file in the GitHub repository.

broker's Issues

SSL routines:ssl3_get_record:wrong version number

Hello,

docker image : broker:4.157.1-bitbucket-server

We have installed the snyk broker on our Openshift instance (We do not have a token yet). Outbound connection to snyk is performed through our enterprise proxy (the url is whitelisted as expected)

But the broker fails because of SSL routines:ssl3_get_record:wrong version number

{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"brokerClientId":"7190f16f-8413-49fc-83e6-3d92880cbeab","msg":"generated broker client id","time":"2023-07-13T12:54:14.970Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"enabled":false,"msg":"checking for HA mode","time":"2023-07-13T12:54:14.970Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"enabled":true,"msg":"verifying if preflight checks are enabled","time":"2023-07-13T12:54:14.970Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"enabled":false,"msg":"checking for HA mode","time":"2023-07-13T12:54:14.971Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:16.238Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"attempt":0,"operation":"http check broker-server-status","timeout":100,"msg":"waiting for 100ms before next try","time":"2023-07-13T12:54:17.250Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:18.372Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"attempt":1,"operation":"http check broker-server-status","timeout":200,"msg":"waiting for 200ms before next try","time":"2023-07-13T12:54:19.387Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:20.576Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"attempt":2,"operation":"http check broker-server-status","timeout":400,"msg":"waiting for 400ms before next try","time":"2023-07-13T12:54:21.565Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":40,"retryCount":3,"errorMessage":"write EPROTO 140403255646144:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:\n","url":"https://broker.snyk.io/healthcheck","msg":"retrying request x 3 ","time":"2023-07-13T12:54:23.042Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":50,"error":{"name":"Error","message":"Error executing check with checkId broker-server-status","stack":"Error: Error executing check with checkId broker-server-status\n at HttpCheckService.run (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/checks/http/http-check-service.js:33:19)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)\n at async retry (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/retry/exponential-backoff.js:7:16)\n at async executePreflightChecks (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/checks/index.js:28:29)\n at async Object.module.exports [as client] (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/index.js:33:37)\n at async Object.main (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/index.js:40:12)\n at async module.exports (/home/node/.npm-global/lib/n...
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"url":"https://broker.snyk.io","serverId":"","msg":"broker client is connecting to broker server","time":"2023-07-13T12:54:24.074Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"rulesCount":110,"msg":"loading new rules","time":"2023-07-13T12:54:24.075Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"port":"8000","msg":"local server is listening","time":"2023-07-13T12:54:24.146Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":30,"rulesCount":1,"msg":"loading new rules","time":"2023-07-13T12:54:24.147Z","v":0}
{"name":"snyk-broker","hostname":"bitbucket-server-broker-8659b74568-zk2pr","pid":1,"level":50,"type":"TransportError","description":404,"msg":"Failed to connect to broker server","time":"2023-07-13T12:54:24.528Z","v":0}

Running the docker image outside of Openshift leads to the same error.
The proxy configuration is OK, because running this from the node cli works as expected from the container itself

const request = require('request');
request('https://broker.snyk.io/healthcheck', function (error, response, body) {
...   console.error('error:', error);
...   console.log('statusCode:', response && response.statusCode);
...   console.log('body:', body); 
... });

Response

> error: null
statusCode: 200
body: {"ok":true,"version":"4.155.0"}

Thanks

Implement /readinesscheck combining both healthchecks for Kubernetes deployments

For deployments on Kubernetes it would be great to have a single health check endpoint the combines the results from the already existing healthchecks /healthcheck and /systemcheck for deployments in Kubernetes, because Kubernetes only allows the configuration of a single readiness or liveness check.

The behavior should be a single AND of the status of the already existing checks to clearly report to the operator that there is an issue with connecting to either network endpoint.

:tada: This PR is included in version 4.86.0 :tada:

    :tada: This PR is included in version 4.86.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

Originally posted by @snyksec in #261 (comment)

Confidential data (hashed broker token) stored in path

file - https://github.com/snyk/broker/blob/master/lib/client/dispatcher/client/api.ts
line - 24

Storing confidential information such as broker token in path is generally a bad idea.
This information should be moved to the POST body to let the TLS do the work.

Additional (auth) headers? And/or features in general

This is more of a feature/project mgmt question than an issue so skipping the issue template.

There are currently two different auth header options we can add on the client side to the outgoing connection, basic and token. We would have the need to use something like bearer or even raw (without any prepended word) as well in our environments.

Is this project open to accept pull requests to add those kind of additions into the package or is this functionality very Snyk specific and you'd recommend to fork the repo instead?

This would be something that we (RoadieHQ) would be looking forward to contribute in.

Filtering incoming Client connections

I think I've discovered a vulnerability with the incoming Client connection.

When Client with BROKER_ID 123 is connected to the server and 2nd (evil) Client connects with the same BROKER_ID, it overtakes original Client's position and all Server calls that should go to the original Client goes only to the evil one. Could cause both data leaks and mocked responses to mess with Server.

There are a few ways to mitigate this, although they are not perfect:

• IP filtering: not sure if we can have a stable IP/IP range from those internal networks, but if so, it sounds reasonable
• per-client Server-Client certificate: Client will specify what cert it expects from Server and Server serves different per-client certificate - could be bypassed with a dedicated client ignoring errors IMHO
• OAuth 1: I know, I know… but it would also come with a MAC, key rotation etc.

What prevents it now is that, Nginx can't distinguish incoming connections before handshake because path looks like this path="/primus/?_primuscb=LU9fg6t&EIO=3&transport=polling&t=LU9fg6v&b64=1" and BROKER_ID is in payload so it might have to be resolved on Server - check that BROKER_ID is allowed in ENV and its ENV value matches IP/IP range (in socket.on('identify'))

tl;dr: I'd propose a Server would only accept Client connections if they are named in the ENV and their IP/IP range matches. So Server would also have a MY-CLIENT-ID=<IP> in ENV for each allowed Client.

Require Linux/ARM64 docker image for snyk/broker

Hi Team,

I have been working on building snyk/broker docker image for both amd64 and arm64 platform. Successfully built and tested the docker image for both the platforms.

Do you have any plans for releasing ARM64 images?

It will be very helpful if an ARM64 image is available.

README.md cites the wrong port

In the README, we provide commands in this format:

docker run --restart=always \
           -p 8000:8000 \
...

However, the mapping of the 8000:8000 port doesn't work since we default to 7341. I recommend instead we specify

-p 7341:7341
as a default, or even
-p 8000:7341
if we're interested in using port 8000. In the present form, 8000:8000 goes nowhere and the misalignment to the actual port may confuse our end users.

Tough-Cookie vulnerability. Possible approaches?

We received a notification that there is a vulnerable version of tough-cookie coming from one of the dependencies of this package.

The CVE is in here:

• https://nvd.nist.gov/vuln/detail/CVE-2023-26136
• https://security.snyk.io/vuln/SNYK-JS-TOUGHCOOKIE-5672873

There was an attempt to remove this dep within this PR: https://github.com/snyk/broker/pull/570/files but that never made to the final image since the dependency is needed.

What would be the best approaches to mitigate this?

Prometheus metrics support?

How can we monitor the broker? We are planning to host it on Kuberentes, so prometheus metrics support (disabled by default, with paramter that enable them) will be great. This express middleware looks promising.

Misleading error message

When installing the Snyk-broker on a Openshift-platform with limitations on file access due to security context a misleading error message is given. This will, at least it did for us, possibly result in time waste for the "sys-admins". We spent many ours trying to figure out what the No Filters found. A Broker requires filters to run. Shutting down.-message meant, as there is no reference to any filters in the documentation or in the Helm-chart. We finally had a look at the source code and was able to backtrack the message

Expected behaviour

If the application is not able to load the files in defaultFilters a complementary and precise error message should be given. In our case the process did not have access to load the files (access denied)

Actual behaviour

The error message No Filters found. A Broker requires filters to run. Shutting down. is given.

broker/lib/index.ts

Line 51 in f001b36

`No Filters found. A Broker requires filters to run. Shutting down.`,

Steps to reproduce

Make the files in defaultFilters inaccessible for the process while the application starts

Other container engine than Docker

Hi,
Are other container engines besides Docker supported for running the broker container ?
For example: Podman - which is currently included and supported in EL8 and newer and Ubuntu 22.10 and newer.
The Community Edition of Docker is a 3rd party package usually not supported by any Linux distro vendors.
Thanks !

missing contributing.md file

Suggest we should have a CONTRIBUTING.md file in this repository.

Audit log

We need to provide a sensible logging for audit - as in who accessed what resource when and with what outcome. Debug is useful for development, but currently, it on one side exposes tokens etc. and doesn't record IP of request origin etc.

We have some internal logging format, but it's far from perfect in current state. Do you have more experience with this? The best resource I found so far was this: https://www.owasp.org/index.php/Logging_Cheat_Sheet I can prepare some proposal/PR in the spirit of OWASP guide.

From a technical standpoint, standard output to console should be sufficient afaik - implemented with https://github.com/winstonjs/winston for example.

How to handle self-signed certs on Internal services that Client interacts with

The best option for these cases seems to have an envvar with a CA certificate. I'd discourage anyone from using it with NODE_TLS_REJECT_UNAUTHORIZED - and let's see if we even get to such situation. However, if this happens, maybe we can improve it with:

• certificate pinning (Client-Server) - not sure if this can be enforced with NODE_TLS_REJECT_UNAUTHORIZED active
• Additional encryption/TLS? MAC signing?

I'd remove the NODE_TLS_REJECT_UNAUTHORIZED note from Readme as it is too dangerous in my opinion.

Helm chart?

Is there a plan to create an official helm chart for simple Kubernetes deployment?

Ability to disable variable substitution

It would be fantastic if it was possible to disable the body and header substitution feature of the broker client/server. Ideally this could be done at the filter level with a new option like "disableVarsSubstitution" and or "disableBodyVarsSubstitution/disableHeaderVarsSubstitution". Another option is to have the ability to disable var substitution globally on a client. e.g. broker --disableVarsSubstitution or something like that.

Documentation: CPU and memory requirements for the docker

Please add CPU and memory requirements for the docker to documentation.

<SPAM>

    > Hi, Are other container engines besides Docker supported for running the broker container ? For example: Podman - which is currently included and supported in EL8 and newer and Ubuntu 22.10 and newer. The Community Edition of Docker is a 3rd party package usually not supported by any Linux distro vendors. Thanks !

Originally posted by @Jimimaku in #455 (comment)

Drop JSON configs in favor of YAML

My reason would be that JSON is more machine readable and we want these configs to be easier to audit. Compare this part of config:

{
  "public":
  [
      {
        "//": "used for pushing up webhooks from github",
        "method": "POST",
        "path": "/webhook/github",
        "valid": [
          {
            "//": "accept all pull request state changes (these don't have files in them)",
            "path": "pull_request.state",
            "value": "open"
          },
          {
            "path": "commits.*.added.*",
            "value": "package.json"
          },
          {
            "path": "commits.*.modified.*",
            "value": "package.json"
          },
          {
            "path": "commits.*.added.*",
            "value": ".snyk"
          },
          {
            "path": "commits.*.modified.*",
            "value": ".snyk"
          }
        ]
      }
  ]
}

public:
# used for pushing up webhooks from github
- method: POST
  path: "/webhook/github"
  valid:
  # accept all pull request state changes (these don't have files in them)
  - path: pull_request.state
    value: open
  - path: commits.*.added.*
    value: package.json
  - path: commits.*.modified.*
    value: package.json
  - path: commits.*.added.*
    value: ".snyk"
  - path: commits.*.modified.*
    value: ".snyk"

Thoughts?

4.129.0 fails to start and core dumps

• node -v:

node -v
v16.15.1

• npm -v:

npm -v
8.11.0

• Command run:

We updated from snyk broker from 4.122.0 to 4.129.0 and now the broker fails to start with the following logs:

node[1]: ../src/node_platform.cc:61:std::unique_ptr<long unsigned int> node::WorkerThreadsTaskRunner::DelayedTaskScheduler::Start(): Assertion `(0) == (uv_thread_create(t.get(), start_thread, this))' failed.
 1: 0xb09c10 node::Abort() [node]
 2: 0xb09c8e  [node]
 3: 0xb794ce  [node]
 4: 0xb79596 node::NodePlatform::NodePlatform(int, v8::TracingController*) [node]
 5: 0xacd0e9 node::InitializeOncePerProcess(int, char**, node::InitializationSettingsFlags, node::ProcessFlags::Flags) [node]
 6: 0xacd399 node::Start(int, char**) [node]
 7: 0x7f4d5d387d90  [/lib/x86_64-linux-gnu/libc.so.6]
 8: 0x7f4d5d387e40 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
 9: 0xa4067c  [node]

Adding debug logging doesn't produce any more logs. Host that its running on has plenty of resources and its not a ulimit issue (nodejs/node#43064)

Expected behaviour

Broker should start.

Actual behaviour

Broker dies.

Steps to reproduce

---

If applicable, please append the --debug flag on your command and include the output here **ensuring to remove any sensitive/personal details or tokens.

Snyk Broker not starting: Illegal operation on a directory

I set up the current snyk-broker version via GCPs Cloud Run via Kubernetes. Unfortunately, the container does not spawn correctly. From the log I can see the default config is being created (filters appear and the server listens on 8080), but an error prevents the container from spinning up completely:

Error: EISDIR: illegal operation on a directory, read
    at main (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/webserver.js:37:25)
    at Object.module.exports [as client] (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/client/index.js:21:52)
    at Object.main (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/lib/index.js:34:23)
    at module.exports (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/cli/exec.js:8:42)
    at Object.<anonymous> (/home/node/.npm-global/lib/node_modules/snyk-broker/dist/cli/index.js:22:5) {

My config:

Any idea what directory is being accessed illegally here?