xmidt-org/scytale

Xmidt API server written in Go

scytale

(pronounced "skit-uh-​lee")

Summary

Scytale is the API server of XMiDT. Scytale will fanout the API request to all the petasoses that scytale knows of.

Table of Contents

• Code of Conduct
• Details
• Build
• Deploy
• Contributing

Code of Conduct

This project and everyone participating in it are governed by the XMiDT Code Of Conduct. By participating, you agree to this Code.

Details

Scytale has two API endpoints to interact with the devices: 1) get the statistics for a device and 2) send a WRP Message to the device. If the device isn't connected, a 404 is returned.

Device Statistics - /api/v2/device/{deviceID}/stat endpoint

This will return the statistics of the connected device, including information such as uptime and bytes sent. This information is retrieved from the talaria that the device is connected to.

Send WRP to Device - /api/v2/device/send endpoint

This will send a WRP message to the device. Scytale will accept a WRP message encoded in a valid WRP representation - generally msgpack or json and will forward the request to the correct talaria.

Build

Source

In order to build from the source, you need a working Go environment with version 1.11 or greater. Find more information on the Go website.

You can directly use go get to put the scytale binary into your GOPATH:

GO111MODULE=on go get github.com/xmidt-org/scytale

You can also clone the repository yourself and build using make:

mkdir -p $GOPATH/src/github.com/xmidt-org
cd $GOPATH/src/github.com/xmidt-org
git clone [email protected]:xmidt-org/scytale.git
cd scytale
make build

Makefile

The Makefile has the following options you may find helpful:

• make build: builds the scytale binary
• make docker: builds a docker image for scytale, making sure to get all dependencies
• make local-docker: builds a docker image for scytale with the assumption that the dependencies can be found already
• make test: runs unit tests with coverage for scytale
• make clean: deletes previously-built binaries and object files

RPM

First have a local clone of the source and go into the root directory of the repository. Then use rpkg to build the rpm:

rpkg srpm --spec <repo location>/<spec file location in repo>
rpkg -C <repo location>/.config/rpkg.conf sources --outdir <repo location>'

Docker

The docker image can be built either with the Makefile or by running a docker command. Either option requires first getting the source code.

See Makefile on specifics of how to build the image that way.

For running a command, either you can run docker build after getting all dependencies, or make the command fetch the dependencies. If you don't want to get the dependencies, run the following command:

docker build -t scytale:local -f deploy/Dockerfile .

If you want to get the dependencies then build, run the following commands:

GO111MODULE=on go mod vendor
docker build -t scytale:local -f deploy/Dockerfile.local .

For either command, if you want the tag to be a version instead of local, then replace local in the docker build command.

Kubernetes

A helm chart can be used to deploy scytale to kubernetes

helm install xmidt-scytale deploy/helm/scytale

Deploy

For deploying a XMiDT cluster refer to getting started.

For running locally, ensure you have the binary built. If it's in your GOPATH, run:

scytale

If the binary is in your current folder, run:

./scytale

Contributing

Refer to CONTRIBUTING.md.

scytale's Issues

Add explanation for capabilityConfig in scytale.yaml

scytale/primaryHandler.go

Line 139 in 5e33f3b

v.UnmarshalKey("capabilityConfig", &capabilityConfig)

The capabilityConfig values should be explained in the tr1d1um.yaml file, but all values should be commented out - if anyone uses the default yaml to test or play around, they won't need to check for capabilities in the jwt.

related to xmidt-org/tr1d1um#124, they could probably have similar or identical explanations

CVE-2019-11253 (High) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2019-11253 - High Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

Improper input validation in the Kubernetes API server in versions v1.0-1.12 and versions prior to v1.13.12, v1.14.8, v1.15.5, and v1.16.2 allows authorized users to send malicious YAML or JSON payloads, causing the API server to consume excessive CPU or memory, potentially crashing and becoming unavailable. Prior to v1.14.0, default RBAC policy authorized anonymous users to submit requests that could trigger this vulnerability. Clusters upgraded from a version prior to v1.14.0 keep the more permissive policy by default for backwards compatibility.

Publish Date: 2019-10-17

URL: CVE-2019-11253

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: kubernetes/kubernetes#83253

Release Date: 2019-10-17

Fix Resolution: v1.13.12;v1.14.8;v1.15.5;v1.16.2

---

Step up your Open Source Security Game with WhiteSource here

Failed compilation in centos 7

Hi, I'm trying to build scytale in a cento 7 server and I get this error:

/primaryHandler.go:78:72: fanoutOptions.NewDecoderPool undefined (type *wrphttp.FanoutOptions has no field or method NewDecoderPool)
./primaryHandler.go:79:55: fanoutOptions.NewEncoderPool undefined (type *wrphttp.FanoutOptions has no field or method NewEncoderPool)
./primaryHandler.go:91:72: fanoutOptions.NewDecoderPool undefined (type *wrphttp.FanoutOptions has no field or method NewDecoderPool)
./primaryHandler.go:92:55: fanoutOptions.NewEncoderPool undefined (type *wrphttp.FanoutOptions has no field or method NewEncoderPool)
./primaryHandler.go:176:43: not enough arguments in call to webhook.NewFactory
have (*viper.Viper)
want (*viper.Viper, *xmetrics.Registry)
./primaryHandler.go:195:27: not enough arguments in call to webHookFactory.Notifier.Initialize
have (*mux.Router, *url.URL, "net/http".Handler, "github.com/go-kit/kit/log".Logger, nil)
want (*mux.Router, *url.URL, "net/http".Handler, "github.com/go-kit/kit/log".Logger, *xmetrics.Registry, func() time.Time)

Whenever I run "go build" I obtained the mentioned error. Already checked that the needed packages are installed and updated. Could you please show me the correct build process?

Thanks!

curl to scytale URL returns 404 Not Found error

We have scytale, talaria, and tr1d1um running on a single machine with the three services currently listening:

tcp 0 0 :::8096 :::* LISTEN 4180/tr1d1um
tcp 0 0 :::8080 :::* LISTEN 4157/talaria
tcp 0 0 :::6000 :::* LISTEN 4136/scytale
tcp 0 0 :::8081 :::* LISTEN 4157/talaria
tcp 0 0 :::6001 :::* LISTEN 4136/scytale
tcp 0 0 :::6002 :::* LISTEN 4136/scytale
tcp 0 0 :::8095 :::* LISTEN 4180/tr1d1um

The RDK-B is currently connected to talaria on port 8080. If a curl command is executed, the following is displayed:

[root@hummingbird talaria]# curl -i http://10.11.160.49:6000/api/v2/device/f8a097ea7206/stat
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Mon, 04 Dec 2017 17:18:18 GMT
Content-Length: 19
404 page not found

scytale.log file only shows the following:

{"configurationFile":"/etc/scytale/scytale.json","level":"info","ts":"2017-12-01T19:47:54.385961766Z"} {"address":":6001","caller":"webpa.go:235","level":"info","msg":"starting server","name":"scytale.health","ts":"2017-12-01T19:47:54.386156863Z"} {"caller":"health.go:118","level":"debug","msg":"Health Monitor Started","ts":"2017-12-01T19:47:54.386196615Z"} {"address":":6002","caller":"webpa.go:244","level":"info","msg":"starting server","name":"scytale.pprof","ts":"2017-12-01T19:47:54.386209981Z"} {"address":":6000","caller":"webpa.go:249","level":"info","msg":"starting server","name":"scytale","ts":"2017-12-01T19:47:54.386221128Z"}

Please see attachments for scytale.json and talaria.json configuration files.
webpa config.zip

Optimize makefile for local docker testing

The solution was already accepted in Talaria (xmidt-org/talaria@3db2d63) and we just need to port it over here

scytale may run out of memory when sent large wrp messages

This should be investigated to determine if scytale can be affected in the same manor.
This is a cross post of the issue in caduceus which may be applicable to scytale
/github/xmidt-org/caduceus#225

Potential use of API Requests Total metric

The suggested idea: Instead of creating a new counter (like this https://github.com/xmidt-org/scytale/blob/master/metrics.go#L10) to keep track of which HTTP requests with a WRP message will be turned down due to lack of partnerID privileges, use the existing API request counter with additional labels (https://github.com/xmidt-org/webpa-common/blob/master/server/metrics.go#L23)

Potential challenges:

• Refactor the webpa-common/server package to take optional pre-fixed labels so existing de API request counter logic decorator still works
  https://pkg.go.dev/github.com/prometheus/client_golang/prometheus?tab=doc#CounterVec.CurryWith

• Refactor in client code to obtain values for the code and method labels (might involve response writer interceptors).

• Potential count complexities to account for
  Suppose we add the label wrp which can take values either true or false (by default false)

Assuming we perform all the refactors above and after two successful api requests, 1 of which was to a wrp endpoint, the time series values should be

api_requests_total{code="200", method="POST", wrp="false"} 2
api_requests_total{code="200", method="POST", wrp="true"} 1

This would change the meaning of total successful api requests as the first will include all values of the second. Notice that this will require updating existing metrics dashboards so values are normalized.

Add partnerIDs to internal webhook store

Context

Currently, we are not filtering events based on partnerIDs but with the switch to argus as the webhook storage strategy, we'd like to add a configuration option to filter events by such values.

Task:

To do that (xmidt-org/caduceus#181), caduceus needs access to the partnerIDs the listener registered with. On webhook registration, the value needs to be captured and saved as part of a registration

If JWT validation is enabled, the list(string) value will come from the claim allowedResources.allowedPartners

If JWT validation is not enabled, a header based fall back should be provided (As an example, Tr1d1um already does this: https://github.com/xmidt-org/tr1d1um/blob/b117f90bda5d29cea5283da3abee1165cfbf4921/translation/transport.go#L94)

CVE-2020-12758 (High) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2020-12758 - High Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

HashiCorp Consul and Consul Enterprise could crash when configured with an abnormally-formed service-router entry. Introduced in 1.6.0, fixed in 1.6.6 and 1.7.4.

Publish Date: 2020-06-11

URL: CVE-2020-12758

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-12758

Release Date: 2020-06-11

Fix Resolution: 1.6.6,1.7.4

---

Step up your Open Source Security Game with WhiteSource here

fanout.endpoints and service.fixed

Both of these configuration values contain a list of petasoses to fanout to:
fanout.endpoints
service.fixed

Do we need both? Can we deprecate/remove one?

Add functionality for a request to be done on behalf of someone else

In addition to the authorization header, we want to add another header that allows client A to submit a request on behalf of another client, B. So client A's jwt will be in the authorization header, and the jwt of client B will be in a separate header. This means that client B may not have access to scytale, while client A does. Client B's jwt token may provide other context that talaria wants to examine.

related to #74. When we start forwarding a jwt to talaria, we want to make sure it's either the one jwt we receive, or it's the jwt from client B in the above scenario.

add control interface to manage fanout to talaria via consul

in part with #112 an additional feature to control the fanout will be required. As an MVP a pause, unpause to update the list of datacenters can come in handy.

Scytale should add support for fanout optimization based on partner-id and theater

Scytale should add support for limiting fanout based on a mapping of partner-id to theater DC list. This issue works with the issues below allowing a user to limit the number of DCs served to a partner-id to improve latency and reduce operational costs.

Related Issues:

• xmidt-org/petasos#57
• xmidt-org/talaria#158

CVE-2021-3121 (High) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2021-3121 - High Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

An issue was discovered in GoGo Protobuf before 1.3.2. plugin/unmarshal/unmarshal.go lacks certain index validation, aka the "skippy peanut butter" issue.

Publish Date: 2021-01-11

URL: CVE-2021-3121

CVSS 3 Score Details (8.6)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-3121

Release Date: 2021-01-11

Fix Resolution: v1.3.2

---

Step up your Open Source Security Game with WhiteSource here

JSON handler not working when using json.Marshal

I tried to send a wrp.SimpleRequestResponse{} to the /device/send endpoint encoded as JSON. There is a handler registered for header Content-Type: application/json.

What happened:
I encoded the message using json.Marshal() and was wondering why petasos complaint with

{....."device_id":"","error":"Missing device name header","level":"error",.....}

When you use json.Marshal() it will use the name of the exported members as json key

{...Type:SimpleRequestResponseMessageType Source: Destination:mac:112233445566 ....}

When you hit the JSON handler it decodes the message using wrp tags, which are not the same as the exported fields. It would expect a json message like this.

{"msg_type":3,"source":"","dest":"mac:112233445566",...}

As the decoder generated with wrp.NewDecoderBytes doesn't spit out any errors when decoding my message (which is ok as he received valid JSON) , i was wondering why i received a empty wrp message + empty X-Webpa-Device-Name header when the fanout request hit petasos.

Solution:
Three things which can prevent future users from making the same mistake:

1. Change the header from application/json to something more appropriate.
2. Add json tags to wrp.SimpleRequestResponse etc. and change https://github.com/xmidt-org/wrp-go/blob/master/wrp/format.go#L34 to json
3. Use json.Decode instead of wrp.Decode

I would opt for option 3, but i might miss some arguments why you actually decided to use wrp.Decode

Scytale should log a Bookkeeping Entry like Tr1d1um

Scytale should log a Bookeeping Entry like Tr1d1um.

https://github.com/Comcast/tr1d1um/blob/46607c2663aae15358ed34d58e5dcdeecf891466/src/tr1d1um/common/utils.go#L35

Data to log:

Wrp.msg_type
Wrp.dest
Wrp.source
Wrp.transaction_uuid
Wrp.status
Wrp.partner_ids
satClientID

Drop default responsibility of populating partnerID field in WRP Messages

Current state:

Scytale takes on the responsibility of populating the partnerIDs field for incoming WRP messages
https://github.com/xmidt-org/scytale/blob/master/primaryHandler.go#L76

Desired state:

Scytale should now assume the partnerIDs field is already populated by the API consumer. HOWEVER, to help with the transition, scytale still needs to do the work for consumers who don't AND take a note (in the form of metrics or logging) of their IDs for user education.

Remove Authorization Header from logged headers

Example patch: https://github.com/xmidt-org/argus/blob/c2562df18c76f64ff59262c55a45bb5804566a00/auth/logger.go#L49-L58
Related to xmidt-org/xmidt#52

CVE-2021-32574 (High) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2021-32574 - High Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

HashiCorp Consul and Consul Enterprise 1.3.0 through 1.10.0 Envoy proxy TLS configuration does not validate destination service identity in the encoded subject alternative name. Fixed in 1.8.14, 1.9.8, and 1.10.1.

Publish Date: 2021-07-17

URL: CVE-2021-32574

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: High
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://github.com/hashicorp/consul/releases/tag/v1.10.1

Release Date: 2021-07-17

Fix Resolution: v1.10.1

---

Step up your Open Source Security Game with WhiteSource here

incorrect SD count reported in metrics

In reviewing the metrics for a new deployment we noticed that the xmidt_scytale_sd_instance_count was incorrect. In a setup with 3 datacenters and 2 scytale each (total of 6 scytale), the metric reported 0 servers on 5 scytales, a 3 on 1 scytale.

CVE-2019-11250 (Medium) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2019-11250 - Medium Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

The Kubernetes client-go library logs request headers at verbosity levels of 7 or higher. This can disclose credentials to unauthorized users via logs or command output. Kubernetes components (such as kube-apiserver) prior to v1.16.0, which make use of basic or bearer token authentication, and run at high verbosity levels, are affected.

Publish Date: 2019-08-29

URL: CVE-2019-11250

CVSS 3 Score Details (6.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: Low
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://osv.dev/vulnerability/GO-2020-0065

Release Date: 2020-10-16

Fix Resolution: v1.16.0-beta.1

---

Step up your Open Source Security Game with WhiteSource here

CVE-2020-26160 (High) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2020-26160 - High Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 5855b3521f79a7ccb3046e0e57a224c6aa737989

Found in base branch: main

Vulnerability Details

jwt-go before 4.0.0-preview1 allows attackers to bypass intended access restrictions in situations with []string{} for m["aud"] (which is allowed by the specification). Because the type assertion fails, "" is the value of aud. This is a security problem if the JWT token is presented to a service that lacks its own audience check.

Publish Date: 2020-09-30

URL: CVE-2020-26160

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://nvd.nist.gov/vuln/detail/CVE-2020-26160

Release Date: 2020-09-30

Fix Resolution: v4.0.0-preview1

---

Step up your Open Source Security Game with WhiteSource here

User documentation request

Is there any documentation on how to configure and stand up this WebPA Server component?

Pass JWT token during fanout

Overview:
As part of our plan to ensure API consumer groups can communicate only with devices they own, we need to pass the API JWT token they present downstream to talaria during the fanout process.

We need to do this as only talaria has device-specific information to perform this check.

Task:
Pass an HTTP header (X-Xmidt-Api-Authorization or a better name) that contains the JWT token value in the outgoing fanout requests.

Authorize WRP messages

Current state:

Scytale authorizes incoming WRP messages as a byproduct of the fact that it populates the partnerIDs field in the WRP on behalf of the user from the trusted source of truth: JWT token

Desired state:

Once #82 is fixed, the assumption will be that users will be expected to configure such field in their WRP messages.

Validation work:
Now, scytale's new task is to perform the following checks on the WRP partnerIDs field:

• Ensure it is not empty. If it is, reject with a 400 http status code
• If the JWT allowedResources.allowedPartners claim contains the * (wildcard) partnerID, then the WRP Message is valid since the user is allowed to send any partnerIDs in the WRP.
• Ensure it is a subset of its counterpart credential in the JWT. If it's not, reject with a 403 or 401? (check with team)

Deployment dependency:

Resolution to: #82

Scytale returns 500 when decoding a bad wrp/msgpack request

Issuing the following request to Scytale results in a 500:
version:

  version: 	v0.4.1
  go version: 	go1.13.3
  built time: 	2020-02-17 20:31:26OURCE
  git commit: 	3892108
  os/arch: 	linux/amd64

curl https://scytale.example.com:443/api/v2/device -H "Authorization:$auth" -H "Content-Type: application/msgpack" -d @out.bin -i

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Scytale-Build: 0.4.1
X-Scytale-Flavor: water
X-Scytale-Region: ch2g
X-Scytale-Server: [redacted]
X-Scytale-Start-Time: 17 Feb 20 21:48 UTC
Date: Wed, 19 Feb 2020 23:26:30 GMT
Content-Length: 83

CVE-2020-8911 (Medium) detected in github.com/aws/aws-sdk-go-v1.31.6 - autoclosed

CVE-2020-8911 - Medium Severity Vulnerability

Vulnerable Library - github.com/aws/aws-sdk-go-v1.31.6

AWS SDK for the Go programming language.

Dependency Hierarchy:

• github.com/xmidt-org/webpa-common/v2-v2.0.4 (Root Library)
  • ❌ github.com/aws/aws-sdk-go-v1.31.6 (Vulnerable Library)

Found in HEAD commit: 17795e7f2e51f2304ab5d7b0f81e8991362ab5c1

Found in base branch: main

Vulnerability Details

A padding oracle vulnerability exists in the AWS S3 Crypto SDK for GoLang versions prior to V2. The SDK allows users to encrypt files with AES-CBC without computing a Message Authentication Code (MAC), which then allows an attacker who has write access to the target's S3 bucket and can observe whether or not an endpoint with access to the key can decrypt a file, they can reconstruct the plaintext with (on average) 128*length (plaintext) queries to the endpoint, by exploiting CBC's ability to manipulate the bytes of the next block and PKCS5 padding errors. It is recommended to update your SDK to V2 or later, and re-encrypt your files.

Publish Date: 2020-08-11

URL: CVE-2020-8911

CVSS 3 Score Details (5.6)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Local
  • Attack Complexity: High
  • Privileges Required: Low
  • User Interaction: None
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: High
  • Integrity Impact: None
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8911

Release Date: 2020-07-21

Fix Resolution: v1.34.1

---

Step up your Open Source Security Game with Mend here

Add duration of calls to WRP.spans driven by WRP.include_spans.

Stop sending the `X-Webpa-Device-Name` header during fanout

We are currently sending this header containing the WRP message destination. Talaria is set to no longer require/use this header (xmidt-org/talaria#132) so Scytale should just drop it.

Update to latest version of bascule

https://github.com/xmidt-org/bascule/releases/tag/v0.10.2 contains a security vulnerability fix that should be addressed ASAP.

Bump wrp-go to v3 and update handler

Bumping wrp-go to v3 in scytale requires modifying the handler in this file:

scytale/WRPHandler.go

Line 22 in 19f0bea

func nonWRPResponseWriterFactory(w http.ResponseWriter, _ *wrphttp.Request) (wrphttp.ResponseWriter, error) {

Part of the v3 update was to modify this interface and add functions to it:
https://github.com/xmidt-org/wrp-go/blob/7fa53e0e507054de21e82c93e9f131fbcee12e94/wrphttp/requestResponse.go#L92

so updating to the newest wrp-go version creates this build error:

./WRPHandler.go:23:9: cannot use &nonWRPResponseWriter literal (type *nonWRPResponseWriter) as type wrphttp.ResponseWriter in return argument:
	*nonWRPResponseWriter does not implement wrphttp.ResponseWriter (missing WRPFormat method)

Scytale Crashes Due to Request

curl -i -H "Authorization: Bearer $XMIDT_SAT" https://api.example.net/api/v2/device -X POST -H "X-Xmle" -H "X-Xmidt-Dest: mac:112233445566/iot" -H "X-Webpa-Device-Name: mac:112233445566/iot" -H "X-Xmidt-Transaction-Uuid: 1234"

with:
runtime error: invalid memory address or nil pointer dereference

Subscribe to signals we care about only

See xmidt-org/talaria#126 for context

Talaria's main.go file already has the solution model.

Scytale should accept hook registration

Today there is a gap in scytale where it is not accepting webhook registration. It should accept/own webhook registration & not tr1d1um.

Add Dynamic Versioning

Here is an example of the changes required xmidt-org/svalinn#111 and xmidt-org/svalinn#112

Not able to install scytale-0.1.1-155.el6.x86_64.rpm on CentOS 7.6

I got this error:

rpm -Uvh scytale-0.1.1-155.el6.x86_64.rpm

Preparing... ################################# [100%]
file /usr/bin from install of scytale-0.1.1-155.el6.x86_64 conflicts with file from package filesystem-3.2-25.el7.x86_64

Is there a version of scytale that will get installed on CentOS 7.6? The latest versions of both talaria and tr1d1um got installed viz talaria-0.1.1-244.el6.x86_64.rpm and tr1d1um-0.1.1-326.el6.x86_64.rpm.

Overwriting the logger in request context

scytale/primaryHandler.go

Line 309 in c6f24fd

logginghttp.SetLogger(

The logger in the context is being set with certain key values pairs at the place above, but when we added bascule to scytale, the logger was added to the context earlier in the middleware chain using this function:

scytale/primaryHandler.go

Line 65 in c6f24fd

func SetLogger(logger log.Logger) func(delegate http.Handler) http.Handler {

Both ultimately use webpa-common/logging's WithLogger() function, so the later setting of the logger is overwriting the initial one. We should probably just set the logger once. The additional key-value pairs provided in the logginghttp call should be added in the earlier logger setting, and then this later one can be removed.

CVE-2020-25201 (High) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2020-25201 - High Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

HashiCorp Consul Enterprise version 1.7.0 up to 1.8.4 includes a namespace replication bug which can be triggered to cause denial of service via infinite Raft writes. Fixed in 1.7.9 and 1.8.5.

Publish Date: 2020-11-04

URL: CVE-2020-25201

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://github.com/hashicorp/consul/blob/master/CHANGELOG.md#185-october-23-2020

Release Date: 2020-09-05

Fix Resolution: 1.8.5

---

Step up your Open Source Security Game with WhiteSource here

Allow requests that don't pass a check through, but log context

If a request's jwt doesn't pass the capabilities check, still allow the request. However, log a warning that the capabilities were wrong. Make sure to include the client id and the endpoint the request is for.

This should be configurable and exist in harmony with the capabilityCheck configuration.

Scytale needs to validate incoming WRPs for correctness.

The wrp-go library will be adding a validate function/utility that will need to be called after the WRP is converted into a struct by Scytale.

Why?

Other components down stream can trust the inputs have been validated so they can use the WRPs without the need to validate them each step of the way. The validation in Scytale prevents a larger system effort for debugging problems spread across the Xmidt system.

Reject a request whose auth doesn't contain the wrp partner ID

There should be a configurable option to enable checking partner IDs in the authorization and ensuring that the partner IDs in the wrp are a subset or equal to the partner IDs in the authorization token. If this check fails, the request should be rejected.

X-Xmidt-Status: 404 results in a 200

The HTTP Status code should be a 599 instead of 200 when the X-Xmidt-Status is not a 2xx value.

-H "X-Xmidt-Message-Type: Retrieve" \
-H "X-Xmidt-Transaction-Uuid: bd4ad2d1-5c9c-486f-8e25-52c242b38f72" \
-H "X-Xmidt-Path:" \
-H "X-Xmidt-Source: dns:example" \
-H "X-Webpa-Device-Name: mac:4ca161000001/aker/schedule" \
-X POST -i

HTTP/1.1 200 OK
X-Webpa-Device-Name: dns:example
X-Xmidt-Message-Type: Retrieve
X-Xmidt-Request-Delivery-Response: 0
X-Xmidt-Source: mac:4ca161000001/aker/schedule
X-Xmidt-Span: "https://fabric.xmidt.example.net:8080/api/v2/device/send","2017-12-22T00:15:18Z","56.772842ms"
X-Xmidt-Status: 404
X-Xmidt-Transaction-Uuid: bd4ad2d1-5c9c-486f-8e25-52c242b38f72
Date: Fri, 22 Dec 2017 00:15:18 GMT
Content-Length: 0
Content-Type: text/plain; charset=utf-8```

Add Periodic Watch Datacenters

Current Results

Scytale will currently get the current list of datacenters(dc) from the consul agent and will query those dc for talaria. That list of DCs will never be updated. So if a new dc is added or if a dc is removed, scytale will be unaware of the change.

Expected Results

Have scytale get a current list of dcs via a configurable periodic interval.

CVE-2021-31525 (Medium) detected in github.com/golang/net-5f4716e94777e714bc2fb3e3a44599cb40817aac, github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2021-31525 - Medium Severity Vulnerability

Vulnerable Libraries - github.com/golang/net-5f4716e94777e714bc2fb3e3a44599cb40817aac, github.com/hashicorp/consul-v1.7.0

github.com/golang/net-5f4716e94777e714bc2fb3e3a44599cb40817aac

[mirror] Go supplementary network libraries

Dependency Hierarchy:

• github.com/xmidt-org/webpa-common/v2-v2.0.1 (Root Library)
  • github.com/miekg/dns-v1.1.26
    • ❌ github.com/golang/net-5f4716e94777e714bc2fb3e3a44599cb40817aac (Vulnerable Library)

github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

net/http in Go before 1.15.12 and 1.16.x before 1.16.4 allows remote attackers to cause a denial of service (panic) via a large header to ReadRequest or ReadResponse. Server, Transport, and Client can each be affected in some configurations.

Publish Date: 2021-05-27

URL: CVE-2021-31525

CVSS 3 Score Details (5.9)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: High
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://bugzilla.redhat.com/show_bug.cgi?id=1958341

Release Date: 2021-04-22

Fix Resolution: golang - v1.15.12,v1.16.4,v1.17.0

---

Step up your Open Source Security Game with WhiteSource here

Make OpenTelemetry tracing an optional feature

We should try and stay consistent with how the feature is configured and set up. Here's an example from Tr1d1um: xmidt-org/tr1d1um#203

CVE-2020-29652 (High) detected in github.com/hashicorp/consul-v1.7.0, github.com/golang/crypto-5c72a883971a4325f8c62bf07b6d38c20ea47a6a - autoclosed

CVE-2020-29652 - High Severity Vulnerability

Vulnerable Libraries - github.com/hashicorp/consul-v1.7.0, github.com/golang/crypto-5c72a883971a4325f8c62bf07b6d38c20ea47a6a

github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

github.com/golang/crypto-5c72a883971a4325f8c62bf07b6d38c20ea47a6a

[mirror] Go supplementary cryptography libraries

Dependency Hierarchy:

• github.com/xmidt-org/webpa-common/v2-v2.0.1 (Root Library)
  • github.com/miekg/dns-v1.1.26
    • ❌ github.com/golang/crypto-5c72a883971a4325f8c62bf07b6d38c20ea47a6a (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

A nil pointer dereference in the golang.org/x/crypto/ssh component through v0.0.0-20201203163018-be400aefbc4c for Go allows remote attackers to cause a denial of service against SSH servers.

Publish Date: 2020-12-17

URL: CVE-2020-29652

CVSS 3 Score Details (7.5)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: None
  • Scope: Unchanged
• Impact Metrics:
  • Confidentiality Impact: None
  • Integrity Impact: None
  • Availability Impact: High

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://groups.google.com/g/golang-announce/c/ouZIlBimOsE?pli=1

Release Date: 2020-12-17

Fix Resolution: v0.0.0-20201216223049-8b5274cf687f

---

Step up your Open Source Security Game with WhiteSource here

Error messages removed from payload and added as header

As part of streamlining the error reporting for api requests:

1. payload returned for a non-success responses should be empty
2. a header (x-xmidt-message-error ) should be return with the appropriate error message

CVE-2020-25864 (Medium) detected in github.com/hashicorp/consul-v1.7.0 - autoclosed

CVE-2020-25864 - Medium Severity Vulnerability

Vulnerable Library - github.com/hashicorp/consul-v1.7.0

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Dependency Hierarchy:

• github.com/go-kit/kit-v0.10.0 (Root Library)
  • ❌ github.com/hashicorp/consul-v1.7.0 (Vulnerable Library)

Found in HEAD commit: 4d0b2272bbd10b29d6b8237b01710a7f5adbe8a6

Found in base branch: main

Vulnerability Details

HashiCorp Consul and Consul Enterprise up to version 1.9.4 key-value (KV) raw mode was vulnerable to cross-site scripting. Fixed in 1.9.5, 1.8.10 and 1.7.14.

Publish Date: 2021-04-20

URL: CVE-2020-25864

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://security.archlinux.org/CVE-2020-25864

Release Date: 2020-09-24

Fix Resolution: v1.7.14, v1.8.10, v1.9.5, v1.10.0

---

Step up your Open Source Security Game with WhiteSource here

Rename master branch to main

Also have to change references to the branch in .travis.yml, README, and CONTRIBUTING. Double check any other markdown files as well - sometimes links have the branch name in them.

Local testing for scytale doesn't work

When trying to test scytale using a local version of webpa-common by using a replace directive in the go.mod, the following error results:

go: github.com/xmidt-org/[email protected]: parsing ../webpa-common/go.mod: open /go/src/github.com/xmidt-org/webpa-common/go.mod: no such file or directory
make: *** [Makefile:20: build] Error 1

This is after making the vendor changes to the Makefile and following the local testing instructions in the talaria README. Strangely, the same changes work in tr1d1um and allows for tr1d1um to be used to test webpa-common locally.