docker-archive/oscalkit

NIST OSCAL SDK and CLI

oscalkit

In development. Since the OSCAL standard is still under active development, parsing errors may occur if running the included CLI tool against OSCAL documents that are developed against iterations of the schemas that aren't supported. Individual Releases of oscalkit will indicate in the notes which commits in the usnistgov/OSCAL repo against which the tool has been tested.

Barebones Go SDK for the Open Security Controls Assessment Language (OSCAL) which is in development by the National Institute of Standards and Technology (NIST). A CLI tool is also included for processing OSCAL documents, converting between OSCAL-formatted XML, JSON and YAML and for converting from OpenControl projects in to OSCAL. The tool also supports Go source code generation from OSCAL formatted catalog and profile artifacts.

Documentation for the OSCAL standard can be found at https://pages.nist.gov/OSCAL.

Supported OSCAL Components

The following OSCAL components are currently supported:

Component	Schemas
Catalog	XSD | JSON schema | metaschema
Profile	XSD | JSON schema | metaschema
Implementation (WIP)	Currently based on a combination of the model being developed in usnistgov/OSCAL#216 and the component definition prototype in this Gist

Installing

You can download the appropriate oscalkit command-line utility for your system from the GitHub Releases page. You can move it to an appropriate directory listed in your $PATH environment variable. A Homebrew recipe is also available for macOS along with a Docker image which has been published to Docker Hub.

Homebrew

$ brew tap docker/homebrew-oscalkit
$ brew install oscalkit

Docker

Running the oscalkit Docker container requires either bind-mounting the directory containing your source files or passing file contents in to the command via stdin.

$ docker pull docker/oscalkit:0.2.0
$ docker run -it --rm -v $PWD:/data -w /data docker/oscalkit:0.2.0 convert oscal-core.xml

via stdin:

$ docker run -it --rm docker/oscalkit:0.2.0 convert < oscal-core.xml

Usage

NAME:
   oscalkit - OSCAL toolkit

USAGE:
   oscalkit [global options] command [command options] [arguments...]

VERSION:
   0.2.0


COMMANDS:
     convert         convert between one or more OSCAL file formats and from OpenControl format
     validate        validate files against OSCAL XML and JSON schemas
     sign            sign OSCAL JSON artifacts
     generate        generates go code against provided profile
     implementation  generates go code for implementation against provided profile and excel sheet
     help, h         Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug, -d    enable debug command output
   --help, -h     show help
   --version, -v  print the version

Convert between XML and JSON

oscalkit can be used to convert one or more source files between OSCAL-formatted XML and JSON.

NAME:
   oscalkit convert oscal - convert between one or more OSCAL file formats

USAGE:
   oscalkit convert oscal [command options] [source-files...]

DESCRIPTION:
   Convert between OSCAL-formatted XML and JSON files. The command accepts
   one or more source file paths and can also be used with source file contents
   piped/redirected from STDIN.

OPTIONS:
   --output-path value, -o value  Output path for converted file(s). Defaults to current working directory
   --output-file value, -f value  File name for converted output from STDIN. Defaults to "stdin.<json|xml|yaml>"
   --yaml                         If source file format is XML or JSON, also generate equivalent YAML output

Examples

Convert OSCAL-formatted NIST 800-53 declarations from XML to JSON:

$ oscalkit convert oscal SP800-53-declarations.xml

Convert OSCAL-formatted NIST 800-53 declarations from XML to JSON via STDIN (note the use of "-"):

$ cat SP800-53-declarations.xml | oscalkit convert oscal -

Signing OSCAL JSON with JWS

oscalkit can be used to sign OSCAL-formatted JSON artifacts using JSON Web Signature (JWS)

NAME:
   oscalkit sign - sign OSCAL JSON artifacts

USAGE:
   oscalkit sign [command options] [files...]

OPTIONS:
   --key value, -k value  private key file for signing. Must be in PEM or DER formats. Supports RSA/EC keys and X.509 certificats with embedded RSA/EC keys
   --alg value, -a value  algorithm for signing. Supports RSASSA-PKCS#1v1.5, RSASSA-PSS, HMAC, ECDSA and Ed25519

The following signing algorithms are supported:

Signing / MAC	Algorithm identifier(s)
RSASSA-PKCS#1v1.5	RS256, RS384, RS512
RSASSA-PSS	PS256, PS384, PS512
HMAC	HS256, HS384, HS512
ECDSA	ES256, ES384, ES512
Ed25519	EdDSA

Examples

Sign OSCAL-formatted JSON using a PEM-encoded private key file and the PS256 signing algorithm:

$ oscalkit sign --key jws-example-key.pem --alg PS256 NIST_SP-800-53_rev4_catalog.json

Convert from OpenControl project to OSCAL [Experimental]

This feature has been temporarily disabled pending usnistgov/OSCAL#216 and usnistgov/OSCAL#215

oscalkit also supports converting OpenControl projects to OSCAL-formatted JSON. You will need both the path to the opencontrol.yaml file and the opencontrols/ directory which is created when you run a compliance-masonry get command.

NAME:
   oscalkit convert opencontrol - convert from OpenControl format to OSCAL "implementation" format

USAGE:
   oscalkit convert opencontrol [command options] [opencontrol.yaml-filepath] [opencontrols-dir-path]

DESCRIPTION:
   Convert OpenControl-formatted "component" and "OpenControl" YAML into
   OSCAL-formatted "implementation" layer JSON

OPTIONS:
   --yaml, -y  Generate YAML in addition to JSON
   --xml, -x   Generate XML in addition to JSON

Examples

Convert OpenControl project to OSCAL-formatted JSON:

$ oscalkit convert opencontrol ./opencontrol.yaml ./opencontrols/

Validate against XML and JSON schemas

The tool supports validation of OSCAL-formatted XML and JSON files against the corresponding OSCAL XML schemas (.xsd) and JSON schemas. Schemas are packaged with the tool and found automatically based on the type of OSCAL file. XML schema validation requires the xmllint tool on the local machine (included with macOS and Linux. Windows installation instructions here)

NAME:
   oscalkit validate - validate files against OSCAL XML and JSON schemas

USAGE:
   oscalkit validate [command options] [files...]

DESCRIPTION:
   Validate OSCAL-formatted XML files against a specific XML schema (.xsd)
   or OSCAL-formatted JSON files against a specific JSON schema

Examples

Validate FedRAMP profile in OSCAL-formatted JSON against the corresponding JSON schema

$ oscalkit validate -s oscal-core.json fedramp-annotated-wrt-SP800-53catalog.json

Developing

oscalkit is developed with Go (1.11+). If you have Docker installed, the included Makefile can be used to run unit tests and compile the application for Linux, macOS and Windows. Otherwise, the native Go toolchain can be used.

Dependency management

Dependencies are managed with Go 1.11 Modules. The vendor/ folder containing the dependencies is checked in with the source for backwards compatibility with previous versions of Go. When using Go 1.11 with GO111MODULE=on, you can verify the dependencies as follows:

$ go mod verify

Compile

You can use the included Makefile to generate binaries for your OS as follows (requires Docker):

Compile for Linux:

$ GOOS=linux GOARCH=amd64 make

Compile for macOS:

$ GOOS=darwin GOARCH=amd64 make

Compile for Windows:

$ GOOS=windows GOARCH=amd64 make

Website and documentation

Both the website and corresponding documentation are being developed in docs/. The content is developed using the Hugo framework. The static content is generated and published in docs/public, which is a separate Git worktree that is tied to the gh-pages branch and publicly accessible via https://docker.github.io/oscalkit.

The GoDoc for the SDK can be found here.

Releasing

The GoReleaser tool is used to publish oscalkit to GitHub Releases. The following release artifacts are currently supported:

• OSX binary
• Linux binary
• Windows binary
• Docker Image
• Homebrew recipe

oscalkit's Issues

Profile processing for catalog code generation

Support processing of profiles for parameter insertions and add, merge, modify, alter and remove to generate catalogs with profile data.

Fix OSCAL profile metaschema discrepancy in code generation

Due to usnistgov/OSCAL#287, the profile types that we're generating from the metaschema don't match those of the actual profile examples. We should implement a temporary workaround in the metaschema generation process until this is addressed in the upstream.

CircleCI test failing due to test utility

Description

Getting this error in CircleCI: https://circleci.com/gh/docker/oscalkit/78

test/main.go:8:2: no Go files in /go/src/github.com/opencontrol/oscalkit
make: *** [test] Error 1
Makefile:32: recipe for target 'test' failed
Exited with code 2 

Reason

This is due to the fact that we have a test utility test/main.go which requires some files to be generated before it could even compile successfully.

Solution

One way to solve this issue is to rename test folder to test_util and then exclude it in test target by replacing line 32 in MakeFile from:

@go test -race -coverprofile=coverage.txt -covermode=atomic -v $(shell go list ./... | grep -v /vendor/)

to

@go test -race -coverprofile=coverage.txt -covermode=atomic -v $(shell go list ./... | grep -v '/vendor/\|/test_util')

Fix imports

The Go imports need to be fixed since this repo was moved to the Docker GitHub org.

Code generation from catalogs and profiles

Support Go code generation from catalogs and profiles

Failing test on fedramp profile

Tests are failing on Fedramp profile with a panic: runtime error: index out of range
The probable culprit for modifying the part incorrectly is:
https://github.com/docker/oscalkit/blob/4d3ebce31f13655d9ab0820a5be6b876d4a46e4e/generator/manipulation.go#L100

cc: @minhaj10p @anweiss

[Ongoing] Update types per latest data model

Update built-in types to match the latest data model.

Include DO NOT EDIT header to type code files generated from metaschemas

The following note should be included at the top of all type code files which were generated from the metaschemas:

// Code generated by go generate; DO NOT EDIT.

Supports #1.

Evaluate GitHub Actions for release management activities

Evaluate GitHub Actions for release management activities. Would eventually replace CircleCI if deemed appropriate.

Update documentation

An initial website has been pushed per #14. All of the usage docs can be found in docs/content/docs. This content needs to be updated.

Failing to marshal Href Attributes

oscalkit failing to convert NIST80053 catalog rev4 json file
Link:
https://github.com/usnistgov/OSCAL/blob/master/content/nist.gov/SP800-53/rev4/NIST_SP-800-53_rev4_declarations.json
Error:
Its unable to marshal all the Href attributes

type Href struct {
	*url.URL
}

Here is a PR for the fix #9

Evaluate TUF for artifact signing

Evaluate the use of The Update Framework (TUF) for signing of OSCAL artifacts.

Related to #4.

Implement helper methods to reduce looping and struct accessor unwieldiness

When creating OSCAL structs from various data sources, there can often be a number of unwieldy looping and struct accessor lines. Even creating a simple control with a single string narrative requires an unnecessary number of lines of code. Instead, we should provide helper methods that make it easy to go from simple strings to various OSCAL components (e.g. Parts, Prose, etc). So for example:

func NewPart(narrative string) catalog.Part {
	return catalog.Part{
		Prose: &catalog.Prose{
			P: []catalog.P{
				{Raw: narrative},
			},
		},
	}
}

Helpers should be created for reading nested information as well so as to avoid unwieldy iteration constructs.

Support XMLDSIG

Add support for XML signing via XMLDSIG

Refactor label type in implementation

Let's go ahead and make type Label struct{} a map[string]string for now.

type Label map[string]string

Assigning @minhaj10p.

[Ongoing] Unit testing

Tracking issue for missing unit tests and test data

Fix failing XML validation tests

The following tests are failing:

--- FAIL: TestXMLValidate (0.00s)
    --- FAIL: TestXMLValidate/successful-validation (0.00s)
        validator_test.go:159: xmlValidator.Validate() error = open ../testdata/fedramp-simple-profile.xml: no such file or directory, wantErr false
    --- PASS: TestXMLValidate/failed-validation (0.00s)

Looks like we need to include some test data for validation against the upstream XSDs

Supports #10

Generate XML and JSON catalogs from processed profiles

In parallel to #12, we should also include the ability to generate XML and JSON catalogs from profiles that have been processed. This will require a somewhat of a refactor of how we parse prose since we'll need to support substituting <insert> elements in catalogs with the contents of <set-param> elements in profiles as part of this issue.

@minhaj10p going to assign you to this one since you've already implemented most of the processing code via #52. I'm thinking we can create two subcommands off of generate to support this. oscalkit generate code would be used for generating catalog source code as we're already doing via #12. oscalkit generate catalogs would be used for addressing this issue. Thoughts?

Generated implementation struct needs to include component definition ID

The oscalkit generated implementation struct does not have componenetDefinition ID as of now, which is essential for patching implementation against a certain component.

Guidance type should rather be an []string instead of string

The guidance present in fedramp baseline implementsProfile[].Parameters[] are an array of multiple p tag.
IMHO, guidance in Parameter struct should be of type []string rather than string.
https://github.com/docker/oscalkit/blob/42bb04d0ffeec2f1979f895ebfc8ea9819dc9223/types/oscal/implementation/implementation.go#L39

Create a new release for 0.2.0

Tracking for the 0.2.0 release.

Support JWS

Add JSON signing support via JWS

Evaluate go modules for Go 1.12

With experimental support for Go modules introduced in Go 1.11 and expected support to be finalized in Go 1.12, we should evaluate the use of Go modules for dependency management.

Oscalkit failing to parse profile where an https import, imports a relative path.

Related to: #57

When executing oscalkit generate code and processing a profile that has been imported using an https:// link, if that profile internally imports a profile/catalog relatively, the path is resolved incorrectly by appending the current system relative path to the https:// path of the parent profile that has imported the catalog/profile. The problem lies in the parentPath handing in:
https://github.com/docker/oscalkit/blob/c5a92172971fa7ea1ccfdf7b08f67ed5bae53683/generator/profile.go#L44

cc: @minhaj10p @anweiss

Fix failing test in test/main.go

Test fails for test/main.go:

test/main.go:13:14: undefined: oscalkit.ApplicableControls

@farhan-khan30 can you address this?

Supports #10.

Website

Develop basic GitHub Pages site for oscalkit. Some static site generator and them options below:

Hugo:

• https://themes.gohugo.io/kube/
• https://themes.gohugo.io/github-project-landing-page/
• https://themes.gohugo.io/evie-hugo/
• https://cssninja.io/themes/fresh

Jekyll:

• https://github.com/18F/uswds-jekyll
• https://learn.cloudcannon.com/templates/edition/

GitBook:

• https://github.com/GitbookIO/theme-default

Generate command doesn't properly handle relative profile imports

When executing oscalkit generate and processing profile import's, relative paths are not being taken into account properly. We're incorrectly taking the path relative to the directory from which the oscalkit CLI is being run. However, we instead should be taking the path relative to the directory in which the profile is stored. The implementation of the following function looks to be the culprit:

https://github.com/docker/oscalkit/blob/d66e30b99ca1454f7dfd40ba5982bd7ed8be8ef2/generator/reader.go#L49

So for example, if I execute oscalkit generate -p /someDir/profile.xml and profile.xml has a relative import like ../catalog.xml, then the directory that needs to be searched is /someDir and not the directory in which the oscalkit generate command is being run.

This supports #18 and #12

Take package name as input in catalog/control code generation

Currently package name is fixed i.e. oscalkit when we generate a catalog file. The command should also take package name as input to seamlessly integrate it in any project.

Address failing JSON unit tests

Given that the upstream OSCAL JSON models are out of sync with both the XML models and examples, the JSON unit tests are failing. These will need to be addressed.

Supports #10