govready/compliancekbs

Compliance Knowledge Base Service for Security Controls Compliance Server

NOTE: ARE YOU LOOKING FOR GRC TOOL? SEE REPO: GOVREADY-Q

govready

An accreditation-helper toolkit to make FISMA easier.

NOTE: THIS REPO IS NOT CURRENTLY BEING ACTIVELY MAINTAINED. THAT MAY CHANGE IN THE FUTURE.

Vision

The GovReady vision is to make FISMA easier for innovators by

• making compliance part of Agile/DevOps
• sharing compliance progress data among Dev, Ops, Sec, and Mgt
• making baseline development collaborative
• providing trusted SCAP content with open source friendly licenses
• embracing compliance as a practice distinct from security

Product

Our first product is "govready", a toolkit for running FISMA scans and managing results with a git-like feel.

Govready uses (and is a contributor to) the NIST Certified SCAP 1.2 toolkit OpenSCAP and Scap-Security-Guide.

Our design goal is to make scanning easier and more collaborative regardless of your knowledge of FISMA.

License

Copyright 2013, 2014 Greg Elin and GovReady. All Rights Reserved.

License: GPL 3.0

Project Status

Govready is under heavy development and is pre-release. The current version is 0.6.x.

We recommend only using govready currently on non-production virtual machines.

Feedback via GitHub issues is appreciated!

The govready toolkit is funded by a generous grant from the John S and James L Knight Foundation

Get Started

Below are several quickstarts. Use the quickstart for your preferred OS.

( Need a vm to test GovReady? Try: https://github.com/GovReady/testmachines )

RedHat 7 quickstart (64 bit)

# Update nss to pre-emptively avoid problems with EPEL certificates (https://github.com/GovReady/govready/issues/64)
# Alternate if epel installed: sudo yum --disablerepo="epel" update nss -y
sudo yum clean all
sudo yum update nss -y

# Install some useful tools to take full advantage of SCAP-Security-Guide project
# scap-scanner is OpenSCAP certified SCAP scanner
# scap-security-guide is the SCAP Security Guide project
# ansible is the ansible infrastructure as code project
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo yum -y install epel-release-latest-7.noarch.rpm
sudo yum install ansible scap-scanner scap-security-guide -y

# Install govready using curl. govready will install OpenSCAP and SCAP-Security-Content
# curl -Lk https://raw.githubusercontent.com/GovReady/govready/master/install.sh | sudo bash
curl -Lk io.govready.org/install | sudo bash

# Switch to root so scanner can run all tests properly
# It's OK. You are using a non-production vm, right?
# (On AWS, may be necessary to run `sudo su -`)
su - 

# Change back to prefered working directory if dropped into "/"
cd /home/myuser

# Create a directory and cd into it
mkdir myfisma
cd myfisma

# Initialize the directory
govready init

# List the scan profiles
govready profiles

# Choose the DISA STIG for RHEL 7 profile
govready profile stig-rhel7-disa

# Run a scan (e.g. before)
# This command runs a multi-parameter OpenSCAP (oscap) command based
# on settings in `GovReadyfile`
govready scan

# Run fix script generated by most recent OpenSCAP (oscap) scan
# This utility generates all the parameters to produce a remediation script and command.
govready fix

# Run an post-remeditation scan (e.g. "after")
# govready utility will track your two most recent scans for easy after / before comparison
govready scan

# Compare before and after scans. Compares rules with 'pass' results in most recent scan to results in second most recent scan.
govready compare

# Compare before and after scans to see if anything fails passed in second most recent scan.
govready compare fail

# List results
ls -l scans

# Install lynx commandline browser to view results from prompt
yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional
yum install lynx

# View repoort from most recent scan
lynx scans/stig-rhel7-disa/[datestamp]-results.html

# Information and evaluation of individual rule (rule must be listed in results.xml file)
govready rule configure_auditd_num_logs

# See available profiles (e.g., baselines)
govready profiles

# Run a scan for a different profile (e.g., baseline)
govready scan usgcb-rhel6-server

# Run an autogenerated fix script from available remediations in the SCAP content
# Example - your file name may differ
bash scan/usgcb-rhel6-server-fix-0822-1552.sh

Centos 6 quickstart (64 bit)

Note: CentOS is NOT RHEL without subscription. There is a general assumption that SSG tests for RHEL work on CentOS, but that is not guaranteed.

# Install govready using curl. govready will install OpenSCAP and SCAP-Security-Content
curl -Lk io.govready.org/install | sudo bash

# Switch to root so scanner can run all tests properly
su - 

# Create a directory and cd into it
mkdir myfisma
cd myfisma

# Initialize the directory
govready init

# Import CentOS cpe-dictionary.xml and cpe-oval.xml SCAP data into local scap/content directory
govready import https://raw.githubusercontent.com/GovReady/govready/xplatform/templates/ssg-centos6-cpe-dictionary.xml
govready import https://raw.githubusercontent.com/GovReady/govready/xplatform/templates/ssg-centos6-cpe-oval.xml

# Update GovReadyfile using sed command (or update the CPE line manually using a text editor)
sed -i 's:^CPE.*:CPE = scap/content/ssg-centos6-cpe-dictionary.xml:' GovReadyfile

# Run a scan
govready scan

# List results
ls -l scans

# View repoort from most recent scan
lynx scans/results.html

# Run fix script generated by most recent OpenSCAP scan
govready fix

# Compare before and after scans. Compares rules with 'pass' results in most recent scan to results in second most recent scan.
govready compare

# Compare before and after scans to see if anything fails passed in second most recent scan.
govready compare fail

# Information and evaluation of individual rule (rule must be listed in results.xml file)
govready rule configure_auditd_num_logs

# See available profiles (e.g., baselines)
govready profiles

# Run a scan for a different profile (e.g., baseline)
govready scan usgcb-rhel6-server

# Run an autogenerated fix script from available remediations in the SCAP content
# Example - your file name may differ
bash scan/usgcb-rhel6-server-fix-0822-1552.sh

Ubuntu 12 and 14 quick start (64 bit)

# Install govready using curl. govready will install OpenSCAP and SCAP-Security-Content
curl -Lk io.govready.org/install | sudo bash

# Switch to root so scanner can run all tests properly
su - 

# Create a directory and cd into it
mkdir myfisma
cd myfisma

# Initialize the directory
govready init

# Run an oscap command just to see things fail because we have no SCAP content installed for Ubuntu
oscap xccdf eval --profile test --results scans/test-results-0822-1319.xml --report scans/test-results-0822-1319.html  /usr/share/xml/scap/ssg/content/ssg-rhel6-xccdf.xml

# Sorry - this is all you can do on Ubuntu at the moment. :-(
# Fork the code and help us include Ubuntu and Debi

Additional Quickstarts available in the repo. See "Quickstart-platform.md"

Remote Scanning (integration with oscap-ssh script)

The recently released oscap-ssh enables Scanning Remote Machines with OpenSCAP. As a user-friendly frontend to the OpenSCAP tools, GovReady has been enhanced to make use of this exciting new feature. There are two ways to make use of this new feature (which may be used separately or together):

1. Update four variables in the (configuration) GovReadyfile:

# All four vars must be set 'sudo OSCAP_USER@OSCAP_HOST OSCAP_PORT' for remote scanning.
# Note that openscap-scanner ('oscap') must be installed on the remote server.
OSCAP_SUDO = sudo
OSCAP_USER = oscap-user
OSCAP_HOST = example.com
OSCAP_PORT = 22

The openscap scanner is best run by root, but enabling direct root SSH access is a security risk. A non-privileged user such as oscap-user can be enabled to run only the oscap binary as root (when the sudo flag is given) by updating the remote machine's sudoers file or adding a file like /etc/sudoers.d/99-oscap-user:

# allow oscap-user to run openscap scanner
Defaults!/bin/oscap !requiretty
oscap-user ALL=(root) NOPASSWD: /bin/oscap

2. Create or override GovReadyfile values with GOVREADY_* environment variables:

This will scan the RHEL 7 machine badwolf.example.com via port 2222:

export GOVREADY_OSCAP_HOST=badwolf.example.com
export GOVREADY_OSCAP_PORT=2222
export GOVREADY_XCCDF=ssg-rhel7-ds.xml
govready scan

Notes and Caveats on remote scanning

• All values in the GovReadyfile can be overridden with GOVREADY_* environment variables. This facilitates integration with many site management tools such as Ansible, Chef and Puppet.
• As @mpreisler notes, currently oscap-ssh must be run as root, while remote ssh access is generally considered a security hole to be remediated. A future release will resolve this.
• OVAL and CPE files can not be separately named on the command line. Rather, use a "datastream" object as the XCCDF file (which can contain OVAL and CPE definitions within it).

Uninstall govready

Using curl

# Uninstall
curl -Lk https://raw.githubusercontent.com/GovReady/govready/master/install.sh | sudo UNINSTALL=1 bash

Install development branches

# Install branch other than master
curl -Lk https://raw.githubusercontent.com/GovReady/govready/master/install.sh | sudo BRANCH=branch_name bash

# Use an installer from a different branch
curl -Lk https://raw.githubusercontent.com/GovReady/govready/branch_name/install.sh | sudo BRANCH=branch_name bash

Testmachines

Use https://github.com/GovReady/testmachines for virtual machines to test GovReady.

compliancekbs's Issues

Add SP Documents - Set #2

Add the following 800 series documents
Add the following 800 series documents

• 47
• 44
• 41
• 36
• 34
• 33
• 32
• 296
• 25
• 27
• 18
• 16 draft (2014)
• 16 (1998)
• 14
• 12

Add SP Documents - Set #1

Add the following NIST 800 series publications

• 125-B
• 117
• 115
• 103
• 94
• 84
• 61
• 60
• 55
• 53A
• 51
• 50

Display thumbnail placeholder graphic when no thumbnail available

Adding proprietary documents via mounted drive or service

How do we make meta data files regarding proprietary documents as part of the service?

Do we set up a second server that provides information based on authentication? Do we mount secure information (then how do we only read for a particular user)? Do we require organization to set up their own version of kbs?

Deploy code to Digital Ocean server

Find document citation for system interconnect. See CA-3 System Interconnections.  https://web.nvd.nist.gov/view/800-53/Rev4/control?controlName=CA-3

Find document that defines system interconnect.

Add NIST 800-128

Title: Guide for Security-Focused Configuration Management of Information Systems

Especially noteworthy is Appendix I, page I-5, I-6 "Attachment 1 Security Impact Worksheet". These are questions by Security family to ask oneself about the impact of a change.

Terms: worksheet, configuration management, security impact worksheet

Track Fips-140-1 140-2 certification page

It would be good to add this page to database though I am not sure how best to label it. This page also changes every year.

• http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140val-all.htm#2441

Update NIST SP documents to have owner "NIST"

@MCupp93 Can you update all the NIST documents to have "NIST" as the owner instead of "~"?

Thanks.

How do we test document data?

@JoshData @MCupp93:

Matthew has been adding more NIST SP 800 series document to KBS. It's making me wonder how we might go about doing regression tests on the documents and the document meta data YAML files.

Is there a strategy you use @JoshData with GovTrack to test document integrity over time?

Add to Knowledge Base Dept of Education Security Requirements for Contractors Doing Business with the Department of Education

Add these documents to knowledge base from this page: http://www2.ed.gov/fund/contract/about/bsp.html

• Information Assurance Security Policy (OCIO-01) [download files] MS WORD (944K)
• Information Technology Security Certification and Accreditation Procedures (OCIO-05) [download files] MS WORD (830K)
• General Support System and Major Application Inventory Procedures (OCIO-09) [download files] MS WORD (856K)
• Information Technology Security Configuration Management Planning Procedures (OCIO-11) [download files] MS WORD (896K)
• Information Security Incident Response and Reporting Procedures (OCIO-14) [download files] MS WORD (841K)
• Protection of Sensitive But Unclassified Information (OCIO-15) [download files] MS WORD (259K)
• Personal Use of Government Equipment (OCIO: 1-104) [download files] MS WORD (124K)
• Lifecycle Management (LCM) Framework (OCIO: 1-106) [download files] PDF (652K)
• Procuring Electronic and Information Technology (EIT) in Conformance with Section 508 of the Rehabilitation Act of 1973 (OCIO: 3-105) [download files] MS WORD (983K)
• Contractor Employee Personnel Security Screenings (OM: 5-101) [download files] MS WORD (160K)
• Department of Education IT Security Awareness Training 2011 [download files] MS WORD (688K)
• External Breach Notification Policy and Plan (OM:6-107) [download files] PDF (496K)

Find document citation for internal system connections.

Get understanding of CA-9 - INTERNAL SYSTEM CONNECTIONS - definition of a "internal connections"

Generate page of all currently defined look up terms

Add SP Documents - Set #3

Add the following Draft documents from NIST and IRs:
http://csrc.nist.gov/publications/PubsDrafts.html

• draft 160
• draft 180
• IR 8011
• IR 8062
• IR 8058
• IR 7621
• 94 revision 2 draft
• IR 7799
• DRAFT whitepaper Project Description--Securing Non-Credit Card, Sensitive...
• DRAFT whiterpaper Project Description--Multifactor Authentication for e-Commerce: Online Authentication for the Retail Sector
• pre-draft 800-63-3

Example identifiers:

• nist-sp-800-160-draft-v3
• nist-ir-7799

punkt install instruction from README fails

Gregs-MacBook-Pro:compliancekbs greg$ python3 -m nltk.downloader punkt
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py:125: RuntimeWarning: 'nltk.downloader' found in sys.modules after import of package 'nltk', but prior to execution of 'nltk.downloader'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
[nltk_data] Error loading punkt: <urlopen error [SSL:
[nltk_data]     CERTIFICATE_VERIFY_FAILED] certificate verify failed
[nltk_data]     (_ssl.c:777)>

Render thumbnail for 18F policies that exist in Markdown

Support RESTful query with JSON responses

Ability to query the knowledge base via an API with json responses

Add Dept of Energy AO training document to KBS

• http://energy.gov/sites/prod/files/2014/06/f17/AO_Core_Competency_Training_01.pdf

Ability to ask KBS for excerpts of a document by lookup term from a small vocabulary of terms and get back an array of excerpts from the document

Add document

https://www.whitehouse.gov/sites/default/files/omb/memoranda/2016/m_16_21.pdf

Add DOD documents

• http://www.dcma.mil/policy/815/DCMA-INST-815.pdf
• http://www.dtic.mil/whs/directives/corres/pdf/850001_2014.pdf
• http://www.dtic.mil/whs/directives/corres/pdf/852002p.pdf

Creating new YAML file should not write over existing YAML file.

Do not overwrite existing YAML file.

Further alter display of question when you cannot answer it?

Although the Submit button is grayed out to indicate inactive, the cursor is still being placed inside the answer box on question type text when visiting a question as a guest. See: https://q.govready.com/tasks/2/animal-compliance?q=q1#discussion

Does it make sense for cursor to be kept out of the answer box when you are not an editor? Should there be a message if you try to answer question?

OMB Memos

• OMB Memo M-04-04, EAuthentication Guidance for Federal Agencies http://www.whitehouse.gov/sites/default/files/omb/memoranda/fy04/m04-04.pdf

Generate thumbnails for HTML documents

Add DoD Guidance documents to database

Add these documents to knowledge base.

• Department of Defense
  INSTRUCTION 8530.01 http://www.dtic.mil/whs/directives/corres/pdf/853001p.pdf - referred to as the 8530.01.

Add 800-64 document to repository

• http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-64r2.pdf title: Security Considerations in the System Development Life Cycle

CCI data

http://iase.disa.mil/stigs/cci/Pages/index.aspx

"The Control Correlation Identifier (CCI) provides a standard identifier and description for each of the singular, actionable statements that comprise an IA control or IA best practice. CCI bridges the gap between high-level policy expressions and low-level technical implementations. CCI allows a security requirement that is expressed in a high-level policy framework to be decomposed and explicitly associated with the low-level security setting(s) that must be assessed to determine compliance with the objectives of that specific security control. This ability to trace security requirements from their origin (e.g., regulations, IA frameworks) to their low-level implementation allows organizations to readily demonstrate compliance to multiple IA compliance frameworks. CCI also provides a means to objectively rollup and compare related compliance assessment results across disparate technologies."

text analysis needs to handle html documents

Initially, text analysis could skip html documents. But some authoritative sources might be HTML and not PDFs.

Log requests in order to generate statistics on requests

Expectation is that sql-lite will initially be used.

PDF page numbering != citable page numbers

I'm putting PDF page numbers (always starts with 1) into the YAML files, but the page number you would use in a human-readable citation (the number actually printed on a page) is typically different.

List documents

Create a page that lists the documents.

Site keeps crashing, need to figure out why

Site keeps crashing every few weeks. Need to resolve.

Are thumbnail assets from DocumentCloud persistent?

Are they persistent by account, document, etc.

AWS documents to be added to KBS

Here are some AWS documents that might be worth adding:

• https://d0.awsstatic.com/whitepapers/aws-architecture-and-security-recommendations-for-fedramp-compliance.pdf - terms should include common controls, shared responsibilities, AWS
• https://d0.awsstatic.com/whitepapers/compliance/AWS_DOD_CSM_Reference_Architecture.pdf
• https://d0.awsstatic.com/whitepapers/compliance/AWS_DOD_CSM_Reference_Architecture.pdf
• http://iase.disa.mil/cloud_security/Documents/u-cloud_computing_srg_v1r1_final.pdf
• https://aws.amazon.com/compliance/dod/
• https://www.fedramp.gov/marketplace/compliant-systems/amazon-web-services-aws-eastwest-us-public-cloud/
• https://aws.amazon.com/compliance/fedramp/
• https://d0.awsstatic.com/whitepapers/compliance/AWS_Risk_and_Compliance_Whitepaper.pdf
• http://d0.awsstatic.com/whitepapers/Security/AWS%20Security%20Whitepaper.pdf
• http://d0.awsstatic.com/whitepapers/Security/AWS%20Security%20Whitepaper.pdf
• https://d0.awsstatic.com/whitepapers/compliance/aws-architecture-and-security-recommendations-for-fedramp-compliance.pdf
• https://d0.awsstatic.com/whitepapers/compliance/AWS_Australian_Signals_Directorate_Cloud_Computing_Security_Considerations_Oct14.pdf
• https://d0.awsstatic.com/whitepapers/compliance/Using_AWS_in_the_context_of_Australian_Privacy_Considerations.pdf

"unauthorized access" is matching roles as well as document

The term "unauthorized access" matched roles as well as documents.

Is there a way to upload HTML documents to DocumentCloud?

Experimenting with additional data

Let's pick a half dozen documents and add some additional information:

• total page numbers (look up in DocumentCloud)
• number of appendices (look at table of content in DocumentCloud)
• year of publication
• authors

authors: | 
   - name: Kelley Dempsey
     organization: National Institute of Standards and Technology (NIST)
   - name: Dr. George Moore
     organization: Applied Physics Laboratory at Johns Hopkins University
   - name: Nadya Bartol
     organization: ~

• version number (if exists, otherwise use: version: ~)

Copy and paste YAML into http://www.yamllint.com to make sure YAML format is valid.

Which documents to do? Pick about half dozen documents that either all share a single term in the title, or two groups of docs that share terms and gather the additional meta data for those docs. We will need to add some code to display the data on search result, but should be interesting.