cyberperspectives/sagacity

Security Assessment Data Management and Analysis Tool

Cyber Perspectives Sagacity

https://www.cyberperspectives.com

Sagacity is a vulnerability assessment and compliance data management tool designed to make security testing more efficient, effective and complete.

Security assessments, especially those done for DoD and Federal organizations, produce tremendous amounts of scan and compliance data that security engineers must sort through and deconflict, identify untested requirements, and somehow analyze to communicate risk to their employers. Sagacity, originally written to support a government customer, was designed to fill that need.

What if an organization could turn massive amounts of irreconcilable vulnerability scan data into true knowledge and insight about their networks? They would be able to make wise decisions resulting in cost-effective actions to improve their security with the best return on investment.

Keen insight. Sound judgment. Wise decisions. Sagacity.

Features

• ingest data from Nessus vulnerability and compliance scans, SCC, nmap, MBSA and other automated tools
• correlate data to applicable STIG and IAVM checklists and deconflict data from multiple scan sources
• identify required manual STIG checks for a complete compliance assessment
• provide an efficient spreadsheet format for conducting manual tests and reporting compliance data
• track assessed hosts, applicable STIGs, OS's, installed software, missing patches, network services and more
• security assessment task tracking to ensure a complete and thorough test
• statistical analysis of compliance rates, assessment completeness, Cat I, II, III findings and more.

Requirements

Windows:

• Processor: 2.0 Ghz+ (recommend Intel i5+)
• Memory: 8GB
• Hard Drive: 50GB free (SSD recommended)

Linux:

• Processor: 2.0 Ghz+ (recommend Intel i5+)
• Memory: 4GB
• Hard Drive: 50GB (SSD recommended)

Software Requirements

Sagacity has the following software requirements. The versions listed are the minimum required for operation. For PHP, we recommend the closest version you can get to the one listed, further versions may deprecate features before we have the chance to update the code.

Please note: Sagacity is not fully compatible with XAMPP 7.3.x or PHP 7.3. Please use XAMPP 7.2 / PHP 7.2 instead.

• PHP 7.2
• MySQL 5.7+ or MariaDB 10+
• Apache 2.4+

For Windows, you can install XAMPP 7.2.x from https://www.apachefriends.org/download.html

Installation

See README.pdf for complete installation instructions.

sagacity's Issues

background_results.php 241 unlink, no such file or directory

Occasionally, we get the following error when ingesting assessment results. I'm not sure the exact conditions that cause it, but I notice it once or twice during every test.

background_results.php, line 321. unlink parse_config.ini - no such file or directory.

The code should check first to see if the file is already gone before attempting deletion. As this happens at the end of the run, it doesn't seem to have any impact on ingesting scan results.

.ckl export - check contents in <finding details> instead of notes

When exporting .ckl files, the resultant files notes field tag contains the check contents, and not the analyst notes.

I exported an eChecklist, changed some note fields, then imported it and re-exported, resulting in changed notes.

I checked the database, and the notes fields from the imported eChecklist contain the Check Contents. I suspect the eChecklist parser or somewhere in the import chain. Other notes from Nessus findings seem fine.

#10 Retain expansion state on Ops page after navigating away

Matt - It would be nice if we could edit a target or import new data and not have to re-expand all our categories.

#62 #39-3 Separate Task Status Page

Task Management should be part of the pro release.

Ticket #39 - 3
Jeff - I don't like how the tasks are laid out - a big red blob in the middle of the page. Maybe select buttons. We have to find a way to narrow down those columns - or maybe move them to a new page entirely - operations tasks, which shows host progress as well as analyst assignments, NR counts, etc. Sorry to say, we haven't used that task tracking much anyway. I vote for moving it out and maybe adding a task page later.

Ryan - I like the idea of creating a separate task status page. It would free up a lot of real estate and I don't think it would be very much work to create. Should the targets be separated by category here as well? I think they should.

Jeff - Task Status Page (1.31) Yes, use the categories. We should have the same look and feel as much as possible. Another idea would be to have a toggle between tasks and stats on the ops page, but I worry about performance. I think better would be a mostly clone of the page, but with tasks instead.

Ryan - I agree.

update_db.php switch from STIG library to a-z listing for download

Use the A-Z listing page as the default for the STIG downloads (like we do it for the sunset STIGs), and always include the sunset STIGs when downloading the STIGS. Fall back to the STIG library if it fails.

The trick on this one may be that it's broken into two pages, and it's actually a sub-page that you need to parse to get the actual file links.

#411 eChecklist RMF Controls listed as CCI

On the eChecklist, the IA controls are listed as CCI's and not RMF controls. Did the RMF table not get populated, or were changes made?

#169 Checklist Page

Similar to the ops page, which organizes the hosts by Category, have a Checklist page that has the associated hosts for each Checklist. It would function almost exactly like the ops page, including the ability to export E-checklists (this time, only for that STIG checklist), percentages of NF, O, NA, NR, the ability to copy or remove hosts (not move) into that checklist category (which adds the checklist to that host)

This per checklist view would be very handy for doing manual checks, since you can have analysts doing the same checklist, even across OS categories (like the IE STIG across workstations and servers simultaneously)

#180 Target Search Actions

Ideas for actions on the target search page. What might I like to do?

Export as eChecklist or .CKL (as though they were in a category)
Bullk edit (add/remove checklist, etc.)
Delete hosts
Move to category
etc. (we can brainstorm more)

#176 Select hosts for manual testing

New Feature - target specific hosts for manual testing (under random or selective sampling) and have the ability to only export E-Checklists for those hosts to perform manual testing on. This would save time having to delete columns for hosts not selected for manual testing.

Use the Ops page check boxes to select hosts, then have the Export-eChecklists button only export those hosts.

disable automatic MS FW STIG

We are seeing more systems where they are using 3rd party software to provide host based firewall.

Disable the automatic setting of the MS FW STIG.

In the future maybe we could have some checkboxes when you create the system to set some global variables for what things are true across the system.

orphaned STIGS

The dot Net STIG is assigned to orphan.jpg instead of MS .net.jpg

The FOUO HBSS/ePO STIGS are assigned to orphan.jpg instead of HBSS.jpg

Assign OS, Checklists based on eChecklist import

When importing a new host from an eChecklist, assign checklists based on the Checklist field (B9) and OS based on the OS/SW Version (G4).

ST&E Ops Page Performance

In the latest 1.3.3 dev release (11 Sep 18), the ST&E Ops page performance has taken a huge hit, to the point where it is hardly usable. Performance starts out fine, but as you add more hosts and more results, it gets exponentially slower.

There are no error messages. We had one system with about 40 hosts taking literally minutes to load the Ops page or expand a category. We have reproduced on several systems.

eChecklist Notes repetition

eChecklist notes should have code to squash repeated content. For example, putting "This is a note" in the eChecklist notes column, importing the eChecklist and then re-exporting - you get one additional instance of the same phrase for each host. (The title of this bug was from two iterations)

The same goes for pulling in SCC data. We should never see (SCC) (SCC) (SCC)... - just (SCC).

This could be solved with a simple regex whenever you are adding to the notes - just check to see if the new content is found in the old content. If so, just leave the note alone.

#303 Assign Nessus Findings to S1-2 (CCI-001237)

The Nessus vulnerability findings in the Orphans tabs are assigned to AC-1.3, not SI-2. They need to be assigned to the correct one (CCI-001237 (SI-2)). We might need to look at some going to CM-6 (CCI-000366), since they are really configuration items and not patches. We can discuss. This applies to vulnerability scan findings, not compliance scan findings.

#228 ste_export_import.php Errors

The ability to export and import ST&E's is becoming more important as we do more tests and wipe out the database before the next test. We've had to recreate the ST&E to make changes/additions and/or re-export .ckl files or eChecklists.

Notice: Undefined index: notes in C:\xampp\www\data\ste_export_import.php on line 265

Notice: Undefined offset: 0 in C:\xampp\www\data\ste_export_import.php on line 271

Fatal error: Uncaught Error: Call to a member function get_ID() on null in C:\xampp\www\inc\database.inc:3979 Stack trace: #0 C:\xampp\www\data\ste_export_import.php(272): db->get_Finding(NULL, Object(stig)) #1 C:\xampp\www\data\ste_export_import.php(43): export_STE() #2 {main} thrown in C:\xampp\www\inc\database.inc on line 3979

Large eChecklist: MySQL server has gone away

Ingesting an eChecklist with more than 7 or 8 hosts produces the following error, and the eChecklist fails to load any data. We were able to delete host columns to 7 or less and get the eChecklist to import, so I think it's possibly a timeout issue, but we weren't able to successfully troubleshoot.

Also, instead of just printing the database error (which is often vague), we should also add a line with the query that bombed, since the code dynamically builds the queries. It would make it easier to troubleshoot.

PHP Warning: mysql::real_query(): Error reading results set's header in database.inc on line 280, followed by "MySQL server has gone away" message
Error reading result sets header in database.inc line 280

eChecklist status case insensitivity

It is possible, and in our experience, a common occurrence for security analysts to hand-enter the status as "not a finding" 'Not a finding" "Not A Finding", etc, which causes an error when the eChecklist is imported. This has happened on almost every test we've done, in spite of training...

This is because the data validation in Excel is case insensitive, so there is no way to guarantee this will never happen. The best solution is to make the status checks case insensitive in parse_echecklist.php.

Large eChecklist - Missing Hosts

Loading the Win_7_-50Hosts-eChecklist-1.xlsx - not all hosts are populated in Sagacity. The file has Paul-Lab, Win7-Master and Dup1 - Dup 48. The import only added up to Dup20, for a total of 22 hosts.

I created a new ST&E and retested with the same result.

#19 eChecklist by-Control Statistics

Add a tab to the eChecklist that provides statistics for each checklist tab based on affected RMF control familiies. This is very useful for analysis and reporting, and would be part of the Sagacity Pro release.

There is a sample with all the formulas needed in Google Drive, Sagacity, RMF called eChecklist-analysis tab. You would add this tab after the cover sheet tab, and add a section for each checklist tab. This would require that the IA Control column on the spreadsheets contain RMF controls rather than CCIs (which is another PRO feature).

I had to build this tab manually, and it was very time consuming. I think it could be done pretty quickly by Sagacity on eChecklist import, since you are just adding the columns to the analysis tab and the appropriate formulas. Excel does all the computation. (Although these numbers would be helpful for a procedural analysis page in Sagacity, too. - maybe 1.3.4)
eChecklist-analysisTab-15Aug17 (1).xlsx

Bulk Edit Improvements

1. On the bulk edit page, move the save button down by/between the STIG selection box and Toggle button. You always have to scroll up to save once you are done selecting hosts and checklists.

2. Provide a way to select multiple checklists from different searches. For example, if you have a couple domain controllers, you might want to add the two active directory STIGs, the Domain Controller version of the OS STIG and add an antivirus STIG as well. In the current system, you have to open the bulk edit page 3 times to do so.

Notify if tmp folder not empty

When opening the Import dialogue (through Import or Add Results), either have a notification that there are result files found, or better, display the current files in the folder in the box. That way you know what files are still there, in case they didn't import correctly before or if they got copied there automatically.

ajax.php / delete-file

The 'filename' parameter can be abused by a directory traversal attack.

`action=delete-file&filename=../../../windows/system32/test.txt

{"success":"Deleted file"}
`

Ops Page excessive load time

I loaded ST&E data from a live assessment with about 25 hosts - 17 Windows 7. Once all the SCC scans were loaded, I hit the recategorize button and it took over 5 minutes to reload the ops page. I tried again, just loading the ops page, and again it took over 5 minutes to load.

I think we need to plus up the number of SCC result files and do some more stress testing.

Merge Hosts Countable Error

When trying to merge two hosts:

Warning: count(): Parameter must be an array or an object that implements Countable in C:\xampp\www\inc\database.inc on line 12221

The merge looks like it worked, but the error still appears.

Class 'Logger' not found in helper.inc

Attempting to export an eChecklist

[05-Sep-2018 10:41:07 America/Denver] PHP Fatal error: Uncaught Error: Class 'Logger' not found in C:\xampp\www\inc\helper.inc:848
Stack trace:
#0 C:\xampp\www\ste\export.php(54): convert_log_level()
#1 {main}
thrown in C:\xampp\www\inc\helper.inc on line 848

#434 Don't count Orphan in Stats

The orphan checklist should be excluded for all counts on the stats page. The high number of "not reviewed" items throw the % assessed way off.

#48 Catalog Status

On the Catalog Management page, include catalog status information, possibly last date updated, record count, etc. for STIGS, NVD, CVEs, CWEs, Nessus Plugins, OpenVAS plugins, Exploit-DB, Metasploit, vendor advisories, etc. I think with some search and correlation tools, the Catalog could be a great research tool and a product in its own right.

Ryan - I have plans to work on a advanced search capability for catalog data. One consideration is that we should offer a way to download reference data from open sites to use on offline systems. Work best on a Linux system because I could thread off a script for each check that would run a wget.

#41 Add DB backup/restore capability

Add a database backup and restore button in the Management/Settings page. Should provide the ability to specify a backup location. Default filename should include hostname and date stamp.

#390 .ckl import does not assign OS or Checklists

When importing a .ckl file, Sagacity does not assign the OS or scanned checklist to the host.

Stig information is contained in:


<STIG_INFO>
<SI_DATA>
<SID_NAME>version</SID_NAME>
<SID_DATA>1</SID_DATA>
</SI_DATA>
<SI_DATA>
<SID_NAME>classification</SID_NAME>
<SID_DATA>UNCLASSIFIED</SID_DATA>
</SI_DATA>
<SI_DATA>
<SID_NAME>customname</SID_NAME>
</SI_DATA>
<SI_DATA>
<SID_NAME>stigid</SID_NAME>
<SID_DATA>Windows_10_STIG</SID_DATA>

There is no cpe data, but if an OS STIG is assessed, you could work backwards from that.

Merge Targets Invalid Argument foreach()

If you try to merge two targets, you get the following error and the targets do not merge:

Invalid argument supplied foreach() in database.inc line 11887
count() parameter must be an array or an object that implements Countable in database.inc line 12207

Configurable Refresh Frequency

Add a UPDATE_FREQ constant to config.inc to control how frequently AJAX requests run to update information (scan status and catalog import).

Catalog Management Improvements

On the catalog management page add buttons for download, offline and update

Download does an download only of all required files for offline installation, equivalent to update_db.php --cpe --nvd --stig --sunset --do, then zips up the resulting files and offers the .zip file to be saved wherever they choose. (Contains the tmp folder files and directories needed for offline install)

Offline opens a dialogue to upload the .zip file described above, then unzips it and does the equivalent of update_db.php --cpe --nvd --stig --sunset --po

Update would simply run update_db.php --cpe --nvd --stig --sunset

Github Release Version - cannot write password file

Trying to install the Github release version (download zip from github), the installation errors after the wizard when it goes to the ste page, saying it cannot find the password file.

We troubleshot the xampp/www permissions, knowing that Windows is stupid and tries to set files from the Internet as read-only. We tried both unchecking the "read-only" check box and setting everyone permissions to full control, but neither one worked.

"Cannot connect to the database because the password file does not exist" Also seeing
"[28-Aug-2018 15:25:06 America/Denver] PHP Warning: mkdir(): Invalid argument in C:\xampp\www\exec\installer.php on line 144"

We went back and pulled the release version from Sourceforge, and the installation worked fine.

Add Sunset check box to installation wizard

On the first page of the installation wizard, add a checkbox to automatically download and install the sunset STIGs in addition to the normal STIG library. This should include the option for offline installation and be tracked on the installation status bars. (You can reuse the STIG status bar.)

#421 NR Stats Not Calculating Correctly

Checking out the statistics page, it has some erroneous NR stats. We need to review how they are calculated. In the attached image, the Windows 7 hosts are 125% not reviewed. The max should be 100%.

You have O, NF, NA adding up to 100%, which is correct (count divided by total number of assessed PDIs = O + NF + NA). Not Reviewed should be count of NR divided by total PDIs in applicable STIGs (or easier would be divided by total findings)

ST&E Ops by Checklist

Provide a version of the ST&E Ops page that is sorted by checklist instead of by category. This would allow the export of eChecklists like antivirus, etc. that apply across multiple host categories. It would also make it easier to see which hosts are tied to each checklist (and if any are missing). This would be very helpful.

database.inc

When loading two simultaneous Nessus scans, I get the following errors.

Illegal string offset 'id' in database.inc lines:
11069
11082
11086 ('os_id')
11101
11111
111123
111133
111140
111148
111156
111163
111166
111163
111166
111168

chdir no such file or directory ajax.php line 55

I had 10 of these in my php_error_log. Not sure what caused them. It does a chdir(DOC_ROOT) and DOC_ROOT is defined in config.inc. I'll see if I can reproduce tomorrow.

[18-Sep-2018 19:13:11 America/Denver] PHP Warning: chdir(): No such file or directory (errno 2) in C:\xampp\www\ajax.php on line 55

#6 Add --CCI to update_db.php

Add the ability to download and parse CCIs to update_db.php (--cci flag)

https://iase.disa.mil/stigs/cci/pages/index.aspx

#56 Ability to remove STIGS from Catalog

As part of catalog management, a user should have the ability to remove selected STIGS (especially old STIGS) from the catalog.

#299 Set OS from SCC Results

Loading an SCC scan by itself does not set the OS or applied checklists. From the cdf:target-facts tag:

<cdf:fact name="urn:scap:fact:asset:identifier:os_name" type="string">Windows Server 2008 R2 Datacenter</cdf:fact>
<cdf:fact name="urn:scap:fact:asset:identifier:os_version" type="string">Windows Server 2008 R2 Datacenter</cdf:fact>
<cdf:fact name="urn:scap:fact:asset:identifier:processor" type="string">Intel(R) Core(TM) i7-4800MQ CPU @ 2.70GHz</cdf:fact>
<cdf:fact name="urn:scap:fact:asset:identifier:processor_architecture" type="string">Intel64 Family 6 Model 60 Stepping 3</cdf:fact>

Sagacity should set the OS and associated checklists based on the SCC contents. The next bug will be to apply the scanned checklist to the host.

ST&E Ops "Expand All Categories" Button

Several team members have asked on a few occasions for a button at the top of the page to "expand all categories" on the ops page. They understand the performance implications, but would like the option.

Delete Scan - default "Delete Targets"

When you delete a scan, the option for "Delete Targets" is highlighted by default. Instead "No" should be the default answer to prevent accidental deletion of targets.

#3 Ops Page Scans Icons - show files

Instead of showing findings count on the mouseover on the Scans Icons, list the file name(s) that were ingested. That way you could see if you were missing any scan data for that host - e.g. "where's my Office scan?"

#16 RMF Phase 1: Assessments

Phase 1: RMF Assessments (Pro Version)

1. Ingest tailored control set from spreadsheet
2. Allow editing of control results
   a. Perhaps a wizard to help with evaluation of Not Reviewed controls
   b. Provide ability (look up table?) to associate related controls - when you edit one, you get links to "related controls" to edit them
   c. Summarize related technical results (via CCI for RMF/STIGs) and have a button "Update based on technical findings" to mark some controls as Non-compliant due to tech findings. (Should this be automatic?) You could add some text to the results field like, "This control is non-compliant in part due to technical findings."
3. Export a spreadsheet with the final results.
4. Add a Procedural Statistics Page (I have an example spreadsheet) - we also have a very helpful statistics tab we can add to the eChecklist

#58 XCCDF XML Export

Export XCCDF XML formatted results - similar to the .ckl export. It would be similar to .ckl, with individual files per host per checklist. This would give us another point of compatibility with commonly used tools. When you code .ckl export, keep the idea of exporting in other formats in mind as it may affect the way you code it.

#233 Create blank eChecklists from Catalog Management

A nice feature would be to export a blank eChecklist for a specific checklist (probably purely manual) with n host columns. A good place for this might be on the Catalog Management Page. The current method requires you to create a category and add your hosts to it, assigning the desired STIGs. It's a little clunky if you don't have scan data, and sometimes you just want a blank eChecklist to work with.

.ckl exports won't open in STIG Viewer 2.7.1

.ckl files exported from Sagacity will not open properly in STIG Viewer 2.7.1, but they will open in the older 2.4.1. I went through and compared a Sagacity exported .ckl and one created by STIG Viewer 2.7.1, and the problem is with the finding status (<STATUS>) - it was set to No_Data. Apparently 2.4.1 can handle it (and convert it to Not_Reviewed), but 2.7.1 cannot.

Change the status tags in the output as follows:
No_Data --> Not_Reviewed
False_Positive --> NotAFinding
Exception --> Open
No_Data --> Not_Reviewed

The only other allowed status is Not_Applicable.

Also, we should add a comment line at the second line of the xml file:
<!--Cyber Perspectives Sagacity :: 1.3.3-->

CVE Progress Bar "No NVD CVEs in Database"

When the Sagacity starts downloading the CVEs, the CVE progress bar keeps going back to the "No NVD CVEs in Database" message, instead of staying on the "Downloading 2000 CVEs" or "Ingesting 2000 CVEs" This makes it look like it's not working, and since it's slow, can cause the user to start troubleshooting or questioning whether the install is working.

The progress should never return to the "No NVD CVE" message, but instead, increment the year being processed until complete.