csaf-tools/cvrf-csaf-converter

A CVRF CSAF Converter, taking care about OASIS specification.

CVRF-CSAF-Converter

• Introduction
• Getting started
• How to use CVRF-CSAF-converter
  • Usage as CLI tool
  • Config
• Specifications
• Developing CVRF-CSAF-converter
  • Developer Guide, Architecture and Technical Design
  • Security Considerations
• Contributing
• Project

Introduction

CVRF-CSAF-converter is a Python tool for converting CSAF CVRF 1.2 documents in CSAF 2.0 documents. It fulfills the conformance target CVRF CSAF converter.

Note: The project is currently still under development. Not all features have been implemented and therefore the conformance goal is not yet fulfilled.

Getting started

Ensure that you have installed python3 (version >= 3.6), python3-pip and python3-venv.

Assume your current directory is also avaliable at the environment variable $ROOT_DIR.

Check out the repository and navigate to the working directory.

   git clone https://github.com/csaf-tools/CVRF-CSAF-Converter.git
   cd CVRF-CSAF-Converter

Afterwards, create a virtual environment and install the package there:

   python3 -m venv venv
   . venv/bin/activate
   pip install .

Hint: If you would like to get the debugger running, try to install the code as follows: pip install -e .

How to use CVRF-CSAF-converter

Usage as CLI tool

To convert the CVRF CSAF 1.2 document $ROOT_DIR/CVRF-CSAF-Converter/examples/1.2/cvrf_example_a.xml use the following command:

   cvrf2csaf --input-file $ROOT_DIR/CVRF-CSAF-Converter/examples/1.2/cvrf_example_a.xml

The default output directory is ./, it can be set using --output-dir.

The output filename is derived from the CSAF field /document/tracking/id.

If there is an ERROR during conversion, the output file will not be written unless --force option is used.

The rest of the options can be shown with:

   cvrf2csaf -h

Config

The config file is installed inside the Python package. For the installation using venv, the config file is located in $PATH_TO_THE_VENV/lib/python3.X/site-packages/cvrf2csaf/config/config.yaml. When installing the PyPI package with pip (--user), the config file is located in $HOME/.local/lib/python3.X/site-packages/cvrf2csaf/config/config.yaml Converter options can be changed there, or overridden by command line arguments/options.

Specifications

We follow the official OASIS specifications in order to provide as much acceptance on the user base as possible.

• Specification CVRF 1.2

  • xsd common
  • xsd cvrf
  • xsd prod
  • xsd vuln

• Specification CSAF 2.0

  • JSON spec

Developing CVRF-CSAF-converter

Developer Guide, Architecture and Technical Design

The converter uses lxml.objectify to parse the whole input document.

Parsing and conversion of the following CSAF CVRF 1.2 XML elements are handled by separate section handlers. These section handlers process the elements recursively (converting also all their sub-elements). These elements are the direct children of the root XML element (<cvrfdoc>).

• DocumentTitle, DocumentType, DocumentDistribution, AggregateSeverity -> DocumentLeafElements handler
• DocumentPublisher -> DocumentPublisher handler
• DocumentTracking -> DocumentTracking handler
• DocumentNotes -> Notes handler
• DocumentReferences -> References handler
• Acknowledgments -> Acknowledgments handler
• ProductTree -> ProductTree handler
• Vulnerability -> Vulnerability handler

Vulnerability handler is reusing Acknowledgments, References and Notes handlers for its child elements.

Each of these section handlers is implemented by own class inheriting from SectionHandler class. This base class contains _process_mandatory_elements and _process_optional_elements methods which are parsing and converting mandatory/optional elements/attributes. Each subclass must implement these methods.

SectionHandler class holds error_occurred class variable. This variable is overwritten by any children class in case some error resulting in invalid output json happened. Depending on --force commandline parameter, the program either quits with error log message without producing output or produce invalid output and warning log message.

Complete conversion together with input and output validation against schemata is handled by the DocumentHandler class.

Security Considerations

These are the TOP OWASP categories of vulnerabilities which potentially affect the CVRF-CSAF-Converter. We are omitting those which do not apply (most of them), since it's a plain command-line tool (e.g. authentication failures)

A03:2021 – Injection

The XML input for the converter is strictly validated for CSAF CVRF 1.2. The converter rejects invalid inputs.

However, there is a known issue for inserting HTML with code/script, which could be executed by a CSAF consumer: Encode HTML in JSON output

A06:2021 - Vulnerable and Outdated Components

A CodeQL action is set in this project to spot vulnerabilities in 3rd party libraries. Especially the lxml library can be susceptible.

A4:2017 - XML External Entities (XXE)

XXE vulnerability present in releases <1.0.0rc2 was fixed in this commit

CVE report: https://nvd.nist.gov/vuln/detail/CVE-2022-27193

Contributing

Please refer to CONTRIBUTING.md for details about how to contribute to the development of CVRF-CSAF-converter.

Project

CVRF-CSAF-Converter is a project between Deutsche Telekom Security GmbH and the Federal Office for Information Security. It aims to provide a CVRF 1.x to CSAF 2.0 converter.

Realization is taking place 100% Open Source. The final delivery will be in Q1/2022.

cvrf-csaf-converter's Issues

Integrate turvallisuusneuvonta

As discussed here:

#14 (comment)

Integrate the mandatory tests into cvrf2csaf.py:

https://github.com/sthagen/turvallisuusneuvonta

Final Handover

• Full Access to Repository (Admin Rights)
• Outcome of #7 => Agree on input set for acceptance test
• Test files to be uploaded to the testing branch (@tschmidtb51 / @mfd2007)
• Communicate Release Candidate commit number (target date: 2022-02-25, latest 2022-02-28; @cgi1)
• Test by customer (30 day timespan)
  • Checking Conformance Targets for CVRF CSAF Converter from CSAF 2.0 specification
  • Convert agreed set of input files and validate output files
  • Document/Communicate problems/findings in GitHub Issues
• Agree on test outcome, possible defects and issues that cannot be fixed in project time/budget ("Abnahmeprotokoll")
• Fix defects as agreed
• Document issues that cannot be fixed in project time/budget
• Project closing

Nit: f.string without placeholders in cvrf2csaf.py

Suggest for spring cleaning to handle the finding
flake8 → cvrf2csaf/cvrf2csaf.py:170:27: F541 f-string is missing placeholders

Just replace:

critical_exit(f'Input document not valid, reason(s).')

with:

critical_exit('Input document not valid, reason(s).')

Implement converter for first recursive XML section: DocumentTracking

As part of #3, we already integrated a 1-to-1 mapping between a flat XML element (DocumentPublisher).

Here, we would go into the first recursive conversion: DocumentTracking

This will provide us the code basis to go over other recursive sections in the further project one by one.

Transfer PyPI project

The package is currently on PyPI. The ownership should be transferred or new maintainers from the project should be added.

Deprecation: Fix the invalid backslash-character pairs

Suggest for spring cleaning to handle the finding
flake8 →
cvrf2csaf/cvrf2csaf.py:40:92: W605 invalid escape sequence '\.'
cvrf2csaf/cvrf2csaf.py:40:95: W605 invalid escape sequence '\-'
cvrf2csaf/cvrf2csaf.py:73:35: W605 invalid escape sequence '\.'
cvrf2csaf/section_handlers/vulnerability.py:162:35: W605 invalid escape sequence '\.'
cvrf2csaf/section_handlers/vulnerability.py:170:29: W605 invalid escape sequence '\.'

Just prefixing the string literals with an r should work

Adding a second backslash \ → \\ (making the combination a valid backslash escape) works also, but who would want to do that 😉 ...?

Nit: Unused variable in except clause

Suggest for spring cleaning to handle the finding
flake8 → cvrf2csaf/cvrf2csaf.py:153:9: F841 local variable 'e' is assigned to but never used

Just replace:

except etree.DocumentInvalid as e:

with:

except etree.DocumentInvalid:

Testing different producers

Each of the producer mostly has the same format or/and errors in validation, thus I pick just some examples

This was a manual testing with commit 867578e. Just to see if we have some conversion errors. Could be automated in the future.

Used producer files:
https://github.com/csaf-tools/CVRF-CSAF-Converter/tree/testing/examples

And this helper script:
https://github.com/csaf-tools/CVRF-CSAF-Converter/blob/testing/tests/test_producers.sh

RedHat - ALL invalid input

Invalid cvrf2doc namespace

INPUT FILE: examples/examples_redhat/RHSA-2021:2040.xml 
2022-02-18 15:04:29,499 - utils - CRITICAL - Input document not valid: Element '{http://www.icasi.org/CVRF/schema/cvrf/1.1}cvrfdoc': No matching global declaration available for the validation root. (<string>, line 0).

Fortiguard - ALL invalid input

Missing mandatory fields

INPUT FILE: examples/examples_fortiguard/FG-IR-21-192.xml 
2022-02-18 15:06:34,935 - utils - CRITICAL - Input document not valid: Element '{http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/cvrf}CurrentReleaseDate': This element is not expected. Expected is ( {http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/cvrf}Status ). (<string>, line 0).

Suse

28 examples OK

INPUT FILE: examples/examples_suse/1.2/cvrf-opensuse-su-2015%3A1968-1.xml 
2022-02-18 15:46:25,227 - cvrf2csaf - INFO - CSAF schema validation OK

210 examples with ERRORs (not conversion errors, but input errors)

2022-02-18 15:30:20,600 - vulnerability - ERROR - No product_id entry for CVSS score set.
2022-02-18 15:30:20,856 - cvrf2csaf - ERROR - CSAF schema validation error. Path: $.vulnerabilities[0].scores[0].products. Message: [] is too short.

282 examples with CRITICAL
e.g. missing ProductID in Vulnerabilities

2022-02-18 15:49:27,111 - utils - CRITICAL - Input document not valid: Element '{http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/vuln}Status': Missing child element(s). Expected is ( {http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/vuln}ProductID ). (<string>, line 0).

Cisco

100 examples OK - took first 100 files from the batch and all of them were valid, no errors.

Siemens

5 examples OK - Haven't found any "browsing" page for Siemens CVRFs, managed to google a few examples, all of them valid, no errors.

Oracle

3 examples with CRITICAL - wrong cvrfdoc namespace used

2022-02-28 17:13:30,646 - cvrf2csaf - ERROR - Errors during input validation occurred, reason(s): [git/CVRF-CSAF-Converter/examples/examples_oracle/cpujan2022cvrf.xml:4:0:ERROR:SCHEMASV:SCHEMAV_CVC_ELT_1: Element '{http://www.icasi.org/CVRF/schema/cvrf/1.1}cvrfdoc': No matching global declaration available for the validation root.].
2022-02-28 17:13:30,646 - utils - CRITICAL - Input document not valid, reason(s).

Brainstorm vulnerability section

From EPIC #10

References:

• 1.2 specification

changed 1.2 -> 2.0 spec

Special to mention here is E.1 and E.2 here, which states:

E.1 Newly introduced elements

• /vulnerabilities[]/involvements[]/date: Holds the date and time of the involvement entry.
• /vulnerabilities[]/product_status/under_investigation: It is not known yet whether these versions are or are not affected by the vulnerability. However, it is still under investigation - the result will be provided in a later release of the document.
• /vulnerabilities[]/remediations[]/restart_required: Provides information on category of restart is required by this remediation to become effective.
• /vulnerabilities[]/scores[]: Specifies information about (at least one) score of the vulnerability and for which products the given value applies. Previously, products where directly tied to the vuln:ScoreSetV2 or vuln:ScoreSetV3.
• /vulnerabilities[]/scores[]/cvss_v2/*: Additional elements were introduced through the use of the FIRST CVSSv2 schema.
• /vulnerabilities[]/scores[]/cvss_v3/*: Additional elements were introduced through the use of the FIRST CVSSv3 schemas.

E.2 Changed elements

| /vulnerabilities | /cvrf:cvrfdoc/vuln:Vulnerability | |
| /vulnerabilities[i] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1] | |
| /vulnerabilities[i]/acknowledgments | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments | |
| /vulnerabilities[i]/acknowledgments[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments/vuln:Acknowledgment[j+1] | |
| /vulnerabilities[i]/acknowledgments[j]/names | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments/vuln:Acknowledgment[j+1]/vuln:Name | |
| /vulnerabilities[i]/acknowledgments[j]/names[k] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments/vuln:Acknowledgment[j+1]/vuln:Name[k+1]/text() | |
| /vulnerabilities[i]/acknowledgments[j]/organization | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments/vuln:Acknowledgment[j+1]/vuln:Organization[1]/text() | see E.2 |
| /vulnerabilities[i]/acknowledgments[j]/summary | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments/vuln:Acknowledgment[j+1]/vuln:Description/text() | |
| /vulnerabilities[i]/acknowledgments[j]/urls | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments/vuln:Acknowledgment[j+1]/vuln:URL| |
| /vulnerabilities[i]/acknowledgments[j]/urls[k] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Acknowledgments/vuln:Acknowledgment[j+1]/vuln:URL[k+1]/text() | |
| /vulnerabilities[i]/cve | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVE/text() | |
| /vulnerabilities[i]/cwe | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CWE | |
| /vulnerabilities[i]/cwe/id | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CWE/@ID | |
| /vulnerabilities[i]/cwe/name | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CWE/text() | |
| /vulnerabilities[i]/discovery_date | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:DiscoveryDate/text() | |
| /vulnerabilities[i]/id | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ID | |
| /vulnerabilities[i]/id/system_name | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ID/@SystemName | |
| /vulnerabilities[i]/id/text | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ID/text() | |
| /vulnerabilities[i]/involvements | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Involvements | |
| /vulnerabilities[i]/involvements[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Involvements/vuln:Involvement[j+1] | |
| /vulnerabilities[i]/involvements[j]/date | | see E.1 |
| /vulnerabilities[i]/involvements[j]/party | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Involvements/vuln:Involvement[j+1]/@Party | |
| /vulnerabilities[i]/involvements[j]/status | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Involvements/vuln:Involvement[j+1]/@Status | |
| /vulnerabilities[i]/involvements[j]/summary | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Involvements/vuln:Involvement[j+1]/vuln:Description/text() | |
| /vulnerabilities[i]/notes | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Notes | |
| /vulnerabilities[i]/notes[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Notes/vuln:Note[j+1] | |
| /vulnerabilities[i]/notes[j]/audience | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Notes/vuln:Note[j+1]/@Audience | |
| /vulnerabilities[i]/notes[j]/category | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Notes/vuln:Note[j+1]/@Type | |
| /vulnerabilities[i]/notes[j]/text | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Notes/vuln:Note[j+1]/text() | |
| /vulnerabilities[i]/notes[j]/title | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Notes/vuln:Note[j+1]/@Title | |
| /vulnerabilities[i]/product_status | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses | |
| /vulnerabilities[i]/product_status/first_affected | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="First Affected"] | |
| /vulnerabilities[i]/product_status/first_affected[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="First Affected"]/vuln:ProductID[j+1] | |
| /vulnerabilities[i]/product_status/first_fixed | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="First Fixed"] | |
| /vulnerabilities[i]/product_status/first_fixed[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="First Fixed"]/vuln:ProductID[j+1] | |
| /vulnerabilities[i]/product_status/fixed | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Fixed"] | |
| /vulnerabilities[i]/product_status/fixed[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Fixed"]/vuln:ProductID[j+1] | |
| /vulnerabilities[i]/product_status/known_affected | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Known Affected"] | |
| /vulnerabilities[i]/product_status/known_affected[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Known Affected"]/vuln:ProductID[j+1] | |
| /vulnerabilities[i]/product_status/known_not_affected | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Known Not Affected"]| |
| /vulnerabilities[i]/product_status/known_not_affected[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Known Not Affected"]/vuln:ProductID[j+1] | |
| /vulnerabilities[i]/product_status/last_affected | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Last Affected"] | |
| /vulnerabilities[i]/product_status/last_affected[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Last Affected"]/vuln:ProductID[j+1] | |
| /vulnerabilities[i]/product_status/recommended | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Recommended"] | |
| /vulnerabilities[i]/product_status/recommended[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ProductStatuses/vuln:Status[@Type="Recommended"]/vuln:ProductID[j+1] | |
| /vulnerabilities[i]/product_status/under_investigation | | see E.1 |
| /vulnerabilities[i]/product_status/under_investigation[] | | see E.1 |
| /vulnerabilities[i]/references | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:References | |
| /vulnerabilities[i]/references[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:References/vuln:Reference[j+1] | |
| /vulnerabilities[i]/references[j]/category | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:References/vuln:Reference[j+1]/@Type | |
| /vulnerabilities[i]/references[j]/summary | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:References/vuln:Reference[j+1]/vuln:Description/text() | |
| /vulnerabilities[i]/references[j]/url | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:References/vuln:Reference[j+1]/vuln:URL/text() | |
| /vulnerabilities[i]/release_date | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:ReleaseDate/text() | |
| /vulnerabilities[i]/remediations | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations | |
| /vulnerabilities[i]/remediations[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1] | |
| /vulnerabilities[i]/remediations[j]/category | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/@Type | |
| /vulnerabilities[i]/remediations[j]/date | | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/@Date |
| /vulnerabilities[i]/remediations[j]/details | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:Description/text() | |
| /vulnerabilities[i]/remediations[j]/entitlements | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:Entitlement | |
| /vulnerabilities[i]/remediations[j]/entitlements[] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:Entitlement[k+1]/text() | |
| /vulnerabilities[i]/remediations[j]/group_ids | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:GroupID | |
| /vulnerabilities[i]/remediations[j]/group_ids[k] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:GroupID[k+1]/text() | |
| /vulnerabilities[i]/remediations[j]/product_ids | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:ProductID | |
| /vulnerabilities[i]/remediations[j]/product_ids[k] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:ProductID[k+1]/text() | |
| /vulnerabilities[i]/remediations[j]/restart_required | | see E.1 |
| /vulnerabilities[i]/remediations[j]/restart_required/category | | see parent |
| /vulnerabilities[i]/remediations[j]/restart_required/details | | see parent |
| /vulnerabilities[i]/remediations[]/url | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Remediations/vuln:Remediation[j+1]/vuln:URL/text() | |
| /vulnerabilities[i]/scores | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets | |
| /vulnerabilities[i]/scores[] | | see E.1, E.2 |
| /vulnerabilities[i]/scores[n]/cvss_v2 | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV2[k] | see E.2 |
| /vulnerabilities[i]/scores[n]/cvss_v2/version | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/vectorString | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV2[j]/vuln:VectorV2/text() | |
| /vulnerabilities[i]/scores[n]/cvss_v2/accessVector | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/accessComplexity | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/authentication | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/confidentialityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/integrityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/availabilityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/baseScore | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV2[j]/vuln:BaseScoreV2/text() | |
| /vulnerabilities[i]/scores[n]/cvss_v2/exploitability | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/remediationLevel | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/reportConfidence | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/temporalScore | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV2[j]/vuln:TemporalScoreV2/text() | |
| /vulnerabilities[i]/scores[n]/cvss_v2/collateralDamagePotential | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/targetDistribution | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/confidentialityRequirement | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/integrityRequirement | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/availabilityRequirement | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v2/environmentalScore | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV2[j]/vuln:EnvironmentalScoreV2/text() | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3 | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV3[k] | see E.2 |
| /vulnerabilities[i]/scores[n]/cvss_v3/version | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/vectorString | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV3[k]/vuln:VectorV3/text() | |
| /vulnerabilities[i]/scores[n]/cvss_v3/attackVector | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/attackComplexity | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/privilegesRequired | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/userInteraction | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/scope | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/confidentialityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/integrityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/availabilityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/baseScore | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV3[k]/vuln:BaseScoreV3/text() | |
| /vulnerabilities[i]/scores[n]/cvss_v3/baseSeverity | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/exploitCodeMaturity | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/remediationLevel | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/reportConfidence | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/temporalScore | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV3[k]/vuln:TemporalScoreV3/text() | |
| /vulnerabilities[i]/scores[n]/cvss_v3/temporalSeverity | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/confidentialityRequirement | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/integrityRequirement | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/availabilityRequirement | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedAttackVector | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedAttackComplexity | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedPrivilegesRequired | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedUserInteraction | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedScope | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedConfidentialityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedIntegrityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/modifiedAvailabilityImpact | | see E.1 |
| /vulnerabilities[i]/scores[n]/cvss_v3/environmentalScore | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV3[k]/vuln:EnvironmentalScoreV3/text() | |
| /vulnerabilities[i]/scores[n]/cvss_v3/environmentalSeverity | | see E.1 |
| /vulnerabilities[i]/scores[n]/products | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV2[j]/vuln:ProductID or /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV3[k]/vuln:ProductID | see E.2 |
| /vulnerabilities[i]/scores[n]/products[l] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV2[j]/vuln:ProductID[l+1]/text() or /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:CVSSScoreSets/vuln:ScoreSetV3[k]/vuln:ProductID[l+1]/text() | see E.2 |
| /vulnerabilities[i]/threats | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats | |
| /vulnerabilities[i]/threats[j] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1] | |
| /vulnerabilities[i]/threats[j]/category | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1]/@Type | |
| /vulnerabilities[i]/threats[j]/date | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1]/@Date | |
| /vulnerabilities[i]/threats[j]/details | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1]/vuln:Description/text() | |
| /vulnerabilities[i]/threats[j]/group_ids | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1]/vuln:GroupID | |
| /vulnerabilities[i]/threats[j]/group_ids[k] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1]/vuln:GroupID[k+1]/text() | |
| /vulnerabilities[i]/threats[j]/product_ids | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1]/vuln:ProductID | |
| /vulnerabilities[i]/threats[j]/product_ids[k] | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Threats/vuln:Threat[j+1]/vuln:ProductID[k+1]/text() | |
| /vulnerabilities[i]/title | /cvrf:cvrfdoc/vuln:Vulnerability[i+1]/vuln:Title/text() | |

Agree on logging semantics

Just to have a clear logging scheme, I propose these log-level meanings:

• Critical - Converter encountered a case which it can not handle and exits immediately without producing any output, exiting with non-zero code. E.g. invalid input CVRF
• Error - TBD
• Warning - Any intervention from the Converter which alters the output (e.g. reindexing versions to Integer)
• Info - Just messages about the progress (e.g. "Section Document Tracking handled successfully")

@tschmidtb51 would you agree? Do you have an idea for use-case for Errors?

Hardening: Survive more CVRF input files (by not resolving entities)

Maybe perform this patch or something alike (there may well be other places where a parser is setup with the default resolve_entities=True setting that I did not spot)?

--- cvrf2csaf/cvrf2csaf.py	2022-03-12 15:13:48.000000000 +0100
+++ ../cvrf2csaf.py	2022-03-12 15:10:35.000000000 +0100
@@ -24,6 +24,7 @@

 logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(module)s - %(levelname)s - %(message)s')

+unresolving_parser = objectify.makeparser(resolve_entities=False)

 class DocumentHandler:
     """
@@ -162,7 +163,7 @@
     @classmethod
     def _open_and_validate_file(cls, file_path):
         try:
-            xml_objectified = objectify.parse(file_path)
+            xml_objectified = objectify.parse(file_path, parser=unresolving_parser)
         except Exception as e:
             critical_exit(f'Failed to open input file {file_path}: {e}.')

Define output name based on CSAF standard

• Output file name is based on DocumentTrackingId
• Change CLI option for for output filename into output-dir

Check CSAF standard 5.1

Implement the DocumentNotes section

EPIC: #10

CVRF 1.1 view

CVRF 1.2

[Oasis v1.2 -> 4.6 Document Notes]http://docs.oasis-open.org/csaf/csaf-cvrf/v1.2/cs01/csaf-cvrf-v1.2-cs01.html#_Toc493508888)

Oasis implementation view CSAF v2

Field Definition 3.1.5 Notes Type

changed 1.2 -> 2.0 spec:

changed 1.2 -> 2.0 spec:

E.1 Newly introduced elements

-- no new elements

E.2 Changed elements

-- no changed elements

E.4 Mapped elements

| /document/notes | /cvrf:cvrfdoc/cvrf:DocumentNotes | |
| /document/notes[i] | /cvrf:cvrfdoc/cvrf:DocumentNotes/cvrf:Note[i+1] | |
| /document/notes[i]/audience | /cvrf:cvrfdoc/cvrf:DocumentNotes/cvrf:Note[i+1]/@Audience | |
| /document/notes[i]/category | /cvrf:cvrfdoc/cvrf:DocumentNotes/cvrf:Note[i+1]/@Type | |
| /document/notes[i]/text | /cvrf:cvrfdoc/cvrf:DocumentNotes/cvrf:Note[i+1]/text() | |
| /document/notes[]/title | /cvrf:cvrfdoc/cvrf:DocumentNotes/cvrf:Note[i+1]/@Title | |

Example Test fiie

Create a Python package for installation

Create a Python package (setup.py, setup.cfg) for installation/distribution

Implement rudimental prototyp for Document Publisher

As a very first test, we would like to convert Document Publisher alone from CVRF 1.2 to CSAF, by establishing a general read / write workflow.

• Create first entrypoint for cvrf2csaf.py (argparse)
• Read CVRF 1.2 sample file
• Establish rudimental mapping for first XML element tree (Document Publisher)
• Save JSON to output file

Implement the Aggregate Severity section

EPIC: #10

CVRF 1.2

[Oasis v1.2 -> Aggregate Severity]http://docs.oasis-open.org/csaf/csaf-cvrf/v1.2/cs01/csaf-cvrf-v1.2-cs01.html#_Toc493508891)

Oasis implementation view CSAF v2

Field Definiition 4.8 Aggregate Severity

Simple mapping

Implement Toplevel Leaf Elements handler

Aggregate some of the top-level elements to a single class.

This reduces the boilerplate code around section handlers

The new class ToplevelLeafElements takes care about all the top-level elements that do not have children elements, these are:

Also, the class handles the leaf (mandatory) elements of the CSAF 'document' JSON, which are not present in the input CVRF, these are (is):

• csaf_version

Discuss: Input validation of XML file

While this is not part of the offer, we would like to also run some basic input validation on XML level.

Are you aware of a good tool to run?

Or should we go into validating the XML schema of CVRF 1.2 (e.g. using lxml)?

In the end, the XML schema validator will help us to reject invalid files as a pre-step.

Validate CSAF output against CSAF schema

A "first-level" validation for validating the output is verifying that it's compliant with the CSAF JSON schema.

The validation comes right before saving the output to a file, and:

Valid -> logging INFO
Invalid -> logging ERROR (writing into a file depends on the --force option, this will be implemented when handling --force in general)

Create License.MD automatically

Try out pip-licenses.

For compliance reasons, everything which is != MIT licence needs to be added to License.MD. Generating a short skript to automatically generate this file ensures licence compliant coding.

Github project setup

• Invite members from DTSEC and BSI
• Setup organization csaf-tools
• Create repository with sufficient rights

Nit: Language in README

Suggest to replace

Assure that you have Python 3 installed.

in README.md with the presumably intended

Ensure that you have Python 3 installed.

I am guessing here, but:

Assure:
removing doubt (or attempting to) from someone’s mind

Ensure:
to make sure, certain, or safe.

README: Complete documentation

The README currently lacks the sections Developer Guide, Architecture and Technical Design and Security Considerations. Those should be added.

Add pylint as GH Action

We should add a GitHub Action to check and maintain the code quality.

Implement the DocumentReferences section

Part of EPIC #10.

CVRF 1.1 view

CVRF 1.2

Oasis v1.2 -> 4.9 Document References

Oasis implementation view CSAF v2

Field Definition 3.1.10 References Type

changed 1.2 -> 2.0 spec:

E.1 Newly introduced elements

-- no new elements

E.2 Changed elements

| /document/references | /cvrf:cvrfdoc/cvrf:DocumentReferences | |
| /document/references[i] | /cvrf:cvrfdoc/cvrf:DocumentReferences/cvrf:Reference[i+1] | |
| /document/references[i]/category | /cvrf:cvrfdoc/cvrf:DocumentReferences/cvrf:Reference[i+1]/@Type | |
| /document/references[i]/summary | /cvrf:cvrfdoc/cvrf:DocumentReferences/cvrf:Reference[i+1]/cvrf:Description/text() | |
| /document/references[]/url | /cvrf:cvrfdoc/cvrf:DocumentReferences/cvrf:Reference[i+1]/cvrf:URL/text() | |

Acknowledgments

The class is correctly named Acknowledgments but the filename is still document_acknowledgments.py. That should be corrected.

Address pyLint results

To ensure code quality, #89 suggests to integrate pyLint as GitHub Action. However, some results must be addressed before doing that.

As discussed today with @pixelkunst-net, the following outputs can be ignored (through adding exemptions to the code):

• cvrf2csaf/common/utils.py:39:9: W0511: TODO: Workaround for now, config file placement is to be discussed (fixme)
• C0103: Variable name "e" doesn't conform to snake_case naming style (invalid-name)

Error: undefined name in handle_boolean_config_values (runtime error)

Smelling copy-pasta-search-replace (CPSR) with a failed R step 😉

def handle_boolean_config_values(key, val):
    try:
        if isinstance(val, bool):
            return val
        if val.strip().lower() in {'true', 'yes', '1', 'y'}:
            return True
        if val.strip().lower() in {'false', 'no', '0', 'n'}:
            return False

        critical_exit(f"Reading config.yaml failed. "
                      f"Invalid value for config key {key}: {val}. {e}.")

    except Exception as e:
        critical_exit(f"Reading config.yaml failed. "
                      f"Invalid value for config key {key}: {val} {e}.")

Problem:

>>> import cvrf2csaf.common.utils as bomb
>>> bomb.handle_boolean_config_values('whatever', 'bomb')
2022-03-12 02:14:48,332 - utils - CRITICAL - Reading config.yaml failed. Invalid value for config key whatever: bomb local variable 'e' referenced before assignment.

Suggest to handle the finding
flake8 → cvrf2csaf/common/utils.py:29:69: F821 undefined name 'e'

Minimal fix:

f"Invalid value for config key {key}: {val}. {e}."

with:

f"Invalid value for config key {key}: {val}."

Real (assuming here) fix would be to not care if a type was wrong (where the .strip() would cause an exception) but simply raise a subsequently handled exception after the if-elif-elif dispatch for any "survivors" and reuse the logging call in the exception handler.

Example implementation:
Narrowing the Exception down to an AttributeError and injecting a ValueErrorif we fall off the normal handling in a soft way should just work fine. So what about:

def handle_boolean_config_values(key, val):
    """Translate boolean like values to boolean or initiate exit."""
    try:
        if isinstance(val, bool):
            return val
        if val.strip().lower() in {'true', 'yes', '1', 'y'}:
            return True
        if val.strip().lower() in {'false', 'no', '0', 'n'}:
            return False

        raise ValueError("unexpected value")

    except (AttributeError, ValueError) as err:
        critical_exit(f"Reading config.yaml failed.  Invalid value for config key {key}: {val} {err}.")

Short test:

>>> import spike  # Sorry, the above function as is in spike.py plus critical_exit = print
>>> spike.handle_boolean_config_values(42, "12")
Reading config.yaml failed.  Invalid value for config key 42: 12 unexpected value. 

Deliver standard README template

BSI has a standard template.

Enable 2FA as mandatory for Github project

Meet OSAF conformance targets on cvrf-csaf-converter

The official conformance target states the following requirements:

---

• satisfies the "CSAF producer" conformance profile
• takes only CVRF documents as input.
• additionally satisfies the normative requirements given below.

Secondly, the program fulfills the following for all items of:

• type /$defs/version_t: If any element doesn't match the semantic versioning, replace the all elements of type /$defs/version_t with the corresponding integer version. For that, CVRF CSAF converter sorts the items of /document/tracking/revision_history by number ascending according to the rules of CVRF. Then, it replaces the value of number with the index number in the array (starting with 1). The value of /document/tracking/version is replaced by value of number of the corresponding revision item. The match must be calculated by the original values used in the CVRF document.
• /document/acknowledgments[]/organization and /vulnerabilities[]/acknowledgments[]/organization: If more than one cvrf:Organization instance is given, the CVRF CSAF converter converts the first one into the organization. In addition the converter outputs a warning that information might be lost during conversion of document or vulnerability acknowledgment.
• /document/publisher/name and /document/publisher/namespace: Sets the value as given in the configuration of the program or the corresponding argument the program was invoked with. If values from both sources are present, the program should prefer the latter one. The program SHALL NOT use hard-coded values.
• /vulnerabilities[]/scores[]: If no product_id is given, the CVRF CSAF converter appends all Product IDs which are listed under ../product_status in the arrays known_affected, first_affected and last_affected.
• /vulnerabilities[]/scores[]: If there are CVSS v3.0 and CVSS v3.1 Vectors available for the same product, the CVRF CSAF converter discards the CVSS v3.0 information and provide in CSAF only the CVSS v3.1 information.
• /product_tree/relationships[]: If more than one prod:FullProductName instance is given, the CVRF CSAF converter converts the first one into the full_product_name. In addition, the converter outputs a warning that information might be lost during conversion of product relationships.

Encode HTML in JSON output

• CSAF producers SHOULD NOT emit messages that contain HTML, even though all variants of Markdown permit it. To include HTML, source code, or any other content that may be interpreted or executed by a CSAF consumer, e.g. to provide a proof-of-concept, the issuing party SHALL use Markdown's fenced code blocks or inline code option.

Source: Safety, Security, and Data Protection Considerations

A/C:

• Check for HTML content in XML input
• Encode the HTML input for the JSON output
• Write CI/CD test case, where the encoding is checked for a sample file containing HTML

Validate CSAF 2.0 6.1.26 Document Category special case

Mandatory Tests

Special explanation for 6.1.26 Prohibited Document Category Name:

Document Category can be one of the five given definitions, or any other value.

The 6.1.26 checks that the Document Category value should not be similar to one of these values, otherwise the profiles from chapter 4 are taken.

  Informational Advisory
  security-incident-response
  Security      Advisory
  veX

We implement this by

0. is the Document Category in the above list? Skip.
1. Generic CSAF check: remove whitespaces/brackets/-, turn into lower case and check if the value is equal to one of the above? Reject.

Nit: Unused import in document leaf element handler

Suggest for spring cleaning to handle the finding
flake8 → cvrf2csaf/section_handlers/document_leaf_elements.py:1:1: F401 'logging' imported but unused

Just remove line 1 stating:

import logging

Install points to discuss

Points to discuss :

1) How to distribute the config file? For now it's inside the package, shall we put it inside /etc/, /usr/local/share or some other path?

See: https://github.com/csaf-tools/CVRF-CSAF-Converter/blob/issue/20-install-package/cvrf2csaf/common/utils.py#L12

2) Usage through CLI command is now possible, shall it be also possible to use it as a python module? (e.g. import cvrf2csaf; cvrf2csaf.convert(...))

Originally posted by @sustefil in #23 (comment)

RegEx validation for CPE not correct

A valid CPE, according to CPE Naming Format 2.3, can't be used in a CVRF-Document. Regex is to strict / not correct.

Attribute CPE

The (Common Platform Enumeration) CPE attribute refers to a method for naming platforms external to CSAF CVRF.

« The CPE attribute if present MUST have a value, that is a valid cpe-lang:namePattern as defined in the external specification [CPE23_N] and related schemas. » [CSAF-5.1.2-2]

Example CPE: cpe:2.3:a:gemalto:sentinel_ldk_rte:3.0:*:*:*:*:*:*:*

Log:
2022-02-17 08:56:31,011 - utils - CRITICAL - Input document not valid: Element '{http://docs.oasis-open.org/csaf/ns/csaf-cvrf/v1.2/prod}FullProductName', attribute 'CPE': [facet 'pattern'] The value 'cpe:2.3:a:gemalto:sentinel_ldk_rte:3.0:*:*:*:*:*:*:*' is not accepted by the pattern '[c][pP][eE]:/[AHOaho]?(:[A-Za-z0-9\._\-~%]*){0,6}'. (, line 0).

Vulnerability:CWE discrepancies

While working on Vulnerabilities section I noticed the following discrepancy in CVRF:
In the specification of CVRF 1.2 is stated that CWE element MUST be present zero or one time, while in xsd scheme of CVRF 1.2 the occurrence is not bounded <xs:element name="CWE" minOccurs="0" maxOccurs="unbounded">.

The second related discrepancy in CSAF:
vulnerabilities property in this specification does not have any property marked as mandatory (not even CWE), while here it says It must be tested that the CWE is given.

These two discrepancies raises questions how to approach this in the implementation:

1. If there is no bound on number of CWE in CVRF document and the program gets >1 CWE on the input. How to handle this issue?
2. If CWE is mandatory field for CSAF and this element is not present in the input CVRF document. How to handle this issue?

Implement the ProductTree group of sections

Part of EPIC #10

CVRF 1.1 view

changed 1.2 -> 2.0 spec

E.1 Newly introduced elements

• /product_tree/*/product/product_identification_helper: Provides at least one method which aids in identifying the product in an asset database. It was introduced to group different ways to identify a product/
• /product_tree/*/product/product_identification_helper/hashes: Contains a list of cryptographic hashes usable to identify files.
• /product_tree/*/product/product_identification_helper/purl: The package URL (purl) attribute refers to a method for reliably identifying and locating software packages external to this specification.
• /product_tree/*/product/product_identification_helper/sbom_urls: Contains a list of URLs where SBOMs for this product can be retrieved.
• /product_tree/*/product/product_identification_helper/serial_numbers: Contains a list of parts, or full serial numbers.
• /product_tree/*/product/product_identification_helper/skus: Contains a list of parts, or full stock keeping units.
• /product_tree/*/product/product_identification_helper/x_generic_uris: Contains a list of identifiers which are either vendor-specific or derived from a standard not yet supported.

E.2 Changed elements

Around 130 element changes are documented --> /cvrf:cvrfdoc/prod:ProductTree

Field definitions

3.2.2 Product Tree Property

/product_tree/relationships[]: If more than one prod:FullProductName instance is given, the CVRF CSAF converter converts the first one into the full_product_name. In addition, the converter outputs a warning that information might be lost during conversion of product relationships. quoted here

Which xs:date timestamp format to use

The xs:date type in Secvisogram looks like this:

• "date": "2021-12-10T08:28:32.996Z"
• That is an ISO 8601 UTC timestamp

Python doesn't have support for military time-zones (Z at the end, indicating UTC), so we can either:

1. Use native Python function, producing this format:

   • function: datetime.now(timezone.utc).isoformat(timespec='milliseconds')
   • format: '2021-12-10T09:05:47.966+00:00'

2. Use the same function, but also replace the string +00:00 with Z

Use csaf package

The package turvallisuusneuvonta was renamed into csaf and is available on PyPI. We should adopt that in the converter.

End2End Test: Validate conversion of 1k CVRF files

A/C

Create a test script, which

• picks 1k CVRF files from an input folder.
• do the conversion into CSAF 2.0 JSON.
• count amount of conversion errors and displays.
• Bonus: Integrate it into Github CI/CD Pipeline in order to continously monitor the code quality on every Master Push.

Implement Language conversion

The conformance clause 5: CVRF CSAF converter states the conversion rule for the document language:

/document/lang: If one or more CVRF element containing an xml:lang attribute exist and contain the exact same value, the CVRF CSAF converter converts this value into lang. If the values of xml:lang attributes are not equal, the CVRF CSAF converter outputs a warning that the language could not be determined and possibly a document with multiple languages was produced. In addition, it SHOULD also present all values of xml:lang attributes as a set in the warning.

This must be implemented in the CVRF-CSAF converter to be conformant.

Background check: Validate changes between CVRF 1.2 and CSAF

Researching for a complete changelog between CVRF 1.2 and CSAF 2.0, we could not find a complete list.

Therefore, we asked in Oasis TCS CSAF project for it and sthagen reported back, that currently there is no such complete change log. However, there is a prelimary mapping which will slightly change over time (CS01), but looks to be the best changelog publicy available.

As this is a moving target, we will check back later if we covered all changes during the converter implementation.

Draft: Implement section ProductTree->Branches

Part of EPIC #10. From bigger recursive section #26.

Sub Issues:

• branches -> #31
• full_product_name -> OPEN
• product_groups -> OPEN
• relationships -> OPEN

CVRF 1.1 view

(yes - recursion!)

CVRF 1.2

Oasis v1.2 -> 2.2.3 Product Branch Type Model

Oasis v1.2 -> Integration into higher level ProductTree leaf-> 5.1.1 Product Tree – Branch

Oasis implementation view CSAF v2

Field definitions 3.1.2 Branches Type

changed 1.2 -> 2.0 spec:

E.1 Newly introduced elements

-- no new elements

E.2 Changed elements

| `/product_tree/branches` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch` |  |
| `/product_tree/branches[i]` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]` |  |
| `/product_tree/branches[i]/branches` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch` |  |
| `/product_tree/branches[i]/branches[j]` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]` |  |
| `/product_tree/branches[i]/branches[j]/branches` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/branches` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:Branch` | |
| `/product_tree/branches[i]/branches[j]/branches[k]/branches[l]` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:Branch[l+1]` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/branches[l]/category` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:Branch[l+1]/@Type` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/branches[l]/name` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:Branch[l+1]/@Name` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/category` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/@Type` | |
| `/product_tree/branches[i]/branches[j]/branches[k]/name` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/@Name` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/product` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:FullProductName` | |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/name` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:FullProductName/text()` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_id` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:FullProductName/@ProductID` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper/cpe` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:Branch[k+1]/prod:FullProductName/@CPE` |  |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper/hashes` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper/purl` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper/sbom_urls` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper/serial_numbers` | | see E.1 |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper/skus` | | see E.1 |
| `/product_tree/branches[i]/branches[j]/branches[k]/product/product_identification_helper/x_generic_uris` | | see E.1 |
| `/product_tree/branches[i]/branches[j]/category` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/@Type` | |
| `/product_tree/branches[i]/branches[j]/name` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/@Name` |  |
| `/product_tree/branches[i]/branches[j]/product` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:FullProductName` | |
| `/product_tree/branches[i]/branches[j]/product/name` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:FullProductName/text()` |  |
| `/product_tree/branches[i]/branches[j]/product/product_id` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:FullProductName/@ProductID` |  |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/cpe` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:Branch[j+1]/prod:FullProductName/@CPE` | |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/hashes` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/hashes[]` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/hashes[]/file_hashes` | | see parent |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/hashes[]/filename` |  | see parent |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/purl` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/sbom_urls` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/sbom_urls[]` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/serial_numbers` | | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/serial_numbers[]` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/skus` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/skus[]` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/x_generic_uris` | | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/x_generic_uris[]` |  | see E.1 |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/x_generic_uris[]/namespace` |  | see parent |
| `/product_tree/branches[i]/branches[j]/product/product_identification_helper/x_generic_uris[]/uri` |  | see parent |
| `/product_tree/branches[i]/category` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/@Type` |  |
| `/product_tree/branches[i]/name` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/@Name` |  |
| `/product_tree/branches[i]/product` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:FullProductName` | |
| `/product_tree/branches[i]/product/name` |  `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:FullProductName/text()` |  |
| `/product_tree/branches[i]/product/product_id` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:FullProductName/@ProductID` |  |
| `/product_tree/branches[i]/product/product_identification_helper` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/cpe` | `/cvrf:cvrfdoc/prod:ProductTree/prod:Branch[i+1]/prod:FullProductName/@CPE` |  |
| `/product_tree/branches[i]/product/product_identification_helper/hashes` | | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/hashes[]` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/hashes[]/file_hashes` | | see parent |
| `/product_tree/branches[i]/product/product_identification_helper/hashes[]/file_hashes[]` |  | see parent |
| `/product_tree/branches[i]/product/product_identification_helper/hashes[]/file_hashes[]/algorithm` |  | see parent |
| `/product_tree/branches[i]/product/product_identification_helper/hashes[]/file_hashes[]/value` |  | see parent |
| `/product_tree/branches[i]/product/product_identification_helper/hashes[]/filename` |  | see parent |
| `/product_tree/branches[i]/product/product_identification_helper/purl` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/sbom_urls` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/sbom_urls[]` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/serial_numbers` | | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/serial_numbers[]` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/skus` | | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/skus[]` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/x_generic_uris` | | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/x_generic_uris[]` |  | see E.1 |
| `/product_tree/branches[i]/product/product_identification_helper/x_generic_uris[]/namespace` |  | see parent |
| `/product_tree/branches[i]/product/product_identification_helper/x_generic_uris[]/uri` |  | see parent |

Convert all XML elements into JSON

After having a code basis for flat (#3) and recursive (#9) conversion, we now have the basis to basically convert all elements of the XML tree into JSON.

Elements (Pre #6)

• Leaf Elements of the document root (DocumentTitle DONE, DocumentType -> category (#13), DocumentDistribution OPEN, AggregateSeverity (#39))
• DocumentPublisher (#3)
• DocumentTracking (#9)
• DocumentNotes (#35)
• DocumentReferences (#28)
• Acknowledgments (#29)
• ProductTree (#26)
• Vulnerability (#30)

Project Kickoff

• Arrange in-person kickoff between client and contractor
• Present team & code quality metrics
• Agree on communication flow / contacts
• Agree on project timeline and delivery date(s)
• Is CVRF1.2 the agreed input format?
• Agree on issues, milestones and deliverables (#12)
• Agree on who is creating issues and how feedback is provided
• What should be added in the Readme? Contacts? Roadmap? Project goals?

Implement the Acknowledgments section

Part of EPIC #10

CVRF 1.1 view

changed 1.2 -> 2.0 spec:

/document/acknowledgments[]/organization and /vulnerabilities[]/acknowledgments[]/organization: If more than one cvrf:Organization instance is given, the CVRF CSAF converter converts the first one into the organization. In addition the converter outputs a warning that information might be lost during conversion of document or vulnerability acknowledgment.

E.1 Newly introduced elements

-- no entries

E.2 Changed elements

• */acknowledgments[]/organization: See conversion rule in section 9.1.5 of CSAF specification.

(9.1.5 is a copy/paste with changed 1.2 -> 2.0 spec - so nothing in addition)

Oasis implementation view CSAF v2

Field definition Acknowledgment Type

Clarify expected behavior of adding a missing version to the revision history

As there was some confusion in #21 (unfortunately caused by a misunderstanding), I try to sketch the expected behavior as a new issue:

• If the current version is not in the revision history (independent from the current release date): throw an error
• The config (and command line) option fix_current_version_not_in_revision_history (feel free to choose a different name) has the default value false. When it is set to true, the converter adds the current version in the revision history if it is missing (independently whether the previous version exists) and adds a warning that the current version was not in the revision history and was added and that this might impact the revision history's integrity.

This requires the following changes:

1. Rename the config option --force-update-revision-history as it is not intuitive. When I read --force-update-revision-history I would expect that this option updates any revision history (independently whether valid before or invalid as e.g. A, B, C) to a valid integer versioning.
   CSAF uses the terminology of quick-fixes for recommended actions if one of the tests from section 6 fails. That's why I came up with fix_current_version_not_in_revision_history. However, I'm also happy with a --force~ if it conveys that it adds only the current version if it is missing.
2. Rewrite the code
   

   CVRF-CSAF-Converter/cvrf2csaf/section_handlers/document_tracking.py

   Lines 79 to 91 in 3fef149

   	if current_version[-1] - latest_history_revision[-1] > 1:
   	if self.force_update_revision_history is True:
   	logging.warning('Forcing update of the revision history and adding the current version. '
   	'This may lead to inconsistent history.')
   	else:
   	self._critical_exit('Too big difference between the current version and the last revision in history. '
   	'This can be fixed by using --force-update-revision-history')
   	elif current_version[-1] - latest_history_revision[-1] == 1:
   	logging.warning('Adding the current version to the revision history (difference is only 1 version).')
   	else:
   	self._critical_exit('Unexpected case occured when trying to fix the revision history.')

    if fix_current_version_not_in_revision_history is True:
        logging.warning('Forcing update of the revision history and adding the current version. ' 
                            'This may lead to inconsistent history.') 
    else: 
       logging.error('Current version is missing in revision history. ' 
                                'This can be fixed by using --fix_current_version_not_in_revision_history')
      return
   

   This suggestion tries to point out the intended function and might not fulfill coding standards nor be in the right place...
3. Obviously, the config parsing code and related code has to be adopted.

Validate mandatory tests

Pre: #10

Please also add CLI arg to skip mandatory tests, because the user could also fix the input in Secvisogram.

Desc

Only the relevant parts are written in the specification, the full examples are written here. They are failing (at least) for the specific cases.

The naming of the examples is encoded like this:

OASIS_CSAF_TC-CSAF_2_0-2021-6-1-05-01.json

6.1.05 here means, that this is a test file for Mandatory test 6.1.5

--> All mandatory tests have already been implemented by secvisogram already. More detailed here

Setup initial code structure

• Take care about the initial directory structure
• Create basic README