bmw-software-engineering/lobster

Lightweight Open BMW Software Traceability Evidence Report

LOBSTER

The Lightweight Open BMW Software Traceability Evidence Report allows you to demonstrate software traceability and requirements coverage, which is essential for meeting standards such as ISO 26262.

This repository contains the prototype for LOBSTER, which is a key ingredient to make TRCL more useful.

It has tools to extract tracing tags from a variety of sources combine them and produce a tracing report. The TRLC tracing report from the TRLC Project is a reasonable example of what is possible.

Installing

All tools can be installed separately from PyPI, but there is a convenient meta-package bmw-lobster which installs everything.

$ pip3 install bmw-lobster

Supported inputs

The following requirements frameworks are supported:

• TRLC (only some use cases supported right now)
• Codebeamer (only some use cases supported right now)

The following programming languages are supported:

• C/C++
• Python3
• SIMULINK and MATLAB using the 3rd-party tool MISS_HIT; supports both code and tests

The following verification and miscellaneous frameworks are supported:

• GoogleTest
• JSON useful if your test framework is encoded in JSON

Documentation

• Writing configuration files for LOBSTER.
• It is easy to expand the languages and activities supported by LOBSTER by adding new tracing tools, as long as they create data in the common interchange format for LOBSTER.

(More to come...)

Installing individual packages

The individual packages that bmw-lobster depends on are:

• bmw-lobster-core the core API and various report generators. All other tools depend on this.
• bmw-lobster-tool-cpp (for C/C++ code)
• bmw-lobster-tool-gtest (for GoogleTest tests)
• bmw-lobster-tool-python (for Python3 code)
• bmw-lobster-tool-beamer (for requirements in Codebeamer)
• bmw-lobster-tool-json (for activities in JSON)
• miss_hit (for MATLAB/Octave code or Simulink models)

Planned inputs

The following inputs are planned but not implemeted yet:

• lobster-java: Java code
• lobster-kotlin: Kotlin code
• lobster-ada: Ada and SPARK code (via libadalang)
• lobster-latex: Requirements written in LaTeX
• lobster-markdown: Requirements written in Markdown

Copyright & License information

The copyright holder of LOBSTER is the Bayerische Motoren Werke Aktiengesellschaft (BMW AG), and LOBSTER is published under the GNU Affero General Public License, Version 3.

lobster's Issues

Support using graphviz with explicit path

Problem:

When using Lobster with Bazel, it clears the PATH variable (to make sure all inputs are exactly known),
But lobster-html-report calls graphviz by looking up the location of the graphviz executible in PATH. Hence it is not possible to generate an image of the tracing policy in the HTML report.

Request:

Add a feature to specify the path to the graphviz executable.

rewrite lobster-gtest

Parse C++ fragment directly

lobster-python: add option to stop on warnings

unified entry point

One entry point with commands, e.g.

• lobster trace cpp <stuff...>
• lobster trace matlab <stuf...>
• lobster report [ --html | --ci ]
  etc.

parse .netrc instead of environment

add feature to allow some failure

e.g. --min-required-coverage=95

plugin system

Especially the lobster trace stuff should support plugins, so that you can add/install modules with a magic name and things just work.

allow configuration of html title

support single-object in lobster-json

For example

{
   "name": "foo",
   "requirements": [1, 2, 3]
}

Vanilla Example

Please create a "vanilla example" that demonstrates how to do the following:

• extract one requirement from a TRLC file with lobster-trlc
• extract one function from a Python file with lobster-python
• extract one test from a Python file with lobster-python --activity
• extract one function from a C++ file with lobster-cpp
• extract one requirement from codebeamer with lobster-codebeamer

and put it all together in a tracing policy, and generate an HTML lobster report.

Please think about the best way to publish the vanilla example. Maybe it is a good idea to provide a zip file, so that it easy extremely easy to download it.

The goal is to train interested users quickly how to use lobster.

lobster-python: false positive on duplicate function definition

lobster-python reports a duplicated function definition for severity() in the following example:

import dataclasses

@dataclasses.dataclass
class Diagnostic: 
    _severity: int = dataclasses.field()

    @property
    def severity(self):
        return self._severity

    @severity.setter
    def severity(self, value: int):
        self._severity = value

These are clearly not duplicate functions but separate ones which can both be called on the same instance if Diagnostic. Hence, this case should be fixed within lobster.

The problem might originate from here

lobster/lobster/tools/python/python.py

Lines 197 to 198 in 6bb274e

	l_item = Implementation(tag = Tracing_Tag("python",
	self.fqn()),

since the function tag is built only using self.fqn(), which is probably the function name. A solution might be to consider the entire function signature as the tag.

lobster-json: loops forever on windows with absolute paths

the name synthesis doesn't deal well with paths like C:\potato\foo.json

fix online report for repos with submodules

allow optional up-linking

e.g.

activity "test" {
  trace to: "system requirement" or "software requirement";
}

add feature to show just justification

perhaps in lobster-ci-report an additional flag --show-justifications

lobster-online-report shall not modify the input, but generate new output, to support bazel

Bazel requires that there is always an input and an output to a tool, so that Bazel can use smart caching mechanisms.
Unfortunately lobster-online-report modifies the input file.

Please add a new command line flag --out that allows to specify the output file. If no value is given, then the input file is modified (for backward compatibility).

include trlc package in requirement name

make lobster-trlc configurable

• need to at least be able to ignore certain objects of configurable types

allow internal trees in one level

Requirement A that inside one level is decomposed into B and C, would be OK if B and C are themselves OK.

Policy example:

requirements "Requirements" {
   source: "potato.lobster";
}

implementation "Code" {
   source: "code.lobster";
   trace to: "Requirements" with internal_links;
}

activity "Test" {
   source: "test.lobster";
   trace to: "Requirements";
}

OR perhaps allow self-references:

requirements "Requirements" {
   source: "potato.lobster";
   requires: "Code" or self_reference;
   requires: "Test";
}

implementation "Code" {
   source: "code.lobster";
   trace to: "Requirements"
}

activity "Test" {
   source: "test.lobster";
   trace to: "Requirements";
}

Would allow for example:

      R1
     / \
    R2  R3
    |   |
    C2  C3

As long as all requirements are individually tested.

Side-note: We now need to worry about cycles, i.e. R1 -> R2, R2 -> R1 and should make sure that these do not count.

This check only needs to be done for levels that allow self_references.

Replace outdated author e-mail

Replace Florian's outdated e-mail address with a valid BMW address.

Improve visual appearance of HTML report with respect to accessibility

Feedback from users with read-green weakness is that, they cannot distinguish the background coloring of the lobster HTML report. For some red and gray look very similar, for others it is red and green.

1. use different colors
2. use icons additionally to indicate "error", "justified exemption" and "ok".

lobster-json to treat tag attribute as optional

lobster-python duplicates tests if using unittest

If lobster-python runs over a Python project where unittest is used instead of pytest, then each test is also interpreted as a regular function.
This leads to the result that the final lobster report shows each unit test twice (as a function and as a test).

We need to check if this happens only for unittest, or also for pytest.

Handling duplicate function definitions

Current Behaviour

If a Python file contains several function definitions with the same name, then the output of lobster-python cannot be forwarded to lobster-report.

Consider the following Python file (call it code.py):

value = False

if value:
    def compute():
        return 42
else:
    def compute():
        return 43

print(compute())

Now run lobster-python:

> lobster-python code.py --out="python.lobster"

The result of python.lobster is the following:

{
  "data": [
    {
      "tag": "python code.compute",
      "location": {
        "kind": "file",
        "file": "code.py",
        "line": 4,
        "column": null
      },
      "name": "code.compute",
      "messages": [],
      "just_up": [],
      "just_down": [],
      "just_global": [],
      "language": "Python",
      "kind": "Function"
    },
    {
      "tag": "python code.compute",
      "location": {
        "kind": "file",
        "file": "code.py",
        "line": 8,
        "column": null
      },
      "name": "code.compute",
      "messages": [],
      "just_up": [],
      "just_down": [],
      "just_global": [],
      "language": "Python",
      "kind": "Function"
    }
  ],
  "generator": "lobster_python",
  "schema": "lobster-imp-trace",
  "version": 3
}

Now lobster-report cannot handle this input, as it detects a duplicate definition of "code.compute".
The error is raised here:

lobster/lobster/io.py

Line 128 in 666f8af

"duplicate definition of %s, "

Expected Behaviour

Now the question is: What is the expected behaviour?
For sure the above Python code is valid and needs to be supported by lobster.
On the other hand, duplicate definitions shall be detected, too.

Any ideas how to handle this best?

change "missing up reference" to concrete level name

HTML output too large

The HTML output is too large. If 800 items are added to 3 levels, then users complain about long loading times.

Furthermore, it is not possible to find e.g. "ok items". The usability of the HTML file is low. Users want to click on the numbers in the summary.

make sure file references are always using /