openscap/openscap-daemon

Manages continuous scans of your infrastructure

OpenSCAP

Open Source Security Compliance Solution

About

The oscap program is a command line tool that allows users to load, scan, validate, edit, and export SCAP documents.

• Homepage of the project: www.open-scap.org
• User Manual: OpenSCAP User Manual
• Compilation, testing and debugging: OpenSCAP Developer Manual
• For new contributors: How to contribute

Contributing

We welcome all contributions to the OpenSCAP project. If you would like to contribute, either by fixing existing issues or adding new features, please check out our contribution guide to get started. If you would like to discuss anything, ask questions, or if you need additional help getting started, you can either send a message to our libera.chat IRC channel, #openscap, or to our mailing list.

Microsoft Windows Support

The Microsoft Windows support is officially void as of Febuary 1, 2022.

Use cases

SCAP Content Validation

• The following example shows how to validate a given source data stream; all components within the data stream are validated (XCCDF, OVAL, OCIL, CPE, and possibly other components):

oscap ds sds-validate scap-ds.xml

Scanning

• To evaluate all definitions within the given OVAL Definition file, run the following command:

oscap oval eval --results oval-results.xml scap-oval.xml

where scap-oval.xml is the OVAL Definition file and oval-results.xml is the OVAL Result file.

• To evaluate all definitions from the OVAL component that are part of a particular data stream within a SCAP data stream collection, run the following command:

oscap oval eval --datastream-id ds.xml --oval-id xccdf.xml --results oval-results.xml scap-ds.xml

where ds.xml is the given data stream, xccdf.xml is an XCCDF file specifying the OVAL component, oval-results.xml is the OVAL Result file, and scap-ds.xml is a file representing the SCAP data stream collection.

• To evaluate a specific profile in an XCCDF file run this command:

oscap xccdf eval --profile Desktop --results xccdf-results.xml --cpe cpe-dictionary.xml scap-xccdf.xml

where scap-xccdf.xml is the XCCDF document, Desktop is the selected profile from the XCCDF document, xccdf-results.xml is a file storing the scan results, and cpe-dictionary.xml is the CPE dictionary.

• To evaluate a specific XCCDF benchmark that is part of a data stream within a SCAP data stream collection run the following command:

oscap xccdf eval --datastream-id ds.xml --xccdf-id xccdf.xml --results xccdf-results.xml scap-ds.xml

where scap-ds.xml is a file representing the SCAP data stream collection, ds.xml is the particular data stream, xccdf.xml is ID of the component-ref pointing to the desired XCCDF document, and xccdf-results.xml is a file containing the scan results.

Document generation

• without XCCDF rules

oscap xccdf generate guide XCCDF-FILE > XCCDF-GUIDE-FILE

• with XCCDF rules

oscap xccdf generate guide --profile PROFILE XCCDF-FILE > XCCDF-GUIDE-FILE

• generate report from scanning

oscap xccdf generate report XCCDF-RESULT-FILE > XCCDF-REPORT-FILE

openscap-daemon's Issues

Automatically prune old results

Provide settings in tasks for how many results it should keep and/or how old results it should keep.

Integration test with a VM or container

The test would take the working copy, tarball it and install it in a VM or container. Then something like to following would be run:

python setup.py install

oscapd-cli task
# should cause dbus error

systemctl enable oscapd
systemctl start oscapd

# no error with no tasks specified
oscapd-cli task

oscapd-cli task_create -i
# put the necessary values in stdin to make the interactive task create finish

# check that the task exists and is saved
oscapd-cli task 1
oscapd-cli result 1 # should be empty

systemctl restart oscapd
# even after restart of the service
oscapd-cli task 1
oscapd-cli result 1 # should be empty

# this should fail, the task is disabled!
oscapd-cli task 1 run

oscapd-cli task 1 enable
# this should run fine
oscapd-cli task 1 run

# should have 1 result
oscapd-cli result 1
oscapd-cli result 1 1
oscapd-cli result 1 1 arf
oscapd-cli result 1 1 html
oscapd-cli result 1 1 stdout
oscapd-cli result 1 1 stderr
oscapd-cli result 1 1 exit_code

oscapd-cli result 1 1 remove

# this should run fine
oscapd-cli task 1 run
oscapd-cli result 1 remove

# and so on...

This will ensure we don't break basic functionality when developing.

PS: I know beaker would be pretty good for this test but I'd like to have an upstream test that all devs can run on their machines. If that turns out to be too difficult we can resort to beaker.

Make sure we are using the correct FSF address everywhere

Found by @isimluk

Also there are files in cve_scanner sub-dir which are LGPLv2+. Note that
FSF address is incorrect in them (not a blocker).

Unify the two configuration classes

The image-scanner and openscap-daemon projects merged together and due to time constraints we have made some compromises.

One of them is the existence of applicationconfiguration next to config.Configuration. The code should be refactored to only use the latter. The hard work of removing the singleton has already been done by @baude in #13. The next step is passing self.system.config in the dbus_daemon class instead of letting the cve_scanner.Worker construct the applicationconfiguration instance.

How will we update SPC images on Docker hub?

Currently, we have on our account on Docker hub two images of super privileged containers. One of them is based on Fedora 23, second ins based on Fedora 22. See https://hub.docker.com/r/openscap/

I would like to ask all of you what are your opinions:
How often we will update these images?
Should they be updated each time after openscap-daemon is updated in Fedora? Or openscap itself?
Also when our Dockerfiles change in upstream?
Will we do it manually or should a script or a Jenkins job be created to automate the task?

Thank you.

Add virtual machine scanning support

See oscap_helpers.py, function get_evaluation_args. This function figures out what daemon should call to evaluate given EvaluationSpec. Right now it supports localhost, ssh://, docker-image:// and docker-container://. It calls oscap, oscap-ssh and oscap-docker respectively.

Since oscap-vm was merged into OpenSCAP recently it makes sense to add vm-storage:// and vm-domain://.

Support Python3

oscapd service should not require docker

Hello!
I have a fresh minimal setup of Fedora 23. I have installed openscap-daemon-0.1.1-4.fc23 from here: http://koji.fedoraproject.org/koji/buildinfo?buildID=712797. I have tried to start OpenSCAP daemon service, but I have found the service depends on docker service:

[root@localhost ~]# systemctl start oscapd
Failed to start oscapd.service: Unit docker.service failed to load: No such file or directory.

That is because oscapd.service requires docker:

[root@localhost ~]# cat /usr/lib/systemd/system/oscapd.service
[Unit]
Description=OpenSCAP Daemon
Documentation=http://open-scap.org/tools/openscap-daemon
After=docker.service
Requires=docker.service

As from I understand, it should be able to run without docker installed.

oscapd-cli should not traceback when a prameter is missing on the command line

For example here, I forgot to specify the title and I got a traceback:

[root@localhost ~]# oscapd-cli task 1 set-title 
Traceback (most recent call last):
  File "/usr/bin/oscapd-cli", line 742, in <module>
    main()
  File "/usr/bin/oscapd-cli", line 728, in main
    cli_task(dbus_iface, task_accessor, args)
  File "/usr/bin/oscapd-cli", line 251, in cli_task
    dbus_iface, args.task_action, args.task_id, args.parameters[0]
  File "/usr/bin/oscapd-cli", line 92, in eval
    casted_args = TaskAccessor.get_string(args)
  File "/usr/bin/oscapd-cli", line 71, in get_string
    raise ValueError("Not expected parameters")
ValueError: Not expected parameters

the same for oscapd-cli task 1 set-input, oscapd-cli task 1 set-target etc.

I think that the ValueError exception should be catched and displayed a message to user.
I have recent deamon from git on Fedora 23 Python3.

`oscapd-cli cve-scan`

The daemon has dbus methods that can scan containers for CVEs. This is used by atomic scan. It would be helpful to support this in the oscapd-cli tool as well for easier testing.

When pushing SPC openscap-daemon Docker images to Docker hub, please sign them

Docker starting from 1.8 introduced concept of Docker Content Trust (The Update Framework):

• https://blog.docker.com/2015/08/content-trust-docker-1-8/

which allows images to be signed when publishing to Docker hub. The consumers can later verify the producer of these images (prevent also image forgery, image replay attacks etc.)

We should start using this functionality when creating openscap/openscap-daemon-* SPC containers:

• https://hub.docker.com/r/openscap/openscap-daemon-f23/
• https://hub.docker.com/r/openscap/openscap-daemon-f22/

This is more RFE, than a real bug.

oscapd-cli traceback without any arguments

Hello, I use openscap-daemon-0.1.4-1.fc23.noarch from Fedora 23 updates testing repository.
and I have seen a traceback when I omitted -i option:

[root@thinkpad jcerny]# oscapd-cli task-create
Traceback (most recent call last):
  File "/usr/bin/oscapd-cli", line 787, in <module>
    main()
  File "/usr/bin/oscapd-cli", line 775, in main
    cli_task_create(dbus_iface, args)
  File "/usr/bin/oscapd-cli", line 386, in cli_task_create
    raise NotImplementedError("Not yet!")
NotImplementedError: Not yet!

oscapd-cli help should provide list of possible actions

I have recent daemon from git on Fedora 23, Python3.

I think that when I want to show help for a particular module, it should give me a list of possible actions. But now it shows nothing.

[root@localhost ~]# oscapd-cli task -h
usage: oscapd-cli task [-h] [-f] [-r]
                       [TASK_ID] [ACTION] [actions [actions ...]]

positional arguments:
  TASK_ID               ID of the task to display, if none is provided a
                        summary of all tasks is displayed
  ACTION
  actions               parameters

optional arguments:
  -h, --help            show this help message and exit
  -f, --force           remove task without confirmation
  -r, --remove-results  remove with results

I would expect there possible values of the ACTION argument.

oscapd-cli task-create -i (ERROR EXCEPTION)

During the creation of new task , i have this result issue :

[root@localhost Bureau]# oscapd-cli task-create -i 
Creating new task in interactive mode
Title: task
Target (empty for localhost): 
Found the following SCAP Security Guide content: 
    1:  /usr/share/xml/scap/ssg/content/ssg-firefox-ds.xml
    2:  /usr/share/xml/scap/ssg/content/ssg-jre-ds.xml
    3:  /usr/share/xml/scap/ssg/content/ssg-rhel6-ds.xml
    4:  /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml
Choose SSG content by number (empty for custom content): 3
Tailoring file (absolute path, empty for no tailoring): 
Found the following possible profiles: 
Traceback (most recent call last):
  File "/usr/bin/oscapd-cli", line 787, in <module>
    main()
  File "/usr/bin/oscapd-cli", line 775, in main
    cli_task_create(dbus_iface, args)
  File "/usr/bin/oscapd-cli", line 313, in cli_task_create
    input_file, tailoring_file
  File "/usr/lib/python2.6/site-packages/dbus/proxies.py", line 140, in __call__
    **keywords)
  File "/usr/lib/python2.6/site-packages/dbus/connection.py", line 630, in call_blocking
    message, timeout)
dbus.exceptions.DBusException: org.freedesktop.DBus.Python.AttributeError: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/dbus/service.py", line 702, in _message_cb
    retval = candidate_method(self, *args, **keywords)
  File "/usr/lib/python2.6/site-packages/openscap_daemon/dbus_daemon.py", line 78, in GetProfileChoicesForInput
    input_file, tailoring_file
  File "/usr/lib/python2.6/site-packages/openscap_daemon/system.py", line 86, in get_profile_choices_for_input
    input_file, tailoring_file
  File "/usr/lib/python2.6/site-packages/openscap_daemon/oscap_helpers.py", line 96, in get_profile_choices_for_input
    input_tree, "http://checklists.nist.gov/xccdf/1.1", ret
  File "/usr/lib/python2.6/site-packages/openscap_daemon/oscap_helpers.py", line 82, in scrape_profiles
    for elem in tree.iter("{%s}Profile" % (namespace)):
AttributeError: ElementTree instance has no attribute 'iter'

Could you give some help , thank you

Timestamp support for tasks and results

Use the created and modified timestamps from the filesystems. For task check the X.xml spec file, for results use the exit_code file, that is the only file guaranteed to be created.

Add dbus methods to query these times, show them in oscapd-cli.

Daemon stops respond after umount error

Sorry, I still don't have enough time to provide more useful information.

While testing, it often(but not always) stops at

# atomic scan --all

It stops responds and it is not killable by ctrl+c (its very annoying!). When I close terminal and try to repeat command # atomic scan --all, I will get

ERROR:dbus.proxies:Introspect error on :1.1837:/OpenSCAP/daemon: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

journalctl -u oscapd 

Nov 26 10:31:29 zmoravec-thinkpad oscapd[16330]: DEBUG:Scanning chroot 95338760b9fb132fe29d6b9369a2333ea48ec8b71ef0e688d078a765aeac60a6
Nov 26 10:31:29 zmoravec-thinkpad oscapd[16330]: DEBUG:Starting evaluation with command 'oscap oval eval --report /var/tmp/image-scanner/reports/95338760b9fb132fe29d6b9369a2333ea48ec8b71ef0e688d078a765aeac60a6.html --results /var/tmp/image-scanner/reports/95338760b9fb132fe29d6b9369a2333ea48ec8b71ef0e688d078a765aeac60a6.xml /var/tmp/image-scanner/com.redhat.rhsa-RHEL7.xml.bz2'.
Nov 26 10:31:31 zmoravec-thinkpad oscapd[16330]: DEBUG:"POST /run/docker.sock/v1.17/commit?container=844af2ebf856b595fee7de44f06276f08ed857eac8d880ffc7eeaeb6f476a7c0 HTTP/1.1" 201 74
Nov 26 10:31:31 zmoravec-thinkpad oscapd[16330]: DEBUG:Scanned chroot for image 7ad37622e7b6a1747b3b61448bf747f05d3d84d1d7b04f83aa58d3c8d5b00829 completed in 1.77401304245 seconds
Nov 26 10:31:31 zmoravec-thinkpad oscapd[16330]: DEBUG:Scanned chroot for image 95338760b9fb132fe29d6b9369a2333ea48ec8b71ef0e688d078a765aeac60a6 completed in 1.89656019211 seconds
Nov 26 10:31:31 zmoravec-thinkpad oscapd[16330]: DEBUG:"POST /run/docker.sock/v1.17/containers/create HTTP/1.1" 201 90
Nov 26 10:31:31 zmoravec-thinkpad oscapd[16330]: DEBUG:"GET /run/docker.sock/v1.17/containers/77372c1ed2f51ed2274d5306b69b75c775c2bff6dcb813d6f0b55173c99573f3/json HTTP/1.1" 200 None
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: DEBUG:Created scanning chroot in 6.61336708069 seconds
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: INFO:844af2ebf856b595fee7de44f06276f08ed857eac8d880ffc7eeaeb6f476a7c0 is Fedora release 22 (Twenty Two)
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: Exception in thread 844af2ebf856b595fee7de44f06276f08ed857eac8d880ffc7eeaeb6f476a7c0:
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: Traceback (most recent call last):
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: File "/usr/lib64/python2.7/threading.py", line 813, in __bootstrap_inner
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: self.run()
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: File "/usr/lib64/python2.7/threading.py", line 766, in run
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: self.__target(*self.__args, **self.__kwargs)
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: File "/usr/lib/python2.7/site-packages/openscap_daemon/cve_scanner/cve_scanner.py", line 301, in search_containers
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: f.DM.unmount_path(f.dest)
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: File "build/bdist.linux-x86_64/egg/Atomic/mount.py", line 162, in unmount_path
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: raise ValueError(r.stderr)
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: ValueError: umount: /tmp/77372c1ed2f51ed2274d: target is busy
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: (In some cases useful info about processes that
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: use the device is found by lsof(8) or fuser(1).)
Nov 26 10:31:32 zmoravec-thinkpad oscapd[16330]: DEBUG:"DELETE /run/docker.sock/v1.17/containers/037537c89f5b93c2a12e?link=False&force=False&v=False HTTP/1.1" 204 0
Nov 26 10:31:33 zmoravec-thinkpad oscapd[16330]: DEBUG:"DELETE /run/docker.sock/v1.17/containers/36f8514e03f5a86d8f5b?link=False&force=False&v=False HTTP/1.1" 204 0

When it freezes, I can see usually something like that in mount

/dev/mapper/docker-253:2-2622353-2a0168f5e564072a8a40e59c402cefa872bcddc465c050069b4a484ae6403aaf on /tmp/2a0168f5e564072a8a40 type ext4 (ro,nosuid,nodev,relatime,context="system_u:object_r:svirt_sandbox_file_t:s0:c287,c651",stripe=16,data=ordered)

Devise a better error handling / exception strategy

Right now we let the raw exceptions bubble through. The daemon is exception safe and cleans up in case of errors but the reporting of the errors is more suitable for developers than users.

@ybznek suggested that we should wrap common errors in exception classes specific to the daemon and show better messages.

Examples:
When task of ID X is not found we show "Task X was not found." instead of "File /a/b/c/d/tasks/1.xml cannot be opened".

This task includes the research of how to handle this.

In current master 'oscapd' service fails to start with Python 2

# cat /etc/redhat-release 
Fedora release 23 (Twenty Three)
# git clone https://github.com/OpenSCAP/openscap-daemon.git
# cd openscap-daemon/
# python setup.py bdist_rpm
# rpm -i dist/openscap_daemon-0.1.1-1.noarch.rpm
# service oscapd restart
Redirecting to /bin/systemctl restart  oscapd.service
Warning: oscapd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Job for oscapd.service failed because the control process exited with error code. See "systemctl status oscapd.service" and "journalctl -xe" for details.

# systemctl status oscapd
● oscapd.service - OpenSCAP Daemon
   Loaded: loaded (/usr/lib/systemd/system/oscapd.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2015-12-22 17:46:01 CET; 33s ago
     Docs: http://open-scap.org/tools/openscap-daemon
  Process: 10167 ExecStart=/usr/bin/oscapd (code=exited, status=1/FAILURE)
 Main PID: 10167 (code=exited, status=1/FAILURE)

Dec 22 17:46:01 localhost.localdomain oscapd[10167]: from openscap_daemon.cve_scanner.cve_scanner import Worker
Dec 22 17:46:01 localhost.localdomain oscapd[10167]: File "/usr/lib/python2.7/site-packages/openscap_daemon/cve_scanner/cve_scanner.py", line 22, in <module>
Dec 22 17:46:01 localhost.localdomain oscapd[10167]: from openscap_daemon.cve_scanner.scan import Scan
Dec 22 17:46:01 localhost.localdomain oscapd[10167]: File "/usr/lib/python2.7/site-packages/openscap_daemon/cve_scanner/scan.py", line 28, in <module>
Dec 22 17:46:01 localhost.localdomain oscapd[10167]: from oscap_docker_python.get_cve_input import getInputCVE
Dec 22 17:46:01 localhost.localdomain oscapd[10167]: ImportError: No module named oscap_docker_python.get_cve_input
Dec 22 17:46:01 localhost.localdomain systemd[1]: oscapd.service: Main process exited, code=exited, status=1/FAILURE
Dec 22 17:46:01 localhost.localdomain systemd[1]: Failed to start OpenSCAP Daemon.
Dec 22 17:46:01 localhost.localdomain systemd[1]: oscapd.service: Unit entered failed state.
Dec 22 17:46:01 localhost.localdomain systemd[1]: oscapd.service: Failed with result 'exit-code'.
Warning: oscapd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
# which python
/usr/bin/python
# file /usr/bin/python
/usr/bin/python: symbolic link to python2

Current result: oscapd fails to start due some import error
Expected result: oscapd service started successfully

Fix remaining python3 errors

• iteritems, iterkeys, itervalues -> items, keys, values DONE
• raw_input -> input DONE
• utf8_strings argument in dbus_python DONE

Remove obsolete cve_scanner package

CVE scanning is now done as a task, the cve_scanner package is of no use. Let's remove it so that we don't have to maintain it any more.

Start of SPC cause 'Unable to start shell:/bin/bash'

This is probably not problem of our code, I only want to have this issue somewhere to open discussion.

I have the issue only last days. after I start our SPC, I am not able to start new terminal on my host, I will get only Unable to start shell:/bin/bash. Problem is still here when I kill docker.

When I use our LABEL RUN="docker run -dt --privileged --pid=host -v /etc/oscapd:/etc/oscapd -v /proc/:/hostproc/ -v /sys/fs/cgroup:/sys/fs/cgroup -v /var/log:/var/log -v /run:/run -v /var/lib/docker/devicemapper/metadata/:/var/lib/docker/devicemapper/metadata/ -v /dev/:/dev/ -v /var/tmp/image-scanner:/var/tmp/image-scanner --env container=docker --net=host --cap-add=SYS_ADMIN --ipc=host IMAGE" for clean fedora:23 image I have this problem too.

atomic run has some --spc option to start container, but Daemon doesn't work only with it - but I don't have the issue.

@jan-cerny said he saw the issue too

Thanks for advice. I will probably investigate it later, maybe the same issue can have more users - I don't know it, now.

fedora23, Docker version 1.9.1, build 7206621

Non-interactive task_create

Currently we only support interactive task creation because that's where the daemon helps users. Non-interactive task_create is possible via the dbus API but not via the CLI tool.

Use async API for CVE scans and one-off evaluations

CVE scans and one-off evaluations stall the dbus message queue which prevents any other request from reaching the daemon. We need to change to a different paradigm where the dbus call will schedule the action and return a token for the client to poll or cancel.

We will use the same scheduling mechanism that is used for continuous task evaluations.

The target date for this feature is 30th November, in time for 0.1.1 release.

This change will NOT break the dbus API already used by atomic scan. Instead it will introduce new methods and the old methods will stay in place for compatibility.

oscapd should be able to start even without Atomic is installed

Reproducer:

• try to install oscapd
• remove docker&atomic.
• Play. :)

Jenkins for this project

Is it practical to run current test suite in jenkins environment? What are the roadblocks?

Remove workaround locks

We have solved occasional tracebacks within atomic mount using locks. However this is only a workaround. This issue is a reminder that it is not a final solution.
See #58

Change COPR references to the openscap-latest COPR

@isimluk's COPR is no longer being maintained.

atomic scan of container creates useless images

Maybe it's bug of atomic, I haven't investigated it.
When I want to use atomic scan CONTAINER it creates temporary image but don't remove it.

Reproduce:

• see docker images
• do atomic scan container
• see docker images - you will see new image which you don' want

sudo docker images -q | sort > /tmp/old_images
sudo atomic scan 90fe1348ce60  # id of container
sudo docker images -q | sort > /tmp/new_images

$ diff /tmp/old_images /tmp/new_images
28a29
> e682a7bac56d

$ docker inspect e682a7bac56d
[
{
    "Id": "e682a7bac56d474645218b9f198dcc87e4717ed5de70befd521193b00fca7ca8",
    "Parent": "6883d5422f4ec2810e1312c0e3e5a902142e2a8185cd3a1124b459a7c38dc55b",
    "Comment": "",
    "Created": "2015-11-30T14:35:56.72295542Z",
    "Container": "90fe1348ce60a311ac5b3e203f60f509fea92d66a9028b61b06873e872d11b11",
    "ContainerConfig": {
        "Hostname": "90fe1348ce60",
        "Domainname": "",
        "User": "",
        "AttachStdin": true,
        "AttachStdout": true,
        "AttachStderr": true,
        "ExposedPorts": null,
        "PublishService": "",
        "Tty": true,
        "OpenStdin": true,
        "StdinOnce": true,
        "Env": [
            "container=docker",
            "PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin"
        ],
        "Cmd": [
            "/bin/sleep",
            "7d"
        ],
        "Image": "docker.io/rhel7",
        "Volumes": {
            "/run": {}
        },
        "VolumeDriver": "",
        "WorkingDir": "",
        "Entrypoint": null,
        "NetworkDisabled": false,
        "MacAddress": "",
        "OnBuild": null,
        "Labels": {}
    },
    "DockerVersion": "1.8.2-fc22",
    "Author": "",
    "Config": {
        "Hostname": "",
        "Domainname": "",
        "User": "",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "ExposedPorts": null,
        "PublishService": "",
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "container=docker",
            "PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin"
        ],
        "Cmd": [
            "/bin/sleep",
            "7d"
        ],
        "Image": "",
        "Volumes": {
            "/run": {}
        },
        "VolumeDriver": "",
        "WorkingDir": "",
        "Entrypoint": null,
        "NetworkDisabled": false,
        "MacAddress": "",
        "OnBuild": null,
        "Labels": {
            "io.projectatomic.Temporary": "true"
        }
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Size": 0,
    "VirtualSize": 201663804,
    "GraphDriver": {
        "Name": "devicemapper",
        "Data": {
            "DeviceId": "7492",
            "DeviceName": "docker-253:2-2622353-e682a7bac56d474645218b9f198dcc87e4717ed5de70befd521193b00fca7ca8",
            "DeviceSize": "10737418240"
        }
    }
}
]

Docker files are missing the label STOP

Atomic has stop command to stop running containers.
To use this feature, the Dockerfile has to contain LABEL STOP, that contains the command to stop the containerized application.

Missing configuration file

Our SPC based on F23 does not contain a config.ini file.
This command: atomic install openscap/openscap-daemon-f23
produces a message:
Installation complete. Be sure to customize /etc/oscapd/config.ini as needed.
However, no such file exists, so user can't customize it.
I think the message is little confusing then.

oscapd-cli can't parse its arguments on Fedora23

Hello!
I have openscap-daemon-0.1.1-4.fc23.noarch on Fedora 23. I can't run the oscapd-cli command. I get:

[root@localhost ~]# oscapd-cli
Traceback (most recent call last):
  File "/usr/bin/oscapd-cli", line 752, in <module>
    main()
  File "/usr/bin/oscapd-cli", line 698, in main
    if args.version:
AttributeError: 'Namespace' object has no attribute 'version'

I think it is wrong processing of object returned by parse_args() from argparse. Perhaps it is because Python 3 is used as interpreter.

"atomic scan --detail" shows bad OS images/containers

@jan-cerny found this while we were fixing some other issue

docker --images
...
docker.io/rhel7            latest              6c3a84d798dc        10 weeks ago        201.7 MB
...

 atomic scan --images --detail
...
6c3a84d798dc
  OS        : Fedora release 23 (Twenty Three)
  Moderate  : 4
...

Our tests doesn't cover it, because it wasn't implemented where tests were written.

oscapd-cli traceback when run with non-root privileges

oscapd-cli eval needs to be run as root. When run as normal user it produces a traceback.
I suggest to display a message instead. For example: "This command needs to be run as root."

Traceback:

$ oscapd-cli eval
ERROR:dbus.proxies:Introspect error on :1.115:/OpenSCAP/daemon: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 3 matched rules; type="method_call", sender=":1.124" (uid=1000 pid=5269 comm="/bin/python3 /usr/bin/oscapd-cli eval ") interface="org.freedesktop.DBus.Introspectable" member="Introspect" error name="(unset)" requested_reply="0" destination=":1.115" (uid=0 pid=4904 comm="/bin/python3 /usr/bin/oscapd ")
Traceback (most recent call last):
  File "/usr/bin/oscapd-cli", line 787, in <module>
    main()
  File "/usr/bin/oscapd-cli", line 752, in main
    dbus_iface.GetVersion()
  File "/usr/lib64/python3.4/site-packages/dbus/proxies.py", line 70, in __call__
    return self._proxy_method(*args, **keywords)
  File "/usr/lib64/python3.4/site-packages/dbus/proxies.py", line 145, in __call__
    **keywords)
  File "/usr/lib64/python3.4/site-packages/dbus/connection.py", line 651, in call_blocking
    message, timeout)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 2 matched rules; type="method_call", sender=":1.124" (uid=1000 pid=5269 comm="/bin/python3 /usr/bin/oscapd-cli eval ") interface="org.OpenSCAP.daemon.Interface" member="GetVersion" error name="(unset)" requested_reply="0" destination=":1.115" (uid=0 pid=4904 comm="/bin/python3 /usr/bin/oscapd ")

I am using deamon 0.1.6 on Fedora 23 with Python3, from git with head 21e884e

Provide a better description for the project

Something in README.md that downstreams can use when packaging.

Daemon fails to start due to missing Python requests module

I have run into issue that I wasn't able to start OpenSCAP Daemon using systemctl start oscapd on Fedora 22 Cloud. It seems to be dependent on Python's "requests" module. Here is a snippet from my journal:

Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com systemd[1]: Starting OpenSCAP Daemon...
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: Traceback (most recent call last):
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: File "/usr/bin/oscapd", line 22, in <module>
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: from openscap_daemon import dbus_daemon
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: File "/usr/lib/python2.7/site-packages/openscap_daemon/dbus_daemon.py", line 23, in <module>
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: from openscap_daemon.cve_scanner.cve_scanner import Worker
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: File "/usr/lib/python2.7/site-packages/openscap_daemon/cve_scanner/cve_scanner.py", line 23, in <module>
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: from openscap_daemon.cve_scanner.generate_summary import Create_Summary
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: File "/usr/lib/python2.7/site-packages/openscap_daemon/cve_scanner/generate_summary.py", line 27, in <module>
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: from openscap_daemon.cve_scanner.image_scanner_client import Client
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: File "/usr/lib/python2.7/site-packages/openscap_daemon/cve_scanner/image_scanner_client.py", line 37, in <module>
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: import requests
Jan 21 08:32:02 nec-em18.rhts.eng.bos.redhat.com oscapd[10409]: ImportError: No module named requests
Jan 21 08:32:03 nec-em18.rhts.eng.bos.redhat.com systemd[1]: oscapd.service: main process exited, code=exited, status=1/FAILURE
Jan 21 08:32:03 nec-em18.rhts.eng.bos.redhat.com systemd[1]: Failed to start OpenSCAP Daemon.

I use openscap-daemon-0.1.1-3.fc22.noarch.

As from I understand, the daemon itself should not depend on requests, only the client part should depend on it.

Better help for `oscapd-cli result`

Similar to #52

Security of OpenSCAP daemon features

Some parts of the daemon are harmless and we don't have to require root permissions to access it -- for example the CVE scan. Some parts are more or less privilege escalation devices, users can get root rights by injecting content with bash remediations. These remediations can do anything root can do, including giving some particular users root rights. This is obviously something we have to avoid!

These are some ways to solve this:

• The tasks can optionally drop rights before evaluating
  • technically this is very nice, users can evaluate using their own rights
  • unfortunately this is not very useful, most content requires root rights to be usable
• We can split the interface, allow anyone within some group to do all the unprivileged safe stuff and only let root do the dangerous stuff
  • what do we allow in the unprivileged interface besides cve scanning?
• We can require root for everything and then nothing is privilege escalation (this is what we do now)
• Combination of the above

Thoughts? I'd like to discuss this a little before committing to any direction.

@ybznek @baude @isimluk

Daemon fails to start when configuration file is invalid

If /etc/oscapd/config.ini contains some nonsense, the deamon fails to start and gives a traceback.
I think we should catch the exception, add an error message into journal and continue without trying to parse the configuration file.

Here is the traceback:

Starting OpenSCAP Daemon...
INFO:OpenSCAP Daemon 0.1.3
Started OpenSCAP Daemon.
DEBUG:Initialized AsyncManager, 4 workers
INFO:Loading configuration from '/etc/oscapd/config.ini'.
detected unhandled Python exception in '/usr/bin/oscapd'
Traceback (most recent call last):
File "/usr/bin/oscapd", line 70, in <module>
main()
File "/usr/bin/oscapd", line 64, in main
obj = dbus_daemon.OpenSCAPDaemonDbus(bus, config_file)
File "/usr/lib/python3.4/site-packages/openscap_daemon/dbus_daemon.py", line 42, in __init__
self.system = system.System(config_file)
File "/usr/lib/python3.4/site-packages/openscap_daemon/system.py", line 47, in __init__
self.config.load(config_file)
File "/usr/lib/python3.4/site-packages/openscap_daemon/config.py", line 154, in load
config.read(config_file)
File "/usr/lib64/python3.4/configparser.py", line 672, in read
self._read(fp, filename)
File "/usr/lib64/python3.4/configparser.py", line 1058, in _read
raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '/etc/oscapd/config.ini', line: 1 
'faoghaio\n'

Implement task result removal

# remove result of id Y of task of id X
$ oscapd-cli result X Y remove
# remove all results of task of id X
$ oscapd-cli result X remove

Both should ask for confirmation unless -f is specified.

Docker exception during scan

I have created ticket from pull request #57 (some comments are there)
"
Deamon should not die on an exception in docker client.

This exception was observed during atomic scan --images on Fedora23.

Addressing:

Exception in thread ed695b131d039f3608c0da4335572f2e1a0ef490b9b67664d046cd6c835655de:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/docker/client.py", line 138, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3.4/site-packages/requests/models.py", line 840, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localunixsocket/v1.20/containers/012d2e837331e17768b5f2df5
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.4/site-packages/openscap_daemon/cve_scanner/cve_scanner.py", line 280, in search_containers
f = Scan(image, cids, output, self.ac)
File "/usr/lib/python3.4/site-packages/openscap_daemon/cve_scanner/scan.py", line 53, in __init__
self.dm_results = self.DM.mount(image_uuid)
File "/usr/lib/python3.4/site-packages/Atomic/mount.py", line 302, in mount
cid, dev_name = self._get_cid_from_mountpoint(self.mountpoint)
File "/usr/lib/python3.4/site-packages/Atomic/mount.py", line 494, in _get_cid_from_mountpoint
graph = self.client.inspect_container(c)["GraphDriver"]
File "/usr/lib/python3.4/site-packages/docker/utils/decorators.py", line 21, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/usr/lib/python3.4/site-packages/docker/api/container.py", line 172, in inspect_container
self._get(self._url("/containers/{0}/json", container)), True
File "/usr/lib/python3.4/site-packages/docker/client.py", line 146, in _result
self._raise_for_status(response)
File "/usr/lib/python3.4/site-packages/docker/client.py", line 141, in _raise_for_status
raise errors.NotFound(e, response, explanation=explanation)
docker.errors.NotFound: 404 Client Error: Not Found ("b'no such id: 012d2e837331e17768b5f2df593a2a4471f8d5a87e7de35391eba2d35e6f4b24'")

"

oscapd-cli to support changing task properties

There are dbus calls to change task properties but they are not exposed in oscapd-cli. Changing task properties is pretty important because it's rare that users get all settings right the first time.

I propose:
oscapd-cli task X set-title "something" but I am open to suggestions.

These are the dbus methods:

    def SetTaskTitle(self, task_id, title):
    def SetTaskEnabled(self, task_id, enabled):
    def SetTaskTarget(self, task_id, target):
    def SetTaskInput(self, task_id, input_):
    def SetTaskTailoring(self, task_id, tailoring):
    def SetTaskProfileID(self, task_id, profile_id):
    def SetTaskOnlineRemediation(self, task_id, online_remediation):
    def SetTaskScheduleNotBefore(self, task_id, schedule_not_before_str):
    def SetTaskScheduleRepeatAfter(self, task_id, schedule_repeat_after):

There definitely are some missing and some of them lack getters. This issue should also track clean-up of that.

This will most likely grow oscapd-cli to a big size, we should think about refactoring it. Maybe moving some of the functionality into a module inside the openscap_daemon package.

Provide help for `oscapd-cli eval`

And improve it so that it's useful, right now it runs evaluation but never stores any results.

"oscapd-evaluate spec" traceback on Python 3

The code in bin/oscapd-evaluate uses .decode() method which is not available in Python3.

Please see the command that I am running and the resulting traceback:

[root@thinkpad openscap-daemon]# python3 bin/oscapd-evaluate spec --input tests/testing_data/ssg-fedora-ds.xml 
INFO:OpenSCAP Daemon one-off evaluator 0.1.3
INFO:Successfully imported 'docker' and 'Atomic.mount', container scanning enabled.
Traceback (most recent call last):
  File "bin/oscapd-evaluate", line 373, in <module>
    main()
  File "bin/oscapd-evaluate", line 239, in main
    spec.input_.set_contents(args.input_.read().decode("utf-8"))
AttributeError: 'str' object has no attribute 'decode'

This also causes that the test suite doesn't pass.

GObject.MainLoop, etc. is deprecated

On Fedora Rawhide(Python 3.5.1) I get

/usr/bin/oscapd:66: PyGIDeprecationWarning: GObject.MainLoop is deprecated; use GLib.MainLoop instead
  loop = gobject.MainLoop()

If somebody fix that, he/she should also test if now version of code is working also with Python 2.*

Provide a REST API

We already provide a dbus API but some prospective users would strongly prefer a REST API. We should refactor the System and DBusDaemon classes so that we can run one service that exposes 2 APIs at the same time.

Provide man pages for oscapd and oscapd-cli

Task removal

# remove task of ID X
$ oscapd-cli task X remove

If -r is provided it should also delete all results. Must refuse to delete task if it has any results.

Should ask for confirmation unless -f is specified.

Depends on #9

oscapd-cli run without any arguments gives a traceback on Python 3

Hello! I run oscapd-cli on Fedora23 without any argument.

[root@localhost ~]# oscapd-cli
Traceback (most recent call last):
  File "/usr/bin/oscapd-cli", line 750, in <module>
    main()
  File "/usr/bin/oscapd-cli", line 731, in main
    if args.action == "status":
AttributeError: 'Namespace' object has no attribute 'action'

I think it would give me some error message or usage or help, but no traceback.

Ability to run remote ssh scans with sudo privileges

I've been testing and it works fine on newer versions of openscap (those which includes --sudo option).

Adding --sudo to the evaluation args works fine, but we need all the program logic to decide whether to add --sudo to the args or not. And probably we need to store a flag to tell daemon when to run scans with sudo privileges.

ret.extend([config.oscap_ssh_path, '--sudo', host, str(port)])

Destination's machine mustn't require TTY to run sudo (scan fails if sudo requires a tty):

/etc/sudoers:
Defaults requiretty -> #Defaults requiretty
OR
Defaults: !requiretty

It would be a "nice to have" feature whether allow users to run scans with sudo privileges or without them.

API for getting overall status

This issue was originally created by @isimluk at the old repository called scap-client. Since it's valuable I have ported it to the new repository.

Hello Martin, here is an idead:

scap-client could provide API for getting some summary information about the scans results. It may return some details per tasks, but the main feature would be green/yellow/red type of information.

Green: If a periodic task is scheduled && the last result included passess && the last result did not include errors and fails.
Yellow: if no periodic task is scheduled
Red: Active task reports errors or fails.
Etc.

Maybe it might return different answers per tasks.

Thoughts?