nids-io/ampt-manager

Management service for the AMPT passive network tools monitor

ampt-manager

Management service for the AMPT passive network tools monitor

AMPT is a practical framework designed to aid those who operate network IDS sensors and similar passive security monitoring systems. A tailored approach is needed to actively monitor the health and functionality of devices that provide a service based on capturing and inspecting network traffic. AMPT supports these types of systems by allowing operators to validate traffic visibility and event logging on monitored network segments. Examples of systems that can benefit from this type of monitoring are:

• Suricata IDS
• Snort IDS
• Bro IDS
• Moloch

ampt-manager is the core component in the AMPT framework. It is simple to deploy and provides the following:

• Web-based management console
• Central point for configuration and management of AMPT nodes, including:
  • Monitored network segments
  • AMPT generator nodes
  • AMPT monitor instances
• State of network visibility from the standpoint of monitored segments
• Logging and accounting of events related to monitoring process
• Configurable alerting/notifications when monitors for configured segments encounter degraded visibility

Other AMPT components include:

• ampt-generator - Health check packet generator for the AMPT passive network tools monitor
• ampt-monitor - Sensor alert monitor core package for the AMPT passive network tools monitor

Installation and usage

See the Wiki for further documentation.

ampt-manager's Issues

Add version output support

Add subcommand to ampt-manager to report version.

Support proper data collection for multiple sensors per monitored segment

Current event logic for tracking dispatched/received/rate on monitored segments works when a single sensor (say, a Suricata box) is monitoring the segment. But in cases where multiple sensors monitor traffic for a segment, or when a packet crosses other monitored segments en route to the target segment, multiple copies of the probe packet will be picked up by deployed monitors and reported to the manager, leading to an imbalance in dispatched/received events (one dispatched probe will be reported by multiple monitors).

Possible solutions:

1. Configure multiple monitored segments per physical network segment, corresponding to each sensor that should report an event for the segment. For example, instead of monitoring DMZ, segments could be set up for DMZ-Suricata and DMZ-Bro.
2. Introduce differentiator per segment:
   2a. One option is to modify each monitored segment to use different Probe Generators (e.g. multiple instances running on different ports) that are configured to use distinct payloads per monitored segment. ampt-generator would have to be modified to support multiple instances on different ports and config options per instance allowing to override the payload in the signature.
   2b. Another option is to modify each monitored segment to specify unique probe destination ports for the Probe Generators to use when crafting packets. This would additionally require introducing a configuration option per ampt-monitor likely at the plugin level) to specify the destination port that monitor will send events for. In this way, the event monitors will handle their respective probes but ignore probes intended for other sensors on the same physical segment (but corresponding to different Monitored Segments in ampt-manager).
3. Potentially modify ampt-manager so that per-segment stats are tracked, but that a concept of a segment group exists and some aggregate view of the health of the segment relative to all sensor segments can be viewed.

Address csrf_enabled FlaskWTFDeprecationWarning

ampt-manager/ampt_manager/web/views.py:302: FlaskWTFDeprecationWarning: "csrf_enabled" is deprecated and will be removed in 1.0. Set "meta.csrf" instead.
  form = ReceivedProbeLogForm(csrf_enabled=False)   

Standardize and normalize logging

As currently implemented, logging in the application is a mish-mash and haphazardly results in different log formats from various packages and modules intermixed in the log stream.

e.g. the first lines at startup are from the app, using one format, and subsequent lines are from Gunicorn using a different format.

$ ./venv.d/ampt-manager/bin/ampt-manager run -l info ./ampt/ampt_manager.conf
2017-09-18 20:01:58,646: [INFO] Starting AMPT Manager
2017-09-18 20:01:58,646: [INFO] Configuring server for TLS using PROTOCOL_TLSv1_2 and cipher set TLSv1 (OpenSSL 1.0.2l  25 May 2017)
[2017-09-18 20:01:58 -0700] [12228] [INFO] Starting gunicorn 19.7.1
[2017-09-18 20:01:58 -0700] [12228] [INFO] Listening at: https://127.0.0.1:8443 (12228)
[2017-09-18 20:01:58 -0700] [12228] [INFO] Using worker: sync
[2017-09-18 20:01:58 -0700] [12231] [INFO] Booting worker with pid: 12231
[2017-09-18 20:01:58 -0700] [12232] [INFO] Booting worker with pid: 12232
[2017-09-18 20:01:58 -0700] [12233] [INFO] Booting worker with pid: 12233
[2017-09-18 20:01:58 -0700] [12234] [INFO] Booting worker with pid: 12234
[2017-09-18 20:01:58 -0700] [12235] [INFO] Booting worker with pid: 12235
...
[2017-09-18 20:01:33 -0700] [11155] [INFO] Handling signal: int
[2017-09-18 20:01:34 -0700] [11160] [INFO] Worker exiting (pid: 11160)
[2017-09-18 20:01:34 -0700] [11161] [INFO] Worker exiting (pid: 11161)
[2017-09-18 20:01:34 -0700] [11159] [INFO] Worker exiting (pid: 11159)
[2017-09-18 20:01:34 -0700] [11158] [INFO] Worker exiting (pid: 11158)
[2017-09-18 20:01:34 -0700] [11162] [INFO] Worker exiting (pid: 11162)
[2017-09-18 20:01:34 -0700] [11155] [INFO] Shutting down: Master

e.g. a resize of the terminal the manager is running in drops this INFO logging:

[2017-09-18 18:02:52 -0700] [11155] [INFO] Handling signal: winch                                                      

e.g. logging sent to stdio logging from Gunicorn doesn't go to the ampt.log.

Add better logging / error handling for 403 FORBIDDEN responses from ampt-generator

ampt-generator returns 403 errors with informative payloads upon failures validating ampt-manager requests. Improve error handling by catching and logging these.

(Should probably also capture dispatch error conditions in event logging to expose it in the manager UI/etc.).