nids-io/ampt-generator

Healthcheck packet generator for the AMPT passive network tools monitor

ampt-generator's Issues

Enable optional error email delivery notifications

http://flask.pocoo.org/docs/dev/logging/#email-errors-to-admins

Should have an empty default ADMINS list, allow it to be overridden in config, and enable conditional SMTPHandler delivery with default/overridable SMTP server settings.

Add version output support

Add support to AMPT Generator commands to report version.

Embed Gunicorn in server and drop development server

Similar to how implemented in ampt-manager (ref), replace the development server with an embedded Gunicorn standalone server.

Normalize loggers in project

Inconsistent logger usage in various parts of application. Some functions using new root logger and some using Flask default logger, and configurations applied inconsistently between them.

Need to settle this out.

Take care of unclean shutdown with Ctrl+C KeyboardInterrupt

$ sudo ./venv.d/ampt-generator/bin/ampt-server 
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/popen_fork.py", line 29, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

A couple of exception handlers for KeyboardInterrupt have improved the situation, but need to figure out how to address a keyboard interrupt causing a messy looking traceback on exit.

Possible resources:

• http://jtushman.github.io/blog/2014/01/14/python-|-multiprocessing-and-interrupts/
• https://stackoverflow.com/questions/11312525/catch-ctrlc-sigint-and-exit-multiprocesses-gracefully-in-python
• https://stackoverflow.com/questions/21104997/keyboard-interrupt-with-pythons-multiprocessing

Too many open files exception in TaskRunner / generate_packet()

Exception raised after being alive for a few hours, possible resource leaks:

2017-10-10 11:30:00,749: AppServer (pid: 32413) - authenticated dispatch request from 127.0.0.1 with valid HMAC                                                                                             [0/772]
127.0.0.1 - - [10/Oct/2017 11:30:00] "GET /api/generate_probe?dest_addr=10.0.1.1&dest_port=5471&src_port=8573&proto=tcp&ts=1507660200.739729&h=8954e089e38b81b4ed31a0c7125b9086cc853565eb2723d21995a4f3d80803ed HTT
P/1.1" 200 -
Process TaskRunner:
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/DarrenSpruell/devel/external/ampt-generator/ampt_generator/cli.py", line 125, in ampt_gen_taskrunner
    packetgen.generate_packet(**packet)
  File "/Users/DarrenSpruell/devel/external/ampt-generator/ampt_generator/packetgen.py", line 41, in generate_packet
    send(packet)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/sendrecv.py", line 255, in send
    __gen_send(conf.L3socket(*args, **kargs), x, inter=inter, loop=loop, count=count,verbose=verbose, realtime=realtime)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/sendrecv.py", line 238, in __gen_send
    s.send(p)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/arch/pcapdnet.py", line 457, in send
    sx = bytes(cls()/x)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/packet.py", line 289, in __bytes__
    return self.build()
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/packet.py", line 357, in build
    p = self.do_build()
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/packet.py", line 346, in do_build
    pkt = self.self_build()
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/packet.py", line 337, in self_build
    p = f.addfield(self, p, val)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/fields.py", line 79, in addfield
    return s+struct.pack(self.fmt, self.i2m(pkt,val))
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/layers/l2.py", line 95, in i2m
    return MACField.i2m(self, pkt, self.i2h(pkt, x))
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/layers/l2.py", line 89, in i2h
    x = conf.neighbor.resolve(pkt,pkt.payload)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/layers/l2.py", line 38, in resolve
    return self.resolvers[k](l2inst,l3inst)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/layers/inet.py", line 855, in <lambda>
    conf.neighbor.register_l3(Ether, IP, lambda l2, l3: getmacbyip(l3.dst))
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/layers/l2.py", line 73, in getmacbyip
    nofilter=1)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/sendrecv.py", line 385, in srp1
    a,b=srp(*args,**kargs)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/sendrecv.py", line 368, in srp
    a,b=sndrcv(s ,x,*args,**kargs)
  File "/Users/DarrenSpruell/venv.d/ampt-generator/lib/python3.6/site-packages/scapy/sendrecv.py", line 73, in sndrcv
    rdpipe,wrpipe = os.pipe()
OSError: [Errno 24] Too many open files