telekom-security/ewsposter

collect logs and alerts from 27 honeypots and send it to backed (eg peba, geba), hpfeeds, influxdb or jSON file.

EWSPOSTER

EWSPoster is a tool, written in Python to, to collect logs and alers from differents honeypots (eq Glastopf v3, Dionaea, Honeytrap, eMobility, Conpot, Cowrie, Elasticpot, Rdpy, Mailoney, Vnclowpot, Heralding, Ciscoasa, Tanner, Snare, Glutton, Honeysap, Adbhoney, Ipphoney, Dicompot, Medpot, Honeypy, Citrixhoneypot, redishoneypot, endlessh), sentrypeer, log4pot also network IDS (eg Suricata, Fatt) and transmit them to InfluxDb, JSON, Hpfeed or an Honeypot backend (eg Peba or Geba).

Requirements

You need to install the libarys list in requirements.txt

pip3 install -r requirements.txt

Usage

Take a look at the usage text.

./ews.py -h
usage: ews.py [-h] [-c CONFIGPATH] [-v] [-d] [-l LOOP]
          [-m {glastopfv3,dionaea,honeytrap,emobility,conpot,cowrie,elasticpot,suricata,rdpy,mailoney,
               vnclowpot,heralding,ciscoasa,tanner,glutton,honeysap,adbhoney,fatt,ipphoney,dicompot,
               medpot,honeypy,citrix,redishoneypot,endlessh,sentrypeer,log4pot}]
          [-s] [-i] [-S] [-E] [-j JSONPATH] [-L SENDLIMIT] [-V]

optional arguments:
   -h, --help                                  show this help message and exit
   -c CONFIGPATH, --configpath CONFIGPATH      Load configuration file from Path
   -v, --verbose                               set output verbosity
   -d, --debug                                 set output debug
   -l LOOP, --loop LOOP                        endless loop. Set {xx} for seconds to wait for next loop
   -m, --modul {glastopfv3, dionaea,           only send alerts for this modul
               honeytrap, emobility,
               conpot, cowrie, elasticpot,
               suricata, rdpy, mailoney,
               vnclowpot, heralding,
               ciscoasa, tanner, glutton,
               honeysap, adbhoney, fatt,
               ipphoney, dicompot, medpot,
               honeypy, citrix, redishoneypot,
               endlessh, sentrypeer, log4pot}
   -s, --silent                                silent mode without output
   -i, --ignorecert                            ignore certificate warnings
   -S, --sendonly                              only send unsend alerts
   -E, --ewsonly                               only generate ews alerts files
   -j JSONPATH, --jsonpath JSONPATH            write JSON output file to path
   -L SENDLIMIT, --sendlimit SENDLIMIT         set {xxx} for max alerts will send in one session
   -V, --version                               show the EWS Poster Version

Configuration

Take a look at the example ews.cfg.default and copy it via

cp ews.cfg.default ews.cfg

ewsposter's Issues

Cowrie logs error

Ewsposter seems to have some hickups with the latest Cowrie:

EWS Poster v1.20 (c) by Markus Schroer <[email protected]>
 => Create lock socket successfull.
 => ESend: checking spooldir and resend alert
    -> [INFO] No jobs to send in spooldir: /opt/ewsposter/spool/.
 => Starting DIONAEA Honeypot Modul.
 => Starting HONEYTRAP Honeypot Modul.
    -> Calculate MD5Sum for payload files and rename files.
 => Starting CONPOT Honeypot Modul.
 => Starting COWRIE Honeypot Modul.
Traceback (most recent call last):
  File "/opt/ewsposter/ews.py", line 1172, in <module>
    eval(honeypot + '()')
  File "<string>", line 1, in <module>
  File "/opt/ewsposter/ews.py", line 930, in cowrie
    cowrieSessions[sid]['version'] = re.search(r"b'(.*)'", line["version"], re.M).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

I sent you the logs via Slack.

Cowrie log bugs

There are bugs in ews.py, so it does not parse the input commands.

Before:
if line['eventid'] == 'cowrieSession.command.input' and line['session'] in cowrieSessions:
It should be:
if line['eventid'] == 'cowrie.command.input' and line['session'] in cowrieSessions:

But it still only records one command, which in this case will be the last command inputted. So we need to add conditions to accommodate multiple commands.

HPFeeds stops working if EWS is disabled

EWSPoster version: dtagdevsec/ewsposter:2006 (Docker Image)
T-Pot version: 20.06

Expected behaviour: If ews is disabled but hpfeed enabled, then alerts are posted to hpfeeds
Actual behaviour: hpfeeds receives no messages

ews.cfg

[MAIN]
homedir = /opt/ewsposter/
spooldir = /opt/ewsposter/spool/
logdir = /opt/ewsposter/log/
del_malware_after_send = false
send_malware = false
sendlimit = 5000
contact = your_email_address
proxy = None
ip_int = None
ip_ext = None

[EWS]
ews = false
username = community-01-user
token = **********
rhost_first = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
rhost_second = https://community.sicherheitstacho.eu/ews-0.1/alert/postSimpleMessage
ignorecert = false

[HPFEED]
hpfeed = %(EWS_HPFEEDS_ENABLE)s
host = %(EWS_HPFEEDS_HOST)s
port = %(EWS_HPFEEDS_PORT)s
channels = %(EWS_HPFEEDS_CHANNELS)s
ident = %(EWS_HPFEEDS_IDENT)s
secret= %(EWS_HPFEEDS_SECRET)s
# path/to/certificate for tls broker - or "false" for non-tls broker
tlscert = %(EWS_HPFEEDS_TLSCERT)s
# hpfeeds submission format: "ews" (xml) or "json"
hpfformat = %(EWS_HPFEEDS_FORMAT)s

[EWSJSON]
json = false
jsondir = /data/ews/json/

[INFLUXDB]
influxdb = false
host = http://localhost
port = 8086
username = <your username for influx 1.8>
password = <your password for influx 1.8>
token = <your token for influx 2.0>
bucket = <your bucket/database for 2.0/1.8>
org = <your org for influx 2.0>

...

tpot.yml

# Ewsposter service
  ewsposter:
    container_name: ewsposter
    restart: always
    networks:
     - ewsposter_local
    environment:
     - EWS_HPFEEDS_ENABLE=true
     - EWS_HPFEEDS_HOST=161.xx.xx.xx
     - EWS_HPFEEDS_PORT=10000
     - EWS_HPFEEDS_CHANNELS=spam
     - EWS_HPFEEDS_IDENT=xxxx
     - EWS_HPFEEDS_SECRET=xxxxx
     - EWS_HPFEEDS_TLSCERT=false
     - EWS_HPFEEDS_FORMAT=json
    env_file:
     - /opt/tpot/etc/compose/elk_environment
    image: "dtagdevsec/ewsposter:2006"
    volumes:
     - /data:/data
     - /data/ews/conf/ews.ip:/opt/ewsposter/ews.ip
     - /data/ews/conf/ews.cfg:/opt/ewsposter/ews.cfg

log

Oct 05 23:39:26 tallopen docker-compose[94604]: ewsposter              |  => ESend: checking spooldir and resend alert
Oct 05 23:39:26 tallopen docker-compose[94604]: ewsposter              |     -> [INFO] No jobs to send in spooldir: /opt/ewsposter/spool/.
Oct 05 23:39:26 tallopen docker-compose[94604]: ewsposter              |  => Starting DIONAEA Honeypot Modul.
Oct 05 23:39:26 tallopen docker-compose[94604]: ewsposter              |  => Starting HONEYTRAP Honeypot Modul.
Oct 05 23:39:26 tallopen docker-compose[94604]: ewsposter              |     -> Calculate MD5Sum for payload files and rename files.
Oct 05 23:39:26 tallopen docker-compose[94604]: ewsposter              |  => Starting CONPOT Honeypot Modul.
Oct 05 23:39:26 tallopen docker-compose[94604]: ewsposter              |  => Starting COWRIE Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting ELASTICPOT Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting RDPY Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting MAILONEY Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting HERALDING Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting CISCOASA Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting TANNER Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting HONEYSAP Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting ADBHONEY Honeypot Modul.
Oct 05 23:39:41 tallopen docker-compose[94604]: ewsposter              |  => Starting FATT Honeypot Modul.
Oct 05 23:39:49 tallopen docker-compose[94604]: ewsposter              |     -> Sendlimit (5000) for Honeypot FATT reached. Skip.
Oct 05 23:39:49 tallopen docker-compose[94604]: ewsposter              |  => Starting MEDPOT Honeypot Modul.
Oct 05 23:39:49 tallopen docker-compose[94604]: ewsposter              |  => Starting HONEYPY Honeypot Modul.
Oct 05 23:39:49 tallopen docker-compose[94604]: ewsposter              |  => Starting CITRIX Honeypot Modul.
Oct 05 23:39:49 tallopen docker-compose[94604]: ewsposter              |  => Sleeping for 46 seconds ...

tanner log post body

currently tanner does not transmit the data from the http request's post body through ewsposter.
Please review the logs and see if they contain the post body for post requests and add them to ewsposter reassembledReq.

Mailoney commands.log

As described here, if the Mailoney commands.log is not present, which is the case if nothing was detected, ewsposter seems to stop working.

EWSPoster crashes with Tanner

Some conditions lead to crashing ewsposter:

 => Starting Tanner Modul.
 => Send Limit is set to : 500. Adapting to limit!
Traceback (most recent call last):
  File "/opt/ewsposter/ews.py", line 2610, in <module>
    eval(i+'()')
  File "<string>", line 1, in <module>
  File "/opt/ewsposter/ews.py", line 2436, in tanner
    REQUEST["raw"] = base64.b64encode(reassembledReq.encode('ascii')).decode()
UnicodeEncodeError: 'ascii' codec can't encode characters in position 459-462: ordinal not in range(128)

I sent you the logs via eMail.

Support new / updated honeypots

Please add support for the following honeypots:

• IPPhoney
• Dicompot
• Updated Elasticpot
• Medpot
• Honeypy

Thanks