Palo Alto Networks PAN-OS v8.1-10.0 Elastic Stack v7.x Configuration
There are several Palo Alto projects for ELK but most seem to be vacated with no updates in the past year. Also could not find any with PAN-OS 9 or 10+ expanded logs (SD-Wan).
Update existing projects to CIM and PAN 10.0 (Will work with PAN-OS 8+) Initial Updates from other projects:
- Support of ELK v7.8
- Added new fields for traffic logs that started with PAN-OS 9.1 and 10.0
- Changed attribute names from the default PA field names to Common Information Model (CIM) where applicable.
Allow you to import CIM Traffic and Threat visualizations - Added DNS filter to provide hostnames not just the IP including DNSMASQ install reference
- Converted all import objects to ndjson (ELK is migrating away from json)
- Import of indexes so visualizations match index UUID
- Added prune to pipeline to get ride of normaly null and duplicated fields.
Also helps when PA adds a new log field, no more COL# fields - Created panos-undefined index to capture logging of other types
- Added destination map to traffic dashboard
If you like the visualizations and dasboards, please buy me a coffee so I can keep going
https://www.patreon.com/gauthig
This project was built on Ubuntu 20.04, and adding the ELK repositories so that the ELK stack stays current. Instructions are provided for this OS base, ELK setup.
Once you have ELK up and running start here
- Download these folders as follows
- elk-pipeline - files that need to be on the elk server
- gui-import - files that will be imported via the kibana web gui
- Set your timezone correctly (Very important), also set you local server timezone so it is not UTC
- RAW Log
The RAW output from the Palo Alto is saved in each document in the message field. This is required
if you are on a PCI or other regulated firewall. This field is not parsed or indexed.
If you want to save space and don't need raw message uncomment this section
to not store the non-parsed raw syslog (Optional):
# mutate {
# # Original message has been fully parsed, so remove it.
# remove_field => [ "message" ]
# }
Copy pan-os.conf to your conf directory. For Ubuntu/Debian this is "/etc/logstash/conf.d/
- Run this command from the same directory where you put panos-template.json
- If running curl from another node please put the correct server IP and ensure port 9200 is open on the network (not a secure practice)
curl -XPUT http://127.0.0.1:9200/_template/panos-template?pretty -H 'Content-Type: application/json' -d @panos-template.json
log into your kibana interface and go to the saved objects page
http://<yourkibana DNS or IP>:5601/app/kibana#/management/kibana/objects
Click on import and select each import file in this order 1-index.ndjson 2-visualizations.ndjson 3-dashboard.ndjson 4-maps.ndjson
sudo systemctl restart elasticsearch.service sudo systemctl restart logstash.service
- Configure your PANW Firewall(s) or Panorama to send syslog messages to your Elastic Stack server
- Use port 5514
- Ensure that your firewall generates at least one traffic, threat, system & config syslog entry each
- Traffic will be generated by just going to a web site (make sure you setup logging for your policies).
- You may have to trigger a threat log entry. Follow this guide from Palo Alto for instructions
- After committing to set your syslog server, you will need to do another commit (any change) to actually send a config log message. Try changing the order of a rule and committing it.
Credit and Contributions
I have found several older OpenSource GitHub projects on Palo Alto to Elk setups and whish to thank the following early developers.
shadow-box - (https://github.com/shadow-box/Palo-Alto-Networks-ELK-Stack)
sm-biz - (https://github.com/sm-biz/paloalto-elasticstack-viz)
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent