edvrfn/get_nsg_logs

How to retrieve NSG logs via Python

Version

The latest version of this script works with Pandas dataframes and supports Azure Firewall logs sent to the same storage account (see parameter mode).

Reading NSG logs from Blob Storage

You might have set your Azure Vnet, with some NSGs. You start rolling apps, to the point where you have many VMs, and many NSGs. Somebody makes an application upgrade, or installs a new application, but traffic is not flowing through. Which NSG is dropping traffic? Which TCP ports should be opened?

One possibility is using Traffic Analytics. Traffic Analytics is a two-step process:

1. NSG logs are stored in a storage account
2. NSG logs from the storage account are processed (consolidated and enriched with additional information) and made queriable

One of the nicest features of Traffic Analytics is being able to query logs with the KQL (Kusto Query Language). For example, you can use this query to find out the dropped flows in the last 3 hours for IP address 1.2.3.4:

AzureNetworkAnalytics_CL
| where TimeGenerated >= ago(1h)
| where SubType_s == "FlowLog"
| where DeniedInFlows_d > 0 or DeniedOutFlows_d > 0
| where SrcIP_s == "1.2.3.4"
| project NSGName=split(NSGList_s, "/")[2],NSGRules_s,DeniedInFlows_d,DeniedOutFlows_d,SrcIP_s,DestIP_s,DestPort_d,L7Protocol_s

However, you will notice that there is a time lag, and you will not find the very latest logs in Log Analytics. The original NSG Flow logs are stored in storage account, in JSON format, so you could get those logs using the Azure Storage SDK.

You can use the Python script in this repository (assuming you have the Python SDK for storage installed). You can use different flags, like the --help option to get usage information. If you do not have handy a Python console you can use Azure Cloud Shell and clone this repository:

git clone https://github.com/erjosito/get_nsg_logs

If you don't have the Azure Storage Blob and pandas Python modules, you need to install them:

pip3 install azure-storage-blob pandas --user

Now you can have a look at the different options:

$ cd ./get_nsg_logs/
$ python3 ./get_nsg_logs.py --help
usage: get_nsg_logs.py [-h] [--account-name ACCOUNT_NAME] [--display-lb]
                       [--display-allowed]
                       [--display-direction DISPLAY_DIRECTION]
                       [--display-hours DISPLAY_HOURS] [--version VERSION]
                       [--only-non-zero] [--flow-state FLOW_STATE_FILTER]
                       [--ip IP_FILTER] [--port PORT_FILTER]
                       [--nsg-name NSG_NAME_FILTER] [--aggregate] [--verbose]

Get the latest flow logs in a storage account

optional arguments:
  -h, --help            show this help message and exit
  --account-name ACCOUNT_NAME
                        you need to supply an storage account name. You can get a list of your storage accounts with this command: az storage account list -o table
  --display-lb          display or hide flows generated by the Azure LB (default: False)
  --display-allowed     display as well flows allowed by NSGs (default: False)
  --display-direction DISPLAY_DIRECTION
                        display flows only in a specific direction. Can be in, out, or both (default in)
  --display-hours DISPLAY_HOURS
                        How many hours to look back (default: 1)
  --display-minutes DISPLAY_MINUTES
                        How many minutes to look back (default: 0/unlimited)
  --only-non-zero       display only v2 flows with non-zero packet/byte counters (default: False)
  --flow-state FLOW_STATE_FILTER
                        filter the output to a specific v2 flow type (B/C/E)
  --ip IP_FILTER        filter the output to a specific IP address
  --port PORT_FILTER    filter the output to a specific TCP/UDP port
  --protocol PROTOCOL_FILTER
                        filter the output to a specific protocol (T/U/I)
  --nsg-name NSG_NAME_FILTER
                        filter the output to a specific NSG
  --mode MODE           can be nsg,fw,both (default: nsg)
  --aggregate           prints byte/packet count aggregates (default: False)
  --no-output           does not print out any output, useful with --verbose flag (default: False)
  --verbose             run in verbose mode (default: False)

There is something you need to do before being able to access Azure Blob Storage: finding out the Azure Storage Account key. The script will read it from the environtment variable STORAGE_ACCOUNT_KEY, that you can set with this command:

storage_account_name=yourstorageaccount123
export STORAGE_ACCOUNT_KEY=$(az storage account keys list -n $storage_account_name --query '[0].value' -o tsv)

For example, in order to show dropped and allowed traffic of ingress NSG logs stored in a storage account (if following the previous instructions, stored in the variable $storage_account_name), excluding Azure LB probe traffic for the last 2 hours:

$ python3 ./get_nsg_logs.py --account-name $storage_account_name --display-hours 2 --display-direction in --display-allowed
                               timestamp type  resource                               rule state       src_ip       dst_ip src_port dst_port protocol direction action
1276 2022-05-06 07:08:02.967603300+00:00  nsg  FLOWLOGS      DefaultRule_AllowVnetOutBound     E  192.168.1.4  192.168.1.6    33286     1433        T         O      A
1262 2022-05-06 07:08:02.967603300+00:00  nsg  FLOWLOGS  DefaultRule_AllowInternetOutBound     B  192.168.1.4  13.69.67.61    36552      443        T         O      A
1261 2022-05-06 07:08:02.967603300+00:00  nsg  FLOWLOGS  DefaultRule_AllowInternetOutBound     E  192.168.1.4  13.69.67.61    36516      443        T         O      A
1277 2022-05-06 07:08:02.967603300+00:00  nsg  FLOWLOGS      DefaultRule_AllowVnetOutBound     B  192.168.1.4  192.168.1.7    55280     1433        T         O      A
1275 2022-05-06 07:08:02.967603300+00:00  nsg  FLOWLOGS      DefaultRule_AllowVnetOutBound     E  192.168.1.4  192.168.1.5    58948       80        T         O      A
...

Another example is showing logs captured to troubleshoot communication between specific machines on a specific port (on the last 3 minutes to reduce output):

❯ python3 ./get_nsg_logs.py --account-name $storage_account_name --mode=nsg --display-hours 1 --display-allowed --display-minutes 3 --ip 192.168.1.4 --ip2 192.168.1.5 --aggregate
                 timestamp              type  resource              rule             state packets_src_to_dst bytes_src_to_dst packets_dst_to_src bytes_dst_to_src    src_ip       dst_ip    src_port dst_port protocol direction action
579 2022-05-06 07:18:34.404934100+00:00  nsg  FLOWLOGS  DefaultRule_AllowVnetInBound    B                                                                           192.168.1.4  192.168.1.5   60142     80        T         I       A  
580 2022-05-06 07:18:34.404934100+00:00  nsg  FLOWLOGS  DefaultRule_AllowVnetInBound    E           1                74                 1                54         192.168.1.4  192.168.1.5   60142     80        T         I       A  
bytes_src_to_dst      74
packets_src_to_dst     1
bytes_dst_to_src      54
packets_dst_to_src     1
dtype: object

Happy troubleshooting!