smilebasti / npmgrafstats Goto Github PK

View Code? Open in Web Editor NEW

131.0 2.0 10.0 101 KB

This project exports NginxProxyManager Logs to InfluxDB for displaying them in a Grafana Dashboard.

License: GNU General Public License v3.0

Dockerfile 6.12% Python 43.32% Shell 50.56%

docker docker-compose grafana influxdb nginx nginx-proxy-manager

npmgrafstats's Introduction

npmGrafStats

NginxProxyManager Grafana Statistics.

This project analyzes the logs of the Nginx Proxy Manager and exports them to InfluxDB to be used in a Grafana Dashboard.

npmGrafStats can save the Revers-Proxy and/or the Redirection Logs. Also a exclusion of IP's from for example external montitoring services is possible.

Following Data is extracted from the Logs:

source IP
target IP in your home network set in NPM
the targeted domain
the measurement time
the Data of the source IP from GeoLite2-City.mmdb
- Country
- Coordinates
- City

A view of the Grafana Dashboard only within a few hours of running:

Newest features

v2.3.2 added in the Grafana dashboard a filter option to select only a few specific domains. Also the latest image is now running python:3-slim as base image (80% smaller than before!).

This project is a modified clone of https://github.com/Festeazy/nginxproxymanagerGraf and independent of https://github.com/jc21/nginx-proxy-manager. Changes to the original project can be found in the changelog.md file.

Obviously I'd appreciate anyhelp or feedback :) Hope you enjoy. If you do so, please star this project.

Installation

If you are only able to use InfluxDB v1 see Branch: https://github.com/smilebasti/npmGrafStats/tree/influx-v1. This Branch is not being developed anymore!

The Installation instructions can now be found in the GitHub Wiki.

Currently supported architectures: amd64, arm/v7 and arm64.

Github Registry is now additionally available to Docker Registry by adding ghcr.io/ in front of smilebasti/npmgrafstats:latest.

A full installation example is available with the docker-compose.yml

npmgrafstats's People

Contributors

Stargazers

Watchers

Forkers

florinmarica electrostyle dangquang9a gyarbij bam91 scotee007 nakedlitttlezombie nordinr maxdd alhazmy13

npmgrafstats's Issues

new installation influx empty

Meanwhile, congratulations on the work, I followed the installation guide and everything seems ok, in the portainer log I see that it reads the NMP logs, but when I went to check influx, I see the bucket empty.

Not a issue: It can't keep up :)

This is amazing... but it can't keep up. I set it to monitor a Matrix server... at 00:30 it was processing 20:00 data... and cpu pegged at 100%.

Reference:

No Issue/More of a feature request

Finally got this working and it's a much better way to visualize what's going on thru NPM. WilliamDonze has a similar product he's come up with that provides some additional level of detail such as paths of URL's accessed, any status codes/errors, browser detail, etc....would be neat to pull this information in your tool. Just a thought. Thanks for putting this together!

https://medium.com/@williamdonze/grafana-monitor-nginx-proxy-manager-website-4d03b60c761f

Mapping no longer works?

Hi there,

Installed npmgrafstats and love it, but something seems to have broken the mapping functionality and I can't seem to find any indication as to what/why/how.

I installed it on 7 JUN when I raised #14 . It's been running since, and I suspect my WAN IP changed at some point, because I observed the logs stopped parsing on or about 19 JUN. I fixed the WAN IP (before pulling the new version of the image where you addressed #15 ) and saw it began parsing again, but the mapping functionality is lost.

The below screenshot shows logs are accurately being parsed, but the mapping function is not functioning (2nd screenshot)

Snippet of docker compose logs (below) show the lat/long is being properly identified as well, but I'm happy to share over any relevant logs or output needed. I'm currently using the latest GeoLite2-City.mmdb file which was recent as of 7 JUN, but can upgrade to a more recent one if needed, thanks!

grafstats-npmgraf-1   | 2023-06-23T13:01:15.441916981Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:15.441978193Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:15.441993814Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442000808Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442006530Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442011860Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442017071Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442022251Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442027401Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442032592Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442037992Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442043173Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442048433Z Measurement Time:  2023-06-23T13:01:13+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:15.442053914Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698429938Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698473695Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698480819Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698486641Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698491972Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698497122Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698502222Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698507392Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698512543Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698517713Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698523214Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698528494Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698533845Z Measurement Time:  2023-06-23T13:01:13+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:15.698539406Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921452719Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921489422Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921496416Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921501967Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921507398Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921512658Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921517859Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921522999Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921528189Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921533299Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921538580Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921543840Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921549512Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:15.921554892Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.178921789Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.178954685Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:16.178961739Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:16.178984243Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:16.178989865Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.178995466Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179000756Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179005947Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179011147Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179016357Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179021688Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179026908Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179032570Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:16.179038371Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435550316Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435594834Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435602159Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435607710Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435613111Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435618381Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435624023Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435629183Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435635305Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435641467Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435648080Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435656056Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435660806Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:16.435665104Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.689939563Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.689970214Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:16.689975565Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:16.689979593Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:16.689983521Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690001417Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690005364Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690009523Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690013531Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690017649Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690021737Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690025635Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690029603Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:16.690033641Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945322981Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945359775Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945366899Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945372289Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945377670Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945382851Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945403953Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945409393Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945414804Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945420075Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945425395Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945430525Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945435786Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:16.945441167Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210669655Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210700146Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210707129Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210712550Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210717851Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210723131Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210728342Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210733512Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210738722Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210743993Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210749343Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210754514Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210759854Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:17.210765255Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463610078Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463651090Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463658314Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463663755Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463669126Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463674386Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463679576Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463684757Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463689977Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463695147Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463700548Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463705989Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463711300Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:17.463716700Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.714968673Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715018652Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715025867Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715031658Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715036949Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715042169Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715047440Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715052660Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715057941Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715084473Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715090105Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715095676Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715100996Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:17.715106357Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971142974Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971172603Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971177904Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971182002Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971186100Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971190078Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971193986Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971198014Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971201952Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971205920Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971210018Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971213835Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971217864Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:17.971221801Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222508744Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222540266Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222545427Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222549555Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222553573Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222557741Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222561699Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222565897Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222569865Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222573783Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222577841Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222581939Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222585947Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:18.222590035Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469925721Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469958667Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469963897Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469967935Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469971913Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469975781Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469979989Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469991071Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:18.469996182Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:18.470000109Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:18.470004438Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:18.470022484Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:18.470026973Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:18.470031201Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724031555Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724067747Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724074751Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724080192Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724085382Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724090532Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724095712Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724101193Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724106444Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724111574Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724116895Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724122105Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724128037Z Measurement Time:  2023-06-23T13:01:14+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:18.724135512Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974684114Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974711408Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974716619Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974720597Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974724534Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974728472Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974732470Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974736408Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974740556Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974744644Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974748652Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974752811Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974757029Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:18.974765546Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231287972Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231329334Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231334805Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231338793Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231342771Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231346628Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231350536Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231354464Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231358612Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231362500Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231366538Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231370426Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231374394Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:19.231378382Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480312931Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480345996Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480353411Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480358852Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480364223Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480369463Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480374633Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480379864Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480385114Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480390345Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480395785Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480401006Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480406326Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:19.480411697Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.734984109Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735016574Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735021875Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735026213Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735030241Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735034199Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735038127Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735042175Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735046253Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735050161Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735054159Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735058187Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735062205Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:19.735066233Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980475023Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980510553Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980517878Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980523319Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980528780Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980534160Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980539611Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980545743Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980550934Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980556154Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980561475Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980566755Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980572366Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:19.980578028Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232396057Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232431047Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232457109Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232463171Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232468501Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232473972Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232479193Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232484573Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232489944Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232495786Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232501136Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232506347Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232511657Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:20.232517419Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.485968900Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486035532Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486041985Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486046143Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486050161Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486054049Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486058308Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486062205Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486066133Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486070061Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486074039Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486077957Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486082095Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:20.486086213Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738317405Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738359579Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738366994Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738372505Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738377866Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738383146Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738388427Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738396262Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738402264Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738406613Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738410721Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738414739Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738418677Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:20.738422755Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988204741Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988239661Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988246735Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988252196Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988275642Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988281243Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988286354Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988291474Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988296804Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988301995Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988307305Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988312446Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988317726Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:20.988323047Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242882684Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242917804Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242924638Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242930029Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242935409Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242940650Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242946011Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242951401Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242956992Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242962393Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242967664Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242972944Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242978255Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:21.242983605Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491933635Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491964256Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491969727Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491973785Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491977763Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491981661Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491985559Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491989456Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491993344Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:21.491997252Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:21.492001290Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:21.492005248Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:21.492009276Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:21.492013314Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746731196Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746767758Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746774863Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746780664Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746786065Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746791285Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746796496Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746820754Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746826746Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746832026Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746837437Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746842828Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746848169Z Measurement Time:  2023-06-23T13:01:18+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:21.746853519Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968353209Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968388208Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968395172Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968400994Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968406475Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968411946Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968417377Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968423058Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968428940Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968434581Z Outside IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968440052Z Target IP:  170.72.231.145
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968445372Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968450953Z Measurement Time:  2023-06-23T13:01:19+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:21.968456334Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219308671Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219347448Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219354422Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219359753Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219365023Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219370935Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219376456Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219381716Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219386917Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219392167Z Outside IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219397768Z Target IP:  170.72.231.13
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219403189Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219408560Z Measurement Time:  2023-06-23T13:01:20+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:22.219414051Z *************** data send ******************
grafstats-npmgraf-1   | 2023-06-23T13:01:22.473982095Z **************** start *********************
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474040241Z Measurement-name: ReverseProxyConnections
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474047536Z United States
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474052946Z Ohio
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474058367Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474063898Z None
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474069760Z -82.9987
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474075561Z 39.9587
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474081664Z US
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474086914Z Outside IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474106924Z Target IP:  170.72.231.205
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474112565Z Domain:  auto.kadaranwar.com
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474117986Z Measurement Time:  2023-06-23T13:01:20+00:00
grafstats-npmgraf-1   | 2023-06-23T13:01:22.474123367Z *************** data send ******************

Adding default-host_access.log

Hello @smilebasti,
i recently received some spam/bruteforce attacks on my default_host which i've noticed they are not parsed by npmGrafStats since they are logged in "default-host_access.log".
Basically if the request is directly targeting the domain and not a subpath it is not logged in "proxy-host-*_access.log"
Do you think it make sense to also add the default-host log?

Support for NPMPlus?

Would be nice if this could work for NPMplus. Something you have been thinking about @smilebasti ?

https://github.com/ZoeyVid/NPMplus

platform: linux/arm64/v8

Docker compose error on Raspi4:
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
exec /root/start.sh: exec format error

error message in portainer from npmgraf

npmGrafStats: v2.2.1

2023-06-06T06:47:36.402618622Z Startup: lets get logs data and send them to influx

2023-06-06T06:47:36.402636619Z Redirection and Reverse-Proxy Logs activated

2023-06-06T06:47:36.410905887Z tail: cannot open '/logs/redirection-host-*_access.log' for reading: No such file or directory

2023-06-06T06:47:36.410932483Z tail: no files remaining

2023-06-06T06:47:36.411440904Z /root/.config/NPMGRAF/sendredirectionips.sh: line 54: reboot: command not found

Container auto crashing

Hello, upon starting up the npmgrafstats container it seems to be auto crashing immediately with the following logs posted:

Startup: lets get logs data and send them to influx
Redirection Logs activated
tail: cannot open '/logs/proxy-host-*_access.log' for reading: No such file or directory
tail: no files remaining
/root/.config/NPMGRAF/sendips.sh: line 38: reboot: command not found

I was just wondering if there is something I'm clearly doing wrong to get this started. Weirdly it kept looping a French IP address over and over again in the logs (possible it was looping through one entry?)

Attaching my config below----------

Redacted npmgrafstats docker-compose:

version: '3.8'
services:

  npmgraf:
    image: smilebasti/npmgrafstats:latest
    restart: unless-stopped
    environment:
      HOME_IPS: 'externalIP'
      REDIRECTION_LOGS: 'TRUE' # TRUE or FALSE
      INFLUX_HOST: 'hostip:8086' # use host IP
      INFLUX_BUCKET: 'npmgrafstats'
      INFLUX_ORG: 'npmgrafstats'
      INFLUX_TOKEN: 'cooltokennumber' # insert after first run and manual token creation
    volumes:
      - /data/compose/1/data/logs:/logs #nginx log location
      - ./geolite:/geolite
     
  influxdb:
    image: influxdb:2.7-alpine
    environment:
      DOCKER_INFLUXDB_INIT_MODE: 'setup'
      DOCKER_INFLUXDB_INIT_USERNAME: 'CoolUserName'
      DOCKER_INFLUXDB_INIT_PASSWORD: 'coolpassword' # atleast 8 characters
      DOCKER_INFLUXDB_INIT_ORG: 'npmgrafstats'
      DOCKER_INFLUXDB_INIT_BUCKET: 'npmgrafstats'
    volumes:
      # Mount for influxdb data directory and configuration
      - ./influxdbv2:/var/lib/influxdb2
    ports:
      - '8086:8086'

  geoipupdate:
    image: maxmindinc/geoipupdate
    environment:
      GEOIPUPDATE_ACCOUNT_ID: 'accountid'
      GEOIPUPDATE_LICENSE_KEY: 'geoiplicense'
      GEOIPUPDATE_EDITION_IDS: 'GeoLite2-City'
      GEOIPUPDATE_FREQUENCY: 24
    volumes:
      - ./geolite:/usr/share/GeoIP

Add support for more domains names extensions (InfluxDB v1)

Hi,
I am unable to get the domain name for a reverse proxy connection when the extension is something else than .com

After looking through the code I found that only some domains extensions are "supported"

According to the file sendips.sh, only .com .de .net .org are catched:

  #Domain or subdomain gets found. only de/net/org/com
  domain=`echo ${line:0:80} | grep -m 1 -o -E "([a-z0-9\-]*\.){1,3}?[a-z0-9\-]*\.(de|net|org|com)"`

Would it be possible to support more domains names extensions ?
Or add a way to customize this with an env variable in Docker ?

Thanks for all the good work you made on this project 👍

Requests not logging into Influx/Visible in Grafana.

Hello,
I installed using docker compose following the guide in the wiki, but the Grafana dashboard is not showing data, and I cannot find the data in the influx db instance.

Things I have checked:
I have entered the shell for the npmgraf container and verified it could ping all other containers with their listed hostnames.
Tokens being correct
The successful download of both .mmdb for geolite

When attached to the containers, this is what is shown when viewing my site (I have hidden to location and IP info):

nginx-home-npmgraf-1      | **************** start *********************
nginx-home-npmgraf-1      | Measurement-name: ReverseProxyConnections
nginx-home-npmgraf-1      | United States
nginx-home-npmgraf-1      | Texas
nginx-home-npmgraf-1      | x.x.x.x 
nginx-home-npmgraf-1      | x.x.x.x
nginx-home-npmgraf-1      | x.x.x.x
nginx-home-npmgraf-1      | x.x.x.x
nginx-home-npmgraf-1      | US
nginx-home-npmgraf-1      | x.x.x.x-TEXAS
nginx-home-npmgraf-1      | Outside IP:  x.x.x.x
nginx-home-npmgraf-1      | Target IP:  x.x.x.x
nginx-home-npmgraf-1      | Domain:  x.x.x.x.org
nginx-home-npmgraf-1      | Measurement Time:  2023-12-21T20:36:33+00:00
nginx-home-npmgraf-1      | Traceback (most recent call last):
nginx-home-npmgraf-1      |   File "/root/.config/NPMGRAF/Getipinfo.py", line 137, in <module>
nginx-home-npmgraf-1      |     write_api.write(bucket=ifbucket, org=iforg, record=point)
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/client/write_api.py", line 378, in write
nginx-home-npmgraf-1      |     results = list(map(write_payload, payloads.items()))
nginx-home-npmgraf-1      |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/client/write_api.py", line 376, in write_payload
nginx-home-npmgraf-1      |     return self._post_write(_async_req, bucket, org, final_string, payload[0])
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/client/write_api.py", line 509, in _post_write
nginx-home-npmgraf-1      |     return self._write_service.post_write(org=org, bucket=bucket, body=body, precision=precision,
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/service/write_service.py", line 60, in post_write
nginx-home-npmgraf-1      |     (data) = self.post_write_with_http_info(org, bucket, body, **kwargs)  # noqa: E501
nginx-home-npmgraf-1      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/service/write_service.py", line 90, in post_write_with_http_info
nginx-home-npmgraf-1      |     return self.api_client.call_api(
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
nginx-home-npmgraf-1      |     return self.__call_api(resource_path, method,
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
nginx-home-npmgraf-1      |     response_data = self.request(
nginx-home-npmgraf-1      |                     ^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/_sync/api_client.py", line 388, in request
nginx-home-npmgraf-1      |     return self.rest_client.POST(url,
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/_sync/rest.py", line 311, in POST
nginx-home-npmgraf-1      |     return self.request("POST", url,
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/influxdb_client/_sync/rest.py", line 220, in request
nginx-home-npmgraf-1      |     r = self.pool_manager.request(
nginx-home-npmgraf-1      |         ^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/urllib3/_request_methods.py", line 118, in request
nginx-home-npmgraf-1      |     return self.request_encode_body(
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/urllib3/_request_methods.py", line 217, in request_encode_body
nginx-home-npmgraf-1      |     return self.urlopen(method, url, **extra_kw)
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/urllib3/poolmanager.py", line 433, in urlopen
nginx-home-npmgraf-1      |     conn = self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
nginx-home-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-home-npmgraf-1      |   File "/usr/local/lib/python3.12/site-packages/urllib3/poolmanager.py", line 295, in connection_from_host
nginx-home-npmgraf-1      |     raise LocationValueError("No host specified.")
nginx-home-npmgraf-1      | urllib3.exceptions.LocationValueError: No host specified.

And this is my dockerfile

version: '3.8'
services:
  npm:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: always
    ports:
      # These ports are in format <host-port>:<container-port>
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
      # Add any other Stream port you want to expose
      # - '21:21' # FTP
    environment:
      # Mysql/Maria connection parameters:
      DB_MYSQL_HOST: "db"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "npm"
      DB_MYSQL_NAME: "npm"
      # Uncomment this if IPv6 is not enabled on your host
      DISABLE_IPV6: 'true'
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
    depends_on:
      - db

  db:
    image: 'jc21/mariadb-aria:latest'
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: 'npm'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'npm'
    volumes:
      - ./mysql:/var/lib/mysql

  npmgraf:
    image: smilebasti/npmgrafstats:latest # change to ghcr.io/smilebasti/npmgrafstats if you prefer Github Container Registry
    restart: always
    environment:
      REDIRECTION_LOGS: 'TRUE' # TRUE or FALSE or ONLY
      INTERNAL_LOGS: 'TRUE' # see Github wiki for more information
      MONITORING_LOGS: 'TRUE' # see Github wiki for more information
      INFLUX_HOST: 'influxdb:8086' # use host IP
      INFLUX_BUCKET: 'npmgrafstats'
      INFLUX_ORG: 'npmgrafstats'
      INFLUX_TOKEN: 'SECRET' # insert after first run and manual token creation
    volumes:
      - ./data/logs:/logs
      - ./geolite:/geolite
    depends_on:
      - geoipupdate
      - npm

  influxdb:
    image: influxdb:2.7-alpine
    restart: always
    environment:
      DOCKER_INFLUXDB_INIT_MODE: 'setup'
      DOCKER_INFLUXDB_INIT_USERNAME: 'SECRET'
      DOCKER_INFLUXDB_INIT_PASSWORD: 'SECRET' # atleast 8 characters
      DOCKER_INFLUXDB_INIT_ORG: 'npmgrafstats'
      DOCKER_INFLUXDB_INIT_BUCKET: 'npmgrafstats'
    volumes:
      # Mount for influxdb data directory and configuration
      - ./influxdbv2:/var/lib/influxdb2
    ports:
      - '8086:8086'

  geoipupdate:
    image: maxmindinc/geoipupdate
    restart: always
    environment:
      GEOIPUPDATE_ACCOUNT_ID: 'SECRET'
      GEOIPUPDATE_LICENSE_KEY: 'SECRET'
      GEOIPUPDATE_EDITION_IDS: 'GeoLite2-City  GeoLite2-ASN'  #GeoLite2-ASN is optional
      GEOIPUPDATE_FREQUENCY: 24
    volumes:
      - ./geolite:/usr/share/GeoIP

Cannot login to InfluxDB

Followed the instructions and used the Wiki as reference. Still cannot login to InfluxDB. I've seen references to you neededing to inject a config volume but that is not in the .yml docker files as an example. Can't Log In is the error I get over and over again, after rebuilding the InfluxDB container

Any thoughts?

affinity:container	=1537af48ec3018ebb1919b2be24e86859da18592e4b44bb5b860d2442b645f11
DOCKER_INFLUXDB_INIT_ADMIN_TOKEN	ABC123abc
DOCKER_INFLUXDB_INIT_BUCKET	npmgrafstats
DOCKER_INFLUXDB_INIT_CLI_CONFIG_NAME	default
DOCKER_INFLUXDB_INIT_MODE	setup
DOCKER_INFLUXDB_INIT_ORG	npmgrafstats
DOCKER_INFLUXDB_INIT_PASSWORD	alphanumericplusspecialcharacter
DOCKER_INFLUXDB_INIT_USERNAME	8alphacharacterpassword
INFLUX_CLI_VERSION	2.7.3
INFLUX_CONFIGS_PATH	/etc/influxdb2/influx-configs
INFLUXD_INIT_PING_ATTEMPTS	600
INFLUXD_INIT_PORT	9999
INFLUXDB_VERSION	2.7.4
PATH	/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Installed and everything working - Grafana has no dashboard - it is just default

Thank you for your help @smilebasti with the Token, all worked well.

All docker-compose containers running, the token, and all user/passwords are set correctly. But when I log in to grafana it is just a completely vanilla installation. Should I not see your dashboard? It is not clear what I now need to do.

What does smilebasti/npmgrafstats container actually do, it has no ports so I cannot connect to it...

Grafana screenshot - what to do now?

[Feature Request] Support Remote NPM Instance

Hi
I run a proxmox Cluster where everything is a bit seperated.

I have currently a NPM linuxcontainer (LXC) and a VM that runs all my docker containers, so it would be really really awesome, if this app could fetch that over the network.

Or have you got any ideas of possible workarounds?

Grafana v9 and maps

First of all, thank you for the work you did on this. It works great.

One thing I noticed was that the maps don't work on version 9 of Grafana. I hadn't updated yet and I thought there was something wrong with the query. But when I looked at the data it was returning it looked it good. I already had Grafana installed and I figured adding the latest tag to the image and telling Portainer to always pull the image when re-deploying would be enough. But turns out the latest tag doesn't pull v10, I had to add the tag main instead.

Just a heads up in case any one else runs into the same problem. We upgraded to v10 from v9 and didn't see any problems FWIW.

Arm Docker image

https://www.reddit.com/r/selfhosted/comments/1418ct3/for_all_of_you_nginxproxymanager_users_and/jmzbpbg

[Request] Whois integration

Hello,
as an enchancement/feature what do you think about improving Getipinfo.py with whois?
Ideally i would love to have a map showing also who is the "registrant" of the ip e.g. google/ISP/mobile carrier
here an example of whois data

https://www.geeksforgeeks.org/how-to-use-the-whois-command-on-ubuntu-linux/

if im not wrong there is a python wrapper as well

Allow/Block from country

Hi! perfect and working on first run! Congrats!
I want to know if there is a way as we have running npm and geoip, to configure npm for allow/blocking from country, and how we need to add that info to the grafana dashboard.

I was using this tutorial but it doesnt work anymore..: https://www.reddit.com/r/nginxproxymanager/comments/sjf1hw/tutorial_nginx_proxy_manager_together_with_geoip2/

It would be so complete on functions if we can make it work!

Grafana dashboard issue

Hello,
i've seen you are using the following two commands in the query

from(bucket: v.defaultBucket)

for me unless i change this to "npmgrafstats" as per the bucket name, it doesnt fetch data.
How to i set the defaultBucket?

this link https://grafana.com/docs/grafana/latest/datasources/influxdb/query-editor/ is mentioning something but i'm wondering whether since i have many buckets it is better to change it to the proper one

Also how do i fill in the "domainquery:regex" below? If i leave it as such nothing can be seen while if i remove it instead it works

|> filter(fn: (r) => r["Domain"] =~ /^${domainquery:regex}$/)

if its fetching the domain from influx i guess i shall also change the variable with the proper bucket

import "influxdata/influxdb/schema"
schema.tagValues(
    bucket: v.defaultBucket,
    tag: "Domain"
)

Im failing to understand the goal of the filter though

Map filter domain

Hi, i really like the map on the dashboard. But is there a way to filter on (sub)domain? Now all the IP's from all the (sub)domains are shown on the map, but I want to select which (sub)domains I see on the map.

Is this already possible? Or is this something you will maybe implement in the future?

Get HOME_IPS variable automatically

Get your outside IP automatically.

See for more: https://www.reddit.com/r/selfhosted/comments/1418ct3/comment/jnbho6v/?utm_source=share&utm_medium=web2x&context=3

Some domains not working and Map not working

I have several sites, eg:

a. xyz.domain.com
b. abc.domain.com
c. def.anotherdomain.com

and also just

d. domain.com (same as www.domain.com).
e. anotherdomain.com (same as www.anotherdomain.com).

Everything works for a. and b. but not for any of the TLDs, only for subdomains it seems. And for some other subdomains like c. it just isn't logging anything at all (can't see anything in the docker-compose log -f command when I visit those sites.

How can I check what is going on with these ones that are not working?

Also, the map is not working for anything, even though the log shows correctly the city names.

InfluxDB Token - where to find it?

Read all the installation docs, but I cant find where to get the influxdb token for this line in docker-compose file:

INFLUX_TOKEN: '<replace>' # insert after first run and manual token creation

Sorry for the stupid question, but where do I get this token? I checked the logs, I could not find it in there after first run of influx..

No Host Specified

I don't think npmgraf likes my logs :/

stats-npmgraf-1  | Internal IP-Source: 130.61.129.53 called: thelounge.aguiarvieira.pt
stats-npmgraf-1  | Traceback (most recent call last):
stats-npmgraf-1  |   File "/root/.config/NPMGRAF/Getipinfo.py", line 122, in <module>
stats-npmgraf-1  | **************** start *********************
stats-npmgraf-1  | Measurement-name: ReverseProxyConnections
stats-npmgraf-1  | Portugal
stats-npmgraf-1  | Porto
stats-npmgraf-1  | Maia
stats-npmgraf-1  | 4470-070
stats-npmgraf-1  | -8.6182
stats-npmgraf-1  | 41.2345
stats-npmgraf-1  | PT
stats-npmgraf-1  | Outside IP:  2.82.30.18
stats-npmgraf-1  | Target IP:  2.82.30.18
stats-npmgraf-1  | Domain:  aguiarvieira.pt
stats-npmgraf-1  | Measurement Time:  2023-09-15T22:27:13+00:00
stats-npmgraf-1  |     write_api.write(bucket=ifbucket, org=iforg, record=point)
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/client/write_api.py", line 378, in write
stats-npmgraf-1  |     results = list(map(write_payload, payloads.items()))
stats-npmgraf-1  |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/client/write_api.py", line 376, in write_payload
stats-npmgraf-1  |     return self._post_write(_async_req, bucket, org, final_string, payload[0])
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/client/write_api.py", line 509, in _post_write
stats-npmgraf-1  |     return self._write_service.post_write(org=org, bucket=bucket, body=body, precision=precision,
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/service/write_service.py", line 60, in post_write
stats-npmgraf-1  |     (data) = self.post_write_with_http_info(org, bucket, body, **kwargs)  # noqa: E501
stats-npmgraf-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/service/write_service.py", line 90, in post_write_with_http_info
stats-npmgraf-1  |     return self.api_client.call_api(
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
stats-npmgraf-1  |     return self.__call_api(resource_path, method,
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
stats-npmgraf-1  |     response_data = self.request(
stats-npmgraf-1  |                     ^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 388, in request
stats-npmgraf-1  |     return self.rest_client.POST(url,
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 311, in POST
stats-npmgraf-1  |     return self.request("POST", url,
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 220, in request
stats-npmgraf-1  |     r = self.pool_manager.request(
stats-npmgraf-1  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/urllib3/_request_methods.py", line 118, in request
stats-npmgraf-1  |     return self.request_encode_body(
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/urllib3/_request_methods.py", line 217, in request_encode_body
stats-npmgraf-1  |     return self.urlopen(method, url, **extra_kw)
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/urllib3/poolmanager.py", line 432, in urlopen
stats-npmgraf-1  |     conn = self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
stats-npmgraf-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
stats-npmgraf-1  |   File "/usr/local/lib/python3.11/site-packages/urllib3/poolmanager.py", line 294, in connection_from_host
stats-npmgraf-1  |     raise LocationValueError("No host specified.")
stats-npmgraf-1  | urllib3.exceptions.LocationValueError: No host specified.

Although i have some custom log formats, they are not on that log dir...

Example of a log:

[15/Sep/2023:22:30:14 +0000] - 200 200 - GET https aguiarvieira.pt "/_matrix/client/versions" [Client 2.82.30.18] [Length 867] [Gzip -] [Sent-to matrix-element] "mtxclient v0.9.2" "-"
[15/Sep/2023:22:30:21 +0000] - 200 200 - POST https aguiarvieira.pt "/_matrix/client/v3/publicRooms" [Client 130.61.129.53] [Length 2916] [Gzip -] [Sent-to matrix-element] "undici" "-"
[15/Sep/2023:22:30:23 +0000] - 200 200 - GET https aguiarvieira.pt "/_synapse/admin/v1/event_reports?dir=f&from=0&dir=f&from=0" [Client 130.61.129.53] [Length 650] [Gzip -] [Sent-to matrix-element] "Python/3.8 aiohttp/3.8.5" "-"
[15/Sep/2023:22:30:29 +0000] - 200 200 - GET https aguiarvieira.pt "/_matrix/client/versions" [Client 2.82.30.18] [Length 867] [Gzip -] [Sent-to matrix-element] "mtxclient v0.9.2" "-"
[15/Sep/2023:22:30:44 +0000] - 200 200 - GET https aguiarvieira.pt "/_matrix/client/versions" [Client 2.82.30.18] [Length 867] [Gzip -] [Sent-to matrix-element] "mtxclient v0.9.2" "-"
[15/Sep/2023:22:30:53 +0000] - 200 200 - GET https aguiarvieira.pt "/_synapse/admin/v1/event_reports?dir=f&from=0&dir=f&from=0" [Client 130.61.129.53] [Length 650] [Gzip -] [Sent-to matrix-element] "Python/3.8 aiohttp/3.8.5" "-"
[15/Sep/2023:22:30:59 +0000] - 304 304 - GET https aguiarvieira.pt "/version" [Client 2.82.30.18] [Length 0] [Gzip -] [Sent-to matrix-element] "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" "-"

Stops Working after a few Days

Odd behavior that the image stops working after few days.

logrotation?
memory leak?

Exclude IP from dashboard ?

Hello, I really like your new feature that it's possible to filter domains on the dashboard.
But is it also possible to exclude IP's from showing up on the dashboard?

The reason i'm asking this is that I have a monitoring server that is checking the uptime of my domains. But this ip shows up very much in the dashboard of course.
Is there a way I can exclude it from showing up? Or is this a feature you need to implement first?

Can't seem to work with LePresidente/docker-nginx-proxy-manager

I've been using this container, as per the suggested configuration, but wanted to switch out jc21/nginx-proxy-manager with LePresidente/docker-nginx-proxy-manager, given the added crowdsec features.
That container doesn't run the npm db (jc21/mariadb-aria) though.
Does that explain why it doesn't work, or is there something special that can be done to make it work?
From testing, I can determine that influxdb isn't receiving any log info, so the graphs are empty.

Add a tld list

A Top Level Domains.
Watch if log is included.

Connection Refused in npmgraf

It seems like the npmgraf container is failing to export logs to InfluxDB. The container's logs show-

nginx-ui-npmgraf-1      | **************** start *********************
nginx-ui-npmgraf-1      | Measurement-name: ReverseProxyConnections
nginx-ui-npmgraf-1      | United States
nginx-ui-npmgraf-1      | Virginia
nginx-ui-npmgraf-1      | Ashburn
nginx-ui-npmgraf-1      | (redacted)
nginx-ui-npmgraf-1      | (redacted)
nginx-ui-npmgraf-1      | (redacted)
nginx-ui-npmgraf-1      | US
nginx-ui-npmgraf-1      | Outside IP:  (redacted)
nginx-ui-npmgraf-1      | Target IP:  (redacted)
nginx-ui-npmgraf-1      | Domain:  (redacted)
nginx-ui-npmgraf-1      | Measurement Time:  2023-07-28T16:25:51+00:00
nginx-ui-npmgraf-1      | Traceback (most recent call last):
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
nginx-ui-npmgraf-1      |     sock = connection.create_connection(
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
nginx-ui-npmgraf-1      |     raise err
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
nginx-ui-npmgraf-1      |     sock.connect(sa)
nginx-ui-npmgraf-1      | ConnectionRefusedError: [Errno 111] Connection refused
nginx-ui-npmgraf-1      | 
nginx-ui-npmgraf-1      | The above exception was the direct cause of the following exception:
nginx-ui-npmgraf-1      | 
nginx-ui-npmgraf-1      | Traceback (most recent call last):
nginx-ui-npmgraf-1      |   File "/root/.config/NPMGRAF/Getipinfo.py", line 122, in <module>
nginx-ui-npmgraf-1      |     write_api.write(bucket=ifbucket, org=iforg, record=point)
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/client/write_api.py", line 378, in write
nginx-ui-npmgraf-1      |     results = list(map(write_payload, payloads.items()))
nginx-ui-npmgraf-1      |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/client/write_api.py", line 376, in write_payload
nginx-ui-npmgraf-1      |     return self._post_write(_async_req, bucket, org, final_string, payload[0])
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/client/write_api.py", line 509, in _post_write
nginx-ui-npmgraf-1      |     return self._write_service.post_write(org=org, bucket=bucket, body=body, precision=precision,
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/service/write_service.py", line 60, in post_write
nginx-ui-npmgraf-1      |     (data) = self.post_write_with_http_info(org, bucket, body, **kwargs)  # noqa: E501
nginx-ui-npmgraf-1      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/service/write_service.py", line 90, in post_write_with_http_info
nginx-ui-npmgraf-1      |     return self.api_client.call_api(
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
nginx-ui-npmgraf-1      |     return self.__call_api(resource_path, method,
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
nginx-ui-npmgraf-1      |     response_data = self.request(
nginx-ui-npmgraf-1      |                     ^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 388, in request
nginx-ui-npmgraf-1      |     return self.rest_client.POST(url,
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 311, in POST
nginx-ui-npmgraf-1      |     return self.request("POST", url,
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 220, in request
nginx-ui-npmgraf-1      |     r = self.pool_manager.request(
nginx-ui-npmgraf-1      |         ^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/_request_methods.py", line 118, in request
nginx-ui-npmgraf-1      |     return self.request_encode_body(
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/_request_methods.py", line 217, in request_encode_body
nginx-ui-npmgraf-1      |     return self.urlopen(method, url, **extra_kw)
nginx-ui-npmgraf-1      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/poolmanager.py", line 443, in urlopen
nginx-ui-npmgraf-1      |     response = conn.urlopen(method, u.request_uri, **kw)
nginx-ui-npmgraf-1      |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
nginx-ui-npmgraf-1      |     retries = retries.increment(
nginx-ui-npmgraf-1      |               ^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 445, in increment
nginx-ui-npmgraf-1      |     raise reraise(type(error), error, _stacktrace)
nginx-ui-npmgraf-1      |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
nginx-ui-npmgraf-1      |     raise value
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
nginx-ui-npmgraf-1      |     response = self._make_request(
nginx-ui-npmgraf-1      |                ^^^^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
nginx-ui-npmgraf-1      |     conn.request(
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 395, in request
nginx-ui-npmgraf-1      |     self.endheaders()
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/http/client.py", line 1281, in endheaders
nginx-ui-npmgraf-1      |     self._send_output(message_body, encode_chunked=encode_chunked)
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/http/client.py", line 1041, in _send_output
nginx-ui-npmgraf-1      |     self.send(msg)
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/http/client.py", line 979, in send
nginx-ui-npmgraf-1      |     self.connect()
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 243, in connect
nginx-ui-npmgraf-1      |     self.sock = self._new_conn()
nginx-ui-npmgraf-1      |                 ^^^^^^^^^^^^^^^^
nginx-ui-npmgraf-1      |   File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 218, in _new_conn
nginx-ui-npmgraf-1      |     raise NewConnectionError(
nginx-ui-npmgraf-1      | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0xffffa97ca750>: Failed to establish a new connection: [Errno 111] Connection refused

The token i inserted into the docker-compose file is an all access token. The host I put in for the influxdb host is HOST_IP:8086 and it is publicly accessible- I can open HOST_IP:8086 in my browser and it works. I tried http://HOST_IP:8086 as well, but that didn't work either. What should I do?

Expand installation instructions for Grafanna setup

The instructions mention setting up a data source, but there's lots of other options that need to be filled out - can the instructions go into more details, please?

For instance, where does the API key go? Which Query language should be used? Any custom headers?

Screenshot shows the available options:

Please let me know if I can help - I'd be happy to submit a PR, thanks!

Feature Request: Log data for redirection hosts

It would be nice to add the same logging function this has for proxy hosts to redirection hosts

AbuseIPDB feature

Hello @smilebasti
I've integrated AbuseIPDB information (totalreports + abuse confidence level) in your script.

Let me know if you want me to create a pull request.
The activation of the feature is based on the presence of the "ABUSEIP_KEY" environmental variable, if it is missing (None) then it won't be included in the Getipinfo execution

Use log timestamps for measurement

Use log timestamps for measurement to get precise logs and prevent duplicate logs in case of a container restart.

Writing time to influx prevents duplication.

This is a requested feature in: Festeazy/nginxproxymanagerGraf#4

no data in influxdb 2

i have installed the tool. unfortunately no data is written to the bucket. i can't find any error message in the logs either

[SURVEY]: keep map dots or change back to original design?

Hi, i need your opinions!

Keep current map style or return to original design?

The original map colors the dots on the map to red for 2 or less calls, yellow for 3 to 10 calls and green for 10 or more calls. This made it possible to see a little more details for fewer calls per IP. The dot size in green is was still used for larger calls.

picture shows the original design
picture shows the current design

React to this with 👍 to switch to original and with 👎 to keep the current design. Leave a comment with your thoughts.

PS: with Grafana 10.1 the map will get a route option. I will try to use that to implement a Missle Map with that.

Support for multiple NPM instances

Fantastic work and brilliant documentation. Had no issues getting it up and running for my NPM instance.

Feedback - it would be a nice to have feature if we can somehow ingest logs from two separate Npm instances. I run separate reverse proxies for my internal and external domains. The world map graph wouldn’t be much since all my internal domains would be from the same external Ip but the other statistics would be slick to see