Comments (2)
Thank you for sharing this. Do you happen to know if this is the default field output or if it has been changed at some point? The current output you provided is:
"version":"TLSv1.3","cipher_suite": "TLS_AES_128_GCM_SHA256"
However, the default in GoAccess includes "cipher_suite" and "proto". In any event, the following should still function properly:
goaccess access.log --log-format='{ "ts": "%x.%^", "request": { "remote_ip": "%h", "proto": "%H", "method": "%m", "host": "%v", "uri": "%U", "headers": { "User-Agent": [ "%u" ], "Referer": [ "%R" ] }, "tls": { "cipher_suite": "%k", "version": "%K" } }, "duration": "%T", "size": "%b", "status": "%s", "resp_headers": { "Content-Type": [ "%M" ] } }' --date-format=%s --time-format=%s
from goaccess.
However, the default in GoAccess includes "cipher_suite" and "proto". In any event, the following should still function properly:
goaccess access.log --log-format='{ "ts": "%x.%^", "request": { "remote_ip": "%h", "proto": "%H", "method": "%m", "host":
Thank your for your help @allinurl.
That solved the problem.
The first JSON posted above is the default field output of Caddy v2.7.4. with
"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"http/1.1","server_name":"arm.stbu.net"}
Using the GoAccess v1.8.0 with `--caddy-format=CADDY" wasn't working.
You might want to reproduce that using:
$ /home/stbu/goaccess --version | grep GoAccess
GoAccess - 1.8.
$ echo '{"level":"info","ts":1696325286.9971106,"logger":"http.log.access.arm.stbu.net","msg":"handled request","request":{"remote_ip":"46.101.122.201","remote_port":"39062","client_ip":"46.101.122.201","proto":"HTTP/1.1","method":"GET","host":"arm.stbu.net","uri":"/","headers":{"Accept-Encoding":["gzip, deflate"],"User-Agent":["OhDear.app (+https://ohdear.app/docs/checks/uptime)"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"http/1.1","server_name":"arm.stbu.net"}},"bytes_read":0,"user_id":"","duration":0.004788534,"size":32,"status":302,"resp_headers":{"Content-Type":["text/plain; charset=utf-8"],"Content-Length":["32"],"Date":["Tue, 03 Oct 2023 09:28:06 GMT"],"X-Frame-Options":["SAMEORIGIN"],"Location":["/dashboard"],"Server":["TheRock/2.0"],"Vary":["Accept"]}}' | \
/home/stbu/goaccess --log-format=CADDY -o /tmp/goaccess-caddy.html -
Since the values of 772
for the TLS version and 4865
for the cipher-suite are not very user-friendly in the default Caddy JSON as well as in the GoAccess HTML report (if they would show up), I have modified the Caddy JSON log configuration like this:
log access-json {
include http.log.access
output file log/access.json
format filter {
wrap json
fields {
request>tls>version tls_version TLSv
request>tls>cipher_suite tls_cipher
}
}
}
and that produces this human friendly output (but the field labels are the same, just now strings instead of integers)
"tls":{"resumed":false,"version":"TLSv1.3","cipher_suite":"TLS_AES_128_GCM_SHA256","proto":"http/1.1","server_name":"arm.stbu.net"}
If I use your suggested --log-format
it's working for me:
echo '{"level":"info","ts":1696324268.7687688,"logger":"http.log.access.arm.stbu.net","msg":"handled request","request":{"remote_ip":"167.71.54.244","remote_port":"36770","client_ip":"167.71.54.244","proto":"HTTP/1.1","method":"GET","host":"arm.stbu.net","uri":"/","headers":{"Accept-Encoding":["gzip, deflate"],"User-Agent":["OhDear.app (+https://ohdear.app/docs/checks/uptime)"],"Accept":["*/*"]},"tls":{"resumed":false,"version":"TLSv1.3","cipher_suite":"TLS_AES_128_GCM_SHA256","proto":"http/1.1","server_name":"arm.stbu.net"}},"bytes_read":0,"user_id":"","duration":0.003010369,"size":32,"status":302,"resp_headers":{"Content-Length":["32"],"Date":["Tue, 03 Oct 2023 09:11:08 GMT"],"Server":["TheRock/2.0"],"X-Frame-Options":["SAMEORIGIN"],"Location":["/dashboard"],"Vary":["Accept"],"Content-Type":["text/plain; charset=utf-8"]}}' | \
/home/stbu/goaccess --log-format='{ "ts": "%x.%^", "request": { "remote_ip": "%h", "proto": "%H", "method": "%m", "host": "%v", "uri": "%U", "headers": { "User-Agent": [ "%u" ], "Referer": [ "%R" ] }, "tls": { "cipher_suite": "%k", "version": "%K" } }, "duration": "%T", "size": "%b", "status": "%s", "resp_headers": { "Content-Type": [ "%M" ] } }' --date-format=%s --time-format=%s -o /tmp/goaccess-caddy.html -
So basically, the --log-format=CADDY
is either not capturing the %K
and %k
properly (maybe because the JSON fields have changed since its inception), or the integer values of e.g. 772
for the TLS version and e.g. 4865
for cipher-suite are not recognized and shown by GoAccess.
However, I am very happy with the modified Caddy JSON and your suggested solution of --log-format='{ "ts": "%x.%^", "request": { "remote_ip": "%h", "proto": "%H", "method": "%m", "host": "%v", "uri": "%U", "headers": { "User-Agent": [ "%u" ], "Referer": [ "%R" ] }, "tls": { "cipher_suite": "%k", "version": "%K" } }, "duration": "%T", "size": "%b", "status": "%s", "resp_headers": { "Content-Type": [ "%M" ] } }' --date-format=%s --time-format=%s
from goaccess.
Related Issues (20)
- Browsers an operating systems aren't recognized HOT 2
- Plans to track other search engines such as Yahoo, Bing, Duckduckguo etc. in addition to Google? HOT 6
- Problem with the Wold Map HOT 11
- The "Unique Visitors Per Day" pane always showing a single dot HOT 16
- Feature request: New panel with last log requests HOT 2
- Goaccess 1.9.2 and 1.9.3 (Those where the ones tested) don't write some panels information that work on CLI HOT 3
- GoAccess crashed :( HOT 1
- Facing Fatal Error. Attempting to analyze log text file in my documents folder for a side project. HOT 1
- Trying to parse custom nginx log file, logging tls versions HOT 4
- Unique visitor count too high, parse problem? HOT 2
- doesn't match specifier '%s' HOT 2
- Web socket configuration for real time html report HOT 8
- Real-time JSON export HOT 2
- How to actually persist?
- Token '0.085]' doesn't match specifier '%s HOT 5
- Custom JSON format for Caddy log structure HOT 6
- confirm config file format for GoAccess config file (for syntax highlighting in Vim) HOT 2
- Strip URLS and ports to get only hostnames HOT 1
- Should Tx. Amount be similar to Elastic Load Blanacing usage?
- What does %H parse? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from goaccess.