Giter Site home page Giter Site logo

memory usage limit about goaccess HOT 5 CLOSED

tyctor avatar tyctor commented on May 23, 2024
memory usage limit

from goaccess.

Comments (5)

allinurl avatar allinurl commented on May 23, 2024 1

Great! Stay tuned for updates! There might be a possibility of an on-disk option in the future.

from goaccess.

allinurl avatar allinurl commented on May 23, 2024

If you're looking to reduce some RAM usage, here are a few things you could try out. First off, do you know if your requests are timestamped? If so, you might want to consider passing -q. Also, are you currently using the legacy GeoIP? If yes, give -g a shot. Another option is to SSH into the remote machine and parse the log locally on your own laptop/desktop machine (assuming it's got more RAM), e.g.,

ssh -n root@server 'tail -F -n +0 /var/log/apache2/access.log' | goaccess - --log-format=COMBINED -o report.html --real-time-html

Note: SSH requires -n so GoAccess can read from stdin. Also, make sure to use SSH keys for authentication as it won't work if a passphrase is required.

from goaccess.

tyctor avatar tyctor commented on May 23, 2024

thanks, for your reply
locally parsing is OK, that works, but i want to have this completely on server, ofcourse only if it will be possible

if your requests are timestamped

i am not sure what you mean by timestamped
my log record look like this:

31.30.175.177 - - [12/May/2022:02:19:02 +0000] "GET /api/radio/programme/playingnow/?_nc=1652321942&device=desktop HTTP/2.0" 200 485 "https://radiopunctum.cz/archive/20220426_chaosfera" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36"

so this is probably not timestamp, only date/time of request

i forgot to put --geoip-database ../geoip/GeoLite2-City_20240412/GeoLite2-City.mmdb in my example
so i am using mmdb instead of legacy GeoIP

mainly what i asked for, is this part of man:

STORAGE
There are three storage options that can be used with GoAccess. Choosing one will depend on your environment and needs.
Default Hash Tables
In-memory storage provides better performance at the cost of limiting the dataset size to the amount of available physical memory. GoAccess uses in-memory hash tables. It has very good memory usage and pretty good performance. This storage has support for on-disk persistence.

i cannot find what are that three options and how to tell goaccess which storage to use.

from goaccess.

allinurl avatar allinurl commented on May 23, 2024

I'm not certain how that text managed to stick around from an older version that stored data on disk, but I've gone ahead and updated the documentation to match the current versions. Did using -q make a difference? In your situation, using -g won't be beneficial since you're utilizing the newer geodb.

from goaccess.

tyctor avatar tyctor commented on May 23, 2024

thanks for reply
i decide to do some log filtering before i send it to goaccess, i think this will be enough to solve memory usage

from goaccess.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.