parabuzzle/cistern

The logs collector

• 👋 Hi, I’m Mike Heijmans
• 👀 I’m interested in finance tech, all things DRY and SOLID, and learning new things
• 🌱 I’m currently learning embedded system design and devlopment
• 💞️ I’m looking to collaborate on animatronics, finance tech, and embedded system realtime operating systems
• 📫 How to reach me message me on Mastodon: https://mastodon.social/@[email protected]

cistern's Issues

[Feature] Agent execute script

It would be nice if the agent could alternatively run a script instead of tailing a file that would return event data on stdout. This would allow someone to roll a script to watch a collection of files in a directory or whatever they want and send events via stdout.

Hide stack trace by defult.

On the front page, hide the stack trace by default and have a little js to expand the full details.

[Browser] Searchbox is in wrong place in Safari and IE

The searchbox pops to the left in safari.

Log rotation needs to be added

There should be auto log rotation that is settable via the frontend. This should also be accompanied with a database maintenance script that can compress the events table for innodb support.

Given that the delete command removes rows from the database but does not free innodb pool space, over time the database will become very bulky and fragmented. There should be some way to dump the events table and then reload it to recover the innodb pool space and defragment the data. This could be done via a frontend trigger or a backend task... don't know yet. Just a thought.

Need a search box on the results pages for agent and logtype

It makes it hard to search for something else inside the model after the first search

Agent hostname does not allow localhost

The hostname field has a validation for FQDN only. This fails if the last section of the hostname doesn't include a valid TLD like .com or .net.

This needs to allow for localhost and localdomain at the very least. Assuming that the person setting up cistern knows what they are doing, we can probably pull that validation check.

Searching within a logtype does not work

doing a search within a logtype doesn't work.

ERROR:
undefined method `logtype' for "#<Staticentry:0x261e084>":String

Delete needs to be added

There needs to be delete functions.

Removing an agent should remove logtype associations and all events keyed to that agent.
Removing a logtype should remove all agent associations and all static entries keyed to that logtype.
When static entries are removed all events for that static entry should be removed.

Filter by Loglevel

It would be nice to filter by loglevel.

[XSS] No input validation and sanitation on Log Collector

Raw characters are sent and saved to the database without any sanitation.

Server crashes when searching

An unknown error occurs when running a search query against the ferret index. The error causes the frontend server to crash.

Command output is not fully sanitized.

The command output should be raw output with full sanitization. Almost like a < code > block

Event data should expand above the event.

The data should expand above the event because a large stack trace makes it hard to see the event data without scrolling. Maybe a nice slide open would be good.

[XSS] No character sanitation on Frontend

Raw characters that have been saved to the database are not sanitized for display. This leaves the frontend open to XSS!

Log collector does not verify agent to type association

I can send log events with any logtype_id for an agent and the event will be logged. This causes two problems:

1. If the agent is not a member of the logtype then there is no way to do cross association filtering and metrics

2. if the logtype doesn't exist, then the event will be shown in the all events view and agent view only and it will cause certain detail views to break.

Proposed Solution: The log collector should verify that the logtype_id in the event is associated to the agent sending the event. If not, punt the event. If so, log it.

Commands output doesn't format well

When a command such as ps auxx is returned to the server the view doesn't respect spaces and command line columns. The view should respect spaces and columns provided by the command so that columns line up properly.

search results need and order by

The search results come back in a seemingly random order. I am sure it's just missing some sort of order by.

Search results don't honor line breaks

Line breaks are sanitized and escaped. So formatting is jacked on search result pages if the event has new lines.

Newline in event not honored on frontend

new lines in a log event are not honored in the frontend. This makes it hard to read stack traces :)

etime column in events database should be a float

Currently the column is a string. This should be changed to a float for better sorting.