jdrews/logstation

Tail log files in web browser

logstation

Tails a set of log files and serves them up on a web server with syntax colors via regex.

Binaries available in releases. See usage below.

Goals:

• Run on anything and everything
• Support as many browsers as possible
• Ease deployment and usage with a single executable with minimal configuration required

Developed with Go and React

Usage

• Call logstation or logstation.exe
  • It will create an logstation.conf in your current directory if one doesn't exist and exit
  • Update logstation.conf as desired
  • Call logstation or logstation.exe again
• Navigate to http://127.0.0.1:8884 to start tailing (refer to logstation.conf for listen IP and port)

You can also use -c your-logstation.conf argument to specify a config file

Take a look at an example logstation.conf here.

Building

logstation uses goreleaser for all releases

Start by building the web frontend:

• cd web; yarn build

To build all targets locally you can run the following in the root directory:

• goreleaser build --snapshot --clean

If you want to build for a specific target you can set environment variables

• In bash this would look like:
  GOOS=linux GOARCH=amd64 goreleaser build --snapshot --clean --single-target
• In powershell this would look like:
  $env:GOOS="linux"; $env:GOARCH="amd64"; goreleaser build --snapshot --clean --single-target

Reference the releases and .goreleaser.yaml for all officially supported targets.

Releasing

• Push a tag following semver
  • git tag -a 2.0.0-beta1 -m "2.0.0-beta1"
  • git push origin 2.0.0-beta1
• Ensure you have an environment variable with GITHUB_TOKEN="YOUR_GH_TOKEN" and minimum of write:packages permissions
• Release!
  • cd web; yarn build
  • cd ..; goreleaser release
• Read the goreleaser quickstart for more details

Versions

Prior to 2.x, this app was built using Scala/Play/JS. At 2.x this app was rewritten in Go and React. If you're looking for the older versions reference the releases prior to 2.x.

logstation's Issues

Regex with + lines after or + lines around

Like grep -A or grep -C

Security vulnerability - Cross-Site Scripting

I used "logstation" to log HTTP requests, I noticed the "logstation" interprets HTML.
For example, if I send http://xxx.xxx.xxx/xxx.php?a=<dETAILS%0aopen%0aonToGgle%0a=%0aa=prompt,a() x>

And I log the requests and display it in "logstation" the HTML will be interpreted

Title change color on exception

turn title red if exception pops up --- really low priority

maxLogLinesPerLog conf doesnt apply

still at 1000. not what the conf says

Publish an official docker image

Feature: highlight in realtime/on the fly (vs config based)

Add ability to add highlighting of words on the fly via regex.

Migrate to Play with Websockets

Most browsers support websockets nowadays. And Scala selected Play as it's framework so best to go with whats canonical.

Add button to stop/restart watch

The lib node-tail used in this project enables to start/stop watch:

tail.unwatch()

It's very good for many logs in little time when necessary analyse the log, so watch stop and after, start watch again.

Marking your spot

Allow user to press enter a few times to mark their spot

Or allow user to highlight section by clicking it so they know when new data comes in.

Wildcards or directories?

This is a life saver. I love it! Thank you. I'm wondering how difficult it would be to add a list of files from the directory so we caught pick which one to view? My log files change each day, so it would be great not to have to reset this every day. Maybe with a wildcard filter like *.log or *.txt?

setup automated dependency management

dependabot, renovatebot, etc

CTRL + C to stop tailing

index all daily logs and supply search service

index all daily logs
1, old logs
2, old logs compressed in gz
3, current logs, update automatically

search service, support line parse:
1, line is a whole string, optional equals to : {"line":"log line"}
2, line => json, ex: {"ip":"address","msg":"message","level":"INFO"}
sometimes line parse is difficult, so option 1 is good enough

Windows Server 2012 Support

Issues with Windows Server 2012 and file handlers. It keeps making new tabs.

buffered messages do not go to client on connect

logstation does not seem to respect "-c" argument

I tried using logstation -c, but it does not seem to respect it.

$ mv logstation.conf logstation_othername.conf
$ java -jar logstation-0.3.10.jar -c logstation_othername.conf 
Welcome to logstation! Creating default logstation.conf...
Please setup your logstation.conf located here: /home/kbock/tincture3/logstation.conf
Then relaunch and logstation will be located at: http://127.0.0.1:0

Am I passing the flags incorrectly? Thanks for your help!

If the window size changes, the snap to bottom tailing fails

Reproducing steps

• Open logstation and begin watching a log
• Scroll up to pause the snap to bottom tailing
• Resize the window to smaller
• Now click the resume button
• The log will scroll to the bottom, but the next log line will cause the resume button to show up again and the snap to bottom tailing will fail.

Connections get dropped

It looks like connections get dropped or stop getting updated. I see "Received shutdown" and "session expired" messaged for threads and don't know if those are the related connections.
Also after refreshing the browser, no additional updates are received (clearing the cookie in Chrome will sometimes allow updates to continue). I'm running on CentOS6.10, java 1.7.0_67

Pause button to stop backend from sending log message

for reducing bandwidth on slow and small links (i.e. legacy satellite uplinks)

Start attached to bottom of log

Currently user has to scroll down to begin tailing. Make it tail at boot.

multiple clients cause duplicate log links

Open more than one client sequentially. Each time the previous clients receive duplicate, or triplicate (sp?) log links, and so forth.

Convert all libraries to use WebJars

Use WebJars and sbt-web to copy libraries to the correct place on compilation.

Some lines are affected with unexpected symbols

I'm using Logstation inside Docker container. And I don't know why, but some lines are breached with this symbols: [31m and [39m.

Expected behavior and the string from a original file:
[2018-11-12T14:49:56.008Z] - ERROR: ESOCKETTIMEDOUT

I'm getting in the Logstation:
[[31m2018-11-12T14:49:56.008Z[39m] - [31mERROR[39m: ESOCKETTIMEDOUT

Maybe it's not an error, but some container environment issue, I don't know.

Too many logging messages going to stdout from command line

Need to cleanup the log messages so it doesn't spam stdout.

Configurable Web Port

webserver port number change via conf file

Enhancement - Log Tab Title

Great project. Thank you.

The tabs for log files display the name truncated from the start of the path. Like "/some/folder/..." but, in my opinion, it would be better to show the end of the path like "../log/service.log" (and truncate at the start).

I know that you can hover over the tab to see the full file path but I thought this might be easier.

Alternatively, it would be cool to provide some way to specify the tab title for a particular log in the config file. This might not work so well for wildcards.

And it would be great if you could specify a tab colour for a particular log (or set of logs).

Make Embeddable

It would be nice if this were available as an embeddable library. Steps required would be:

1. Allow application to be started with programmatically generated Conf instance. At the moment it must be a file specified in command line args
2. Make JAR available on Maven Central

Use font-awesome or glyphicons instead of text

increase default log amount to 1500

Cleanup comments

Pause Log Button

Add pause button so it doesn't update on slow links.

Confirm no data transfer if log not showing.

Sometimes logstation does not start tailing files

Original problem: Sometimes when logstation starts, it just does not tail any files.

I cannot tell the difference between a launch that will fail or succeed. Once the next run does not work, I will post the full traceback.

==================================================
Original issue I opened/errors reported: (though all of these appear in both the successful and unsuccessful runs)

I'm seeing these two errors often before logstation stops working, so I figured I'd open an issue just to ask if they are a problem, or something else.

06:44:41.194 [main] INFO o.e.j.w.StandardDescriptorProcessor - NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet

06:44:41.407 [main] ERROR net.liftweb.util.Props - Failed to find a properties file (but properties were accessed). Searched: /props/root.ubuntu.props, /props/root.props, /props/ubuntu.props, /props/default.props, /root.ubuntu.props, /root.props, /ubuntu.props, /default.props

On shutdown (when nothing shows up in logstation):
[INFO] [10/15/2018 06:49:44.371] [LogStation-akka.actor.default-dispatcher-7] [akka://LogStation/user/LogStationServiceActor] Message [akka.actor.Terminated] from Actor[akka://LogStation/user/LogStationServiceActor/LogTailerActor-:worker0:#-649459367] to Actor[akka://LogStation/user/LogStationServiceActor#1772589764] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.

Regex log filtering

Filter logs based on regex (so you could watch one resource -- or type of message) --- idea is to be able to filter log on a parameter line --- think about making it happen on the server side to conserve bandwidth

Make temporary directory configurable

I saw that logstation is writing stuff in "/tmp".
It would be nice to have an option in the config to set another place for temporary storage.

Possibility of colliding lines

The beginning and end of subsequent lines can collide on select situations. This is an upstream problem in react-log-viewer which is being tracked in patternfly/react-log-viewer#3

Feature Request: Custom html title

Please create a configuration parameter to set custom html title other than "logstation". I want to differ multiple logstation browser tabs...

And thanks for this nice tool. It's great.

Don't auto switch to new log when created in GUI

gui improvements

1. Follow bottom of page if you scroll to bottom, otherwise don't follow
2. regex based log color syntax
3. check to make sure it handles more than two logs
   make sure you can scroll top bar (when logging many log files at once and top bar gets crowded)
4. reduce font size to show more code
5. do I need to drop off some lines so the page doesn't get too large

Is Authentication available?

Hello,

The app is really cool, which could help me a lot.Just wanted to check if authentication is available.

Logstation stops working after refresh

It's unclear to me why this is, and it seems to affect 3.10 and 3.11. If you get logstation open and are tailing files, if you refresh the page for any reason, logstation stops working - no files show up to be tailed.

Minimize header height to increase log visibility

Gets slow when truncating

log 3 or more logs
then when it gets to working on truncating lines it crashes out

Feature Request: Close log file

Close a log file so we don't listen to it's messages anymore.

Make sure footer is visible at all times

When you scroll up it disappears, which can make it hard to enable following or pausing

logstation seems to crash if BROWSE is not available

I'm excited to use logstation and I'd like to configure to use it automatically in my environment. However, when I start a logstation instance programatically, it immediately dies:

Exception in thread "main" java.lang.UnsupportedOperationException: The BROWSE action is not supported on the current platform!
	at java.awt.Desktop.checkActionSupport(Desktop.java:225)
	at java.awt.Desktop.browse(Desktop.java:381)
	at com.jdrews.logstation.LogStation$delayedInit$body.apply(LogStation.scala:124)
	at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App$$anonfun$main$1.apply(App.scala:71)
	at scala.App$$anonfun$main$1.apply(App.scala:71)
	at scala.collection.immutable.List.foreach(List.scala:318)
	at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
	at scala.App$class.main(App.scala:71)
	at com.jdrews.logstation.LogStation$.main(LogStation.scala:34)
	at com.jdrews.logstation.LogStation.main(LogStation.scala)

Am I missing something with how I should run it?

binding to 0.0.0.0 issue

When running the application, service binds to all interfaces by default. That is, it binds to 0.0.0.0 on the specified port.
From a security perspective, this is not desirable. The default behavior of any service should be to bind only to the localhost (127.0.0.1) interface by default, and allow overriding the bind address via config file.

truncate titles/links to X number of characters (show endings)

Setup goreleaser

https://goreleaser.com/

Screenshot

Add Screenshot below to Readme.md

new client connection causes old client to reset logs