mjfryc/mjaron-tinyloki-java

Tiny Grafana Loki client (log sender) written in pure Java 1.8 without any dependencies.

mjaron-tinyloki-java

Tiny Grafana Loki client (log sender) written in pure Java 1.8 without any external dependencies.

• Implements JSON variant of Loki API
• Works with Android and Java SE
• Thread safe

API description

HTTP sender requires http URL, optionally basic authentication credentials may be set.

Short example:

import pl.mjaron.tinyloki.*;

public class Sample {
    public static void main(String[] args) {

        LogController logController = TinyLoki
                .withUrl("http://localhost/loki/api/v1/push")
                .withBasicAuth("user", "pass")
                .start();

        ILogStream stream = logController.stream() // Before v0.3.2 use createStream()
                .info()
                .l("host", "MyComputerName")
                .l("customLabel", "custom_value")
                .build();

        stream.log("Hello world.");

        logController.softStop().hardStop();
    }
}

Verbose example:

import pl.mjaron.tinyloki.*;

public class Sample {
    public static void main(String[] args) {

        // Initialize log controller instance with URL.
        // Usually creating more than one LogController instance doesn't make sense.
        // LogController owns separate thread which sends logs periodically.
        LogController logController = TinyLoki
                .withUrl("http://localhost/loki/api/v1/push")
                .withBasicAuth("user", "pass")
                .start();

        // Create streams. It is thread-safe.
        ILogStream stream = logController.createStream(
                // Define stream labels...
                // Initializing log level to verbose and adding some custom labels.
                TinyLoki.verbose()
                        .l("host", "MyComputerName")        // Custom static label.
                        .l("customLabel", "custom_value")   // Custom static label.
                // Label names should start with letter
                //     and contain letters, digits and '_' only.
                // Bad characters will be replaced by '_'.
                //     If first character is bad, it will be replaced by 'A'.
        );

        // ... new streams and other logs here (thread-safe, non-blocking).
        stream.log("Hello world.");

        // Optionally flush logs before application exit.
        logController
                .softStop()     // Try to send logs last time. Blocking method.
                .hardStop();    // If it doesn't work (soft timeout) - force stop sending thread.
    }
}

Integration

Maven Central

    implementation 'io.github.mjfryc:mjaron-tinyloki-java:0.3.11'

Maven Central page, Maven Central repository URL

GitHub Packages

Click the Packages section on the right.

Download directly

1. Click the Packages section on the right.
2. Find and download jar package from files list to e.g. your_project_root/libs dir.
3. Add this jar to project dependencies in build.gradle, e.g:

    implementation files(project.rootDir.absolutePath + '/libs/mjaron-tinyloki-java-0.3.11.jar')

API design

classDiagram
class Labels {
    l(name, value) // Add the label
}

class ILogStream {
    <<Interface>>
    log(content)
    log(timestamp, content)
    release()
}

class ILogCollector {
    <<Interface>>
    createStream(labels)
    collect()
    contentType()
    waitForLogs(timeout)
}

class ILogEncoder {
    <<Interface>>
    contentEncoding(): String
    encode(final byte[] what): byte[]
}

class ILogMonitor {
    <<Interface>>
    onConfigured(contentType, contentEncoding)
    onEncoded(in, out)
    send(message)
    sendOk(status)
    sendErr(status, message)
    onException(exception)
    onWorkerThreadExit(isSoft)
}

class ILogSender {
    <<Interface>>
    configure(logSenderSettings, logMonitor)
    send(bytes)
}

class LogController {
    workerThread: Thread
    stream(): StreamBuilder
    createStream(labels)
    start()
    softStop()
    hardStop()
}

class GzipLogEncoder

class VerboseLogMonitor
class ErrorLogmonitor
class JsonLogStream
class JsonLogCollector

class Settings {
    +start(): LogController
}

class TinyLoki {
    +withUrl(url)$
}

VerboseLogMonitor --|> ILogMonitor: implements
ErrorLogmonitor --|> ILogMonitor: implements

ILogStream <.. ILogCollector: create
ILogCollector --* LogController
Labels <.. ILogCollector : use
ILogSender --* LogController
GzipLogEncoder --|> ILogEncoder: implements
ILogEncoder --* LogController
ILogMonitor --* LogController
JsonLogStream --|>  ILogStream: implements
JsonLogCollector --|> ILogCollector: implements
JsonLogStream <.. JsonLogCollector: create
HttpLogSender --|> ILogSender: implements
DummyLogSender --|> ILogSender: implements
Settings <.. TinyLoki: create
LogController <.. Settings: create

mjaron-tinyloki-java's Issues

API: Create Labels from other Labels (copy constructor), with delegation in TinyLoki static method.

Implement label length limit

Loki Server may not accept logs due to label length limit.

Grafana Loki server configuration with default label length limits:

https://grafana.com/docs/loki/latest/configuration/

# Maximum length of a label name.
# CLI flag: -validation.max-length-label-name
[max_label_name_length: <int> | default = 1024]

# Maximum length of a label value.
# CLI flag: -validation.max-length-label-value
[max_label_value_length: <int> | default = 2048]

Separate the worker thread from `LogController`

Worker thread - periodic action of log collecting - should be separated, by:

• Extracting interface of any worker thread, e.g. IWorker
• Write IWorker implementation with single thread, named WorkerThread
• Write WorkerReactiveX implementation which use rx-java instead of separated thread. Add it as different java project [In future, not covered by this issue].
• Re-thing about softStop() and hardStop() methods implementation.
• Re-thing how to implement it, to give possibility to implement #3.

Thanks for you work.... this library is perfect for our production solution :)

Create test coverage badge

Make Prettification optional

Hi, I have added the builder methods withPrettify() and withoutPrettify() to Labels (locally) as my labels value were being corrupted and I couldn't link between Loki and Tempo.

I tested and it would seem that Loki can now (v2.6.1) accept unprettified labels/values.

Do you think this is a good idea?

Paul.

Implement gzip content encoding

Key of label should be json-string-escaped

mjaron-tinyloki-java/src/main/java/pl/mjaron/tinyloki/JsonLogStream.java

Line 31 in 0b4155f

b.append(entry.getKey());

Remove "Connection: Close" header

The connection close header is unnecessary and should be removed.

Class Labels overrides equals but not hashCode.

@SuppressWarnings("unused")
public class Labels implements Cloneable {
Class Labels overrides [equals](https://lgtm.com/projects/g/mjfryc/mjaron-tinyloki-java/snapshot/d3f67ecc523b952ef7c612d302fab279b3ed770e/files/src/main/java/pl/mjaron/tinyloki/Labels.java#xd72d4c252d47884:1) but not hashCode.

Write unit tests

Dereferenced variable may be null

Variable logEncoder may be null here because of this null argument.

Enable Gzip content encoding by default

• There is no reason to skip Gzip content encoding
• Skipping encoding make any sense for diagnostic purposes only
• It is a functional change - it requires minor version update

Add possibility to resend logs on connection failure.

This task is similar to #20.