brianpetro/logger

use your IDE as Logging UI - a fast, simple, extensible, zero dependency Node.js logging tool that starts simple in development and is easily extended for production

smol logger

An extensible, fast logging tool that uses the filesystem as Log UI.

Features

• Logs to both the console and local json files for easy navigation/versioning
• Logs time elapsed, filepath/line number of log call
• Extensible
  • Extend the log store to a remote store, e.g. LogFlare
  • Customize everything from naming to terminal log color
• Zero dependency, <100 LOC
• MIT Open Source: https://github.com/smol-ai/logger
• (todo) tested thanks to Codium AI

Non-goals:

• no log levels. too complicated - just add a descriptive log name, and either send all logs or none at all
• not going for running in the browser - for now. open an issue to discuss

Usage

install:

npm install @smol-ai/logger

use:

import { SmolLogger } from '@smol-ai/logger';

const logger = new SmolLogger(true)
const log = logger.log

log('name of log (required)', payload) // basic usage

All logs are sent to the console and file system by default, so you can easily navigate in your IDE:

The logs look like this!

The log function is a single arity "identity function" - returns the payload so you can modify in place.

mySingleArityFunction({foo, bar, baz}) // oh no i need to log bar and baz
mySingleArityFunction({foo, ...log({bar, baz})) // done!
mySingleArityFunction(log({foo, bar, baz})) // log them all why not

myBadMultiArityFunction(foo, bar, baz) // oh no i need to log bar
myBadMultiArityFunction(foo, log(bar), baz) // done!
myBadMultiArityFunction(...log({foo, bar, baz}).values()) // log them all why not

We default to single arity to encourage this in the JS ecosystem.

Extend to Remote Storage

For production logging, Smol Logger's storage destination can be overwritten. We like Logflare!

// OPTIONAL: store the local file store for reuse
const oldStore = logger.store

// override the default local file store with your own remote filestore
// { logName: string, loggedLine: string | null, payload: any, secondsSinceStart: number, secondsSinceLastLog: number }
logger.store = ({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog }) => {
  fetch("https://api.logflare.app/logs/json?source=YOUR_SOURCE_ID", {
    method: "POST",
    headers: {
      "Content-Type": "application/json; charset=utf-8",
      "X-API-KEY": "YOUR_API_KEY_HERE"
    },
    body: JSON.stringify({ message: logName, metadata: {loggedLine, payload, secondsSinceStart, secondsSinceLastLog }})
  })
  // OPTIONAL: run the local file store as well
  oldStore({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog })
}

If you expect high volumes of logs, you should batch them:

const logMessages = []

function throttle(func, delay = 1000) {
  let timeout = null;
  return function(...args) {
    if (!timeout) {
      timeout = setTimeout(() => {
        func.call(this, ...args);
        timeout = null;
      }, delay);
    }
  };
}

const sendToLogFlare = ({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog }) => {
  fetch("https://api.logflare.app/logs/json?source=YOUR_SOURCE_ID", {
    method: "POST",
    headers: {
      "Content-Type": "application/json; charset=utf-8",
      "X-API-KEY": "YOUR_API_KEY"
    },
    body: JSON.stringify({"batch": logMessages})
  })
  .then(() => logMessages = [])
}

log.store = throttle(sendToLogFlare, 1000)

Async/Blocking logging

Logging is synchronous by default in smol logger.

Note that in the above example we are firing off an async fetch inside of a synchronous call. If your application crashes there is a smol chance that the log may not complete sending since it is running asynchronously. If you need to block for an async call, you can overwrite the asyncLog method:

// OPTIONAL: store the local file store for reuse
const oldStore = logger.store

// override the default local file store with your own remote filestore
// { logName: string, loggedLine: string | null, payload: any, secondsSinceStart: number, secondsSinceLastLog: number }
logger.store = async ({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog }) => {
  const res = await fetch("https://api.logflare.app/logs/json?source=YOUR_SOURCE_ID", {
    method: "POST",
    headers: {
      "Content-Type": "application/json; charset=utf-8",
      "X-API-KEY": "YOUR_API_KEY_HERE"
    },
    body: JSON.stringify({ message: logName, metadata: {loggedLine, payload, secondsSinceStart, secondsSinceLastLog }})
  }).then(res => res.json())
  // just demonstrating that you can await inside this store
}

// now you can block execution in an async context
await logger.asyncLog('my message here', { foo: 'bar' })

Note: this funcitonality is new and not tested, please try out and give fixes/feedback

Advanced: Intercept input vs output

This logs BOTH input and output of a function that you want to monitor.

const wrapped = logger.intercept(openai.createChatCompletion.bind(openai)) // binding is impt bc of how OpenAI internally retrieves its config
const response = await wrapped({
        model: process.env.OPENAI_MODEL || 'gpt-3.5-turbo',
        messages: [/* etc */ ],
    });

Sometimes the output can be very verbose (as is the case with OpenAI chatCompletion). so we also allow you to expose a simple "log transformer" that is an arbitrary function to modify the intercepted output to a format of your choice that you like:

// custom 
const wrapped = logger.intercept(
  openai.createChatCompletion.bind(openai),  // binding is impt bc of how OpenAI internally retrieves its config
  {
    logTransformer: (args, result) => ({
      // ...result, // optional - if you want the full raw result itself
      prompt: args[0].messages,
      response: result.data.choices[0].message,
    })
  }
)
const response = await wrapped({
        model: process.env.OPENAI_MODEL || 'gpt-3.5-turbo',
        messages: [/* etc */ ],
    });
// now the logged response has highlighted the specific fields of our interest

The logTransformer function can be async in case you need it.

Customize everything else

General rule for customizing is just overwrite any of the variables and methods exposed in the class:

logger.logDirectory = '.smol-logs' // change default file store directory

logger.logToConsole = false // turn off logging to terminal
logger.logToStore = false // turn off logging to store

logger.logName = (name: string) => `My custom logName: ${name}` // change Log naming!

logger.LOGCOLOR = (logName: string) => "\x1b[35m" + logName + "\x1b[0m"; // set log colors to magenta instead of yellow

Other log colors to try:

\x1b[31m: Red
\x1b[32m: Green
\x1b[33m: Yellow
\x1b[34m: Blue
\x1b[35m: Magenta
\x1b[36m: Cyan
\x1b[37m: White

Get creative! For example you can stack logname functions...

// for an advanced application, you can add log names in a stack
const logStack = [logger.logName] // store original logName fn at the bottom of the stack
logger.logName = (name) => logStack.reduce((prev, fn) => fn(prev), name)
let temp = 0
do {
  logStack.unshift(name => '   ' + name)
  // everything logged here is one indent in
  log('logname1 here ' + temp, temp)
    let temp2 = 0
    do {
      logStack.unshift(name => '   ' + name)
      // everything logged here is two indent in
      log('logname2 here ' + temp, temp)
      logStack.shift()
    } while (temp2++ < 5)
  logStack.shift()
} while (temp++ < 5)

Contributor notes

this repo was bootstrapped with https://github.com/alexjoverm/typescript-library-starter (must modify with alexjoverm/typescript-library-starter#333)

Publishing

Follow the console instructions to install semantic release and run it (answer NO to "Do you want a .travis.yml file with semantic-release setup?").

Note: make sure you've setup repository.url in your package.json file

npm install -g semantic-release-cli
semantic-release-cli setup
# IMPORTANT!! Answer NO to "Do you want a `.travis.yml` file with semantic-release setup?" question. It is already prepared for you :P

From now on, you'll need to use npm run commit, which is a convenient way to create conventional commits.

Automatic releases are possible thanks to semantic release, which publishes your code automatically on github and npm, plus generates automatically a changelog.

note from swyx - wasnt able to get the publishing to work, kept failing.

for now:

npm version patch
npm publish --access=public