An extensible, fast logging tool that uses the filesystem as Log UI.
![image](https://private-user-images.githubusercontent.com/6764957/246623036-a862f61d-9459-42d2-bab7-572231c5c8e8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ3MDUzMDUsIm5iZiI6MTcyNDcwNTAwNSwicGF0aCI6Ii82NzY0OTU3LzI0NjYyMzAzNi1hODYyZjYxZC05NDU5LTQyZDItYmFiNy01NzIyMzFjNWM4ZTgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjZUMjA0MzI1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTQ0NmJlODYxYzEyMjQ3NjE4ZjY5MWExMTIwY2Q3ZGU1MGRhOTUxMTEzMTg0ODQzMGJjM2E3NTMxMDRiNmQ0NCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.NdBGRRWU66KmjWRh5Gsa3wUoGxC8-nwUY1VUcmf6fKw)
- Logs to both the console and local json files for easy navigation/versioning
- Logs time elapsed, filepath/line number of log call
- Extensible
- Extend the log store to a remote store, e.g. LogFlare
- Customize everything from naming to terminal log color
- Zero dependency, <100 LOC
- MIT Open Source: https://github.com/smol-ai/logger
- (todo) tested thanks to Codium AI
Non-goals:
- no log levels. too complicated - just add a descriptive log name, and either send all logs or none at all
- not going for running in the browser - for now. open an issue to discuss
install:
npm install @smol-ai/logger
use:
import { SmolLogger } from '@smol-ai/logger';
const logger = new SmolLogger(true)
const log = logger.log
log('name of log (required)', payload) // basic usage
All logs are sent to the console and file system by default, so you can easily navigate in your IDE:
![image](https://private-user-images.githubusercontent.com/6764957/246622850-348a3f49-1022-4763-9b63-cf0e3080b85a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ3MDUzMDUsIm5iZiI6MTcyNDcwNTAwNSwicGF0aCI6Ii82NzY0OTU3LzI0NjYyMjg1MC0zNDhhM2Y0OS0xMDIyLTQ3NjMtOWI2My1jZjBlMzA4MGI4NWEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjZUMjA0MzI1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTljYzFmYjY2MmYyM2Q5MjRmMjQ0ZTBkNjMyYmViNGUyN2E1MmUwZDA3MzllN2Q5ZjQ2ODUxZDViNTc4OTQ2YSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.Q7R6tcQhXjgKOkmWQeXtvB2148hJPRCrQg7-SK8Jpvo)
The logs look like this!
![image](https://private-user-images.githubusercontent.com/6764957/246623036-a862f61d-9459-42d2-bab7-572231c5c8e8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ3MDUzMDUsIm5iZiI6MTcyNDcwNTAwNSwicGF0aCI6Ii82NzY0OTU3LzI0NjYyMzAzNi1hODYyZjYxZC05NDU5LTQyZDItYmFiNy01NzIyMzFjNWM4ZTgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjZUMjA0MzI1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTQ0NmJlODYxYzEyMjQ3NjE4ZjY5MWExMTIwY2Q3ZGU1MGRhOTUxMTEzMTg0ODQzMGJjM2E3NTMxMDRiNmQ0NCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.NdBGRRWU66KmjWRh5Gsa3wUoGxC8-nwUY1VUcmf6fKw)
The log
function is a single arity "identity function" - returns the payload so you can modify in place.
mySingleArityFunction({foo, bar, baz}) // oh no i need to log bar and baz
mySingleArityFunction({foo, ...log({bar, baz})) // done!
mySingleArityFunction(log({foo, bar, baz})) // log them all why not
myBadMultiArityFunction(foo, bar, baz) // oh no i need to log bar
myBadMultiArityFunction(foo, log(bar), baz) // done!
myBadMultiArityFunction(...log({foo, bar, baz}).values()) // log them all why not
We default to single arity to encourage this in the JS ecosystem.
For production logging, Smol Logger's storage destination can be overwritten. We like Logflare!
// OPTIONAL: store the local file store for reuse
const oldStore = logger.store
// override the default local file store with your own remote filestore
// { logName: string, loggedLine: string | null, payload: any, secondsSinceStart: number, secondsSinceLastLog: number }
logger.store = ({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog }) => {
fetch("https://api.logflare.app/logs/json?source=YOUR_SOURCE_ID", {
method: "POST",
headers: {
"Content-Type": "application/json; charset=utf-8",
"X-API-KEY": "YOUR_API_KEY_HERE"
},
body: JSON.stringify({ message: logName, metadata: {loggedLine, payload, secondsSinceStart, secondsSinceLastLog }})
})
// OPTIONAL: run the local file store as well
oldStore({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog })
}
![image](https://private-user-images.githubusercontent.com/6764957/246623568-69b546ec-23d7-4099-9432-5fd37ca1436e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ3MDUzMDUsIm5iZiI6MTcyNDcwNTAwNSwicGF0aCI6Ii82NzY0OTU3LzI0NjYyMzU2OC02OWI1NDZlYy0yM2Q3LTQwOTktOTQzMi01ZmQzN2NhMTQzNmUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjZUMjA0MzI1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZGY5NzIwOTk0N2NkMTY4OThlZWE3YWEzNDMzNDZmNjkwNDgwNmI5ZTMyY2ZmYzQyMjg0NzFkZjU1MzJkMDUyMSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.fg0cXA_aQ7O8AAbLcSlTtJUmcC7cYGwgYVU_OcvtMIE)
If you expect high volumes of logs, you should batch them:
const logMessages = []
function throttle(func, delay = 1000) {
let timeout = null;
return function(...args) {
if (!timeout) {
timeout = setTimeout(() => {
func.call(this, ...args);
timeout = null;
}, delay);
}
};
}
const sendToLogFlare = ({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog }) => {
fetch("https://api.logflare.app/logs/json?source=YOUR_SOURCE_ID", {
method: "POST",
headers: {
"Content-Type": "application/json; charset=utf-8",
"X-API-KEY": "YOUR_API_KEY"
},
body: JSON.stringify({"batch": logMessages})
})
.then(() => logMessages = [])
}
log.store = throttle(sendToLogFlare, 1000)
Async/Blocking logging
Logging is synchronous by default in smol logger.
Note that in the above example we are firing off an async fetch inside of a synchronous call. If your application crashes there is a smol chance that the log may not complete sending since it is running asynchronously. If you need to block for an async call, you can overwrite the asyncLog
method:
// OPTIONAL: store the local file store for reuse
const oldStore = logger.store
// override the default local file store with your own remote filestore
// { logName: string, loggedLine: string | null, payload: any, secondsSinceStart: number, secondsSinceLastLog: number }
logger.store = async ({ logName, loggedLine, payload, secondsSinceStart, secondsSinceLastLog }) => {
const res = await fetch("https://api.logflare.app/logs/json?source=YOUR_SOURCE_ID", {
method: "POST",
headers: {
"Content-Type": "application/json; charset=utf-8",
"X-API-KEY": "YOUR_API_KEY_HERE"
},
body: JSON.stringify({ message: logName, metadata: {loggedLine, payload, secondsSinceStart, secondsSinceLastLog }})
}).then(res => res.json())
// just demonstrating that you can await inside this store
}
// now you can block execution in an async context
await logger.asyncLog('my message here', { foo: 'bar' })
Note: this funcitonality is new and not tested, please try out and give fixes/feedback
This logs BOTH input and output of a function that you want to monitor.
const wrapped = logger.intercept(openai.createChatCompletion.bind(openai)) // binding is impt bc of how OpenAI internally retrieves its config
const response = await wrapped({
model: process.env.OPENAI_MODEL || 'gpt-3.5-turbo',
messages: [/* etc */ ],
});
Sometimes the output can be very verbose (as is the case with OpenAI chatCompletion). so we also allow you to expose a simple "log transformer" that is an arbitrary function to modify the intercepted output to a format of your choice that you like:
// custom
const wrapped = logger.intercept(
openai.createChatCompletion.bind(openai), // binding is impt bc of how OpenAI internally retrieves its config
{
logTransformer: (args, result) => ({
// ...result, // optional - if you want the full raw result itself
prompt: args[0].messages,
response: result.data.choices[0].message,
})
}
)
const response = await wrapped({
model: process.env.OPENAI_MODEL || 'gpt-3.5-turbo',
messages: [/* etc */ ],
});
// now the logged response has highlighted the specific fields of our interest
![image](https://private-user-images.githubusercontent.com/6764957/246631281-0b15a508-db3b-4a52-8a8f-a94a7e0282a6.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ3MDUzMDUsIm5iZiI6MTcyNDcwNTAwNSwicGF0aCI6Ii82NzY0OTU3LzI0NjYzMTI4MS0wYjE1YTUwOC1kYjNiLTRhNTItOGE4Zi1hOTRhN2UwMjgyYTYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjZUMjA0MzI1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9Y2ExMDdjYjc1N2EwZWEzOWQ4YmM3M2FkMjFkNTQ4Mzk0MDUzNzQzN2E2MDUwMDdiNGRlNWQyMTI1NDA3MGY1MyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.UB_SpafC_FEw6w36jlv6_hlMYSd5OZWjXI-5PvbvhQw)
The logTransformer
function can be async in case you need it.
General rule for customizing is just overwrite any of the variables and methods exposed in the class:
logger.logDirectory = '.smol-logs' // change default file store directory
logger.logToConsole = false // turn off logging to terminal
logger.logToStore = false // turn off logging to store
logger.logName = (name: string) => `My custom logName: ${name}` // change Log naming!
logger.LOGCOLOR = (logName: string) => "\x1b[35m" + logName + "\x1b[0m"; // set log colors to magenta instead of yellow
Other log colors to try:
\x1b[31m: Red
\x1b[32m: Green
\x1b[33m: Yellow
\x1b[34m: Blue
\x1b[35m: Magenta
\x1b[36m: Cyan
\x1b[37m: White
Get creative! For example you can stack logname functions...
// for an advanced application, you can add log names in a stack
const logStack = [logger.logName] // store original logName fn at the bottom of the stack
logger.logName = (name) => logStack.reduce((prev, fn) => fn(prev), name)
let temp = 0
do {
logStack.unshift(name => ' ' + name)
// everything logged here is one indent in
log('logname1 here ' + temp, temp)
let temp2 = 0
do {
logStack.unshift(name => ' ' + name)
// everything logged here is two indent in
log('logname2 here ' + temp, temp)
logStack.shift()
} while (temp2++ < 5)
logStack.shift()
} while (temp++ < 5)
![image](https://private-user-images.githubusercontent.com/6764957/246627693-2f15bf5a-a25b-4b69-b7e5-38524beeef70.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ3MDUzMDUsIm5iZiI6MTcyNDcwNTAwNSwicGF0aCI6Ii82NzY0OTU3LzI0NjYyNzY5My0yZjE1YmY1YS1hMjViLTRiNjktYjdlNS0zODUyNGJlZWVmNzAucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjZUMjA0MzI1WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YmY3YWFmODBiMDkwMjZkOTVmMGZhYWU3MjVhODFmZTM5MmIyN2RkMTlmYTI2N2EwMGNmM2Y0ZDE3Mjc1ZWJlMSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.LifKtOZDe8ZMmNQn8Pnx8-A4302wvVfxFj1moqAZpqg)
this repo was bootstrapped with https://github.com/alexjoverm/typescript-library-starter (must modify with alexjoverm/typescript-library-starter#333)
Follow the console instructions to install semantic release and run it (answer NO to "Do you want a .travis.yml
file with semantic-release setup?").
Note: make sure you've setup repository.url
in your package.json
file
npm install -g semantic-release-cli
semantic-release-cli setup
# IMPORTANT!! Answer NO to "Do you want a `.travis.yml` file with semantic-release setup?" question. It is already prepared for you :P
From now on, you'll need to use npm run commit
, which is a convenient way to create conventional commits.
Automatic releases are possible thanks to semantic release, which publishes your code automatically on github and npm, plus generates automatically a changelog.
note from swyx - wasnt able to get the publishing to work, kept failing.
for now:
npm version patch
npm publish --access=public