Giter Site home page Giter Site logo

fgm / filog Goto Github PK

View Code? Open in Web Editor NEW
18.0 6.0 6.0 5.05 MB

A fine logger package for Meteor.JS applications (client + server)

License: GNU General Public License v3.0

JavaScript 0.83% TypeScript 99.17%
logging logger syslog meteorjs meteor mongodb psr-3 log typescript

filog's Introduction

FiLog: a Meteor 1.9 logging package

Build Status CodeCov Test Coverage codebeat badge Known Vulnerabilities FOSSA Status

FiLog is a highly configurable isomorphic logger for Meteor applications, supporting configurable logging strategies and custom processing and log sending destinations.

Out of the box, it can log to the console (browser, server), the Meteor database, any Syslog server, and includes a "tee" logger to send to multiple destinations. On the browser, it can log to its own Meteor server over DDP, or to a centralized FiLog log aggregation server over HTTP.

FiLog is based upon low-level NPM packages::

Read the Documentation.

Configuration and usage

Main ideas:

  • A logged event is a (level, message, context) triplet, in which
    • level is an RFC5424 number (0..7)
    • message is a plain string or can be converted to one automatically
    • context is a plain Object
  • the package has no configuration of its own, it is entirely configured by its users, the client- and server-side Meteor parts of the application.
  • Both client and server applications may capture exception stack traces and log them for plain errors (window.onerror handling) and exception reporting. This is triggered by the logger.arm() method, and can be disabled by the logger.disarm() method.

Typical use case:

  • client/main.js:
    • instantiates any number of "processors": instances of classes derived from ProcessorBase. These are able to add information to a message on its way to the logging destination. Current concrete processors are:

      • BrowserProcessor: adds browser-related information, such as user agent, platform, operating system, client memory
      • MeteorUserProcessor: adds user-related information if any is available. Assumes the application is configured with some Meteor accounts package, like fgm:accounts-drupal.
      • RoutingProcessor: adds request-related information, such as the path being served.
    • instantiates any number of "senders": instances of classes derived from SenderBase. These are able to receive a (level, message, context) triplet ("event") and forward it somwhere else. Current concrete senders are:

      • ConsoleSender: output the message using console.log
      • MeteorClientHttpSender: HTTP POST the message to the Meteor server present at the URL defined by the Meteor-standard ROOT_URL environment variable.
      • MongodbSender: store the event in a collection in the Meteor MongoDB database instance (or the minimongo on the client).
      • NullSender: ignore the message.
      • SyslogSender: send the event to syslog, on server only: including it on client will fail.
      • TeeSender: send the event to all sender instances passed to its constructor as an array. Useful to send logs to multiple destinations.
    • instantiate a sending "strategy": instance of a class derived from StrategyBase. There are able to decide, based on an event, where it should be sent by available senders. They may also modify the logger instance at the last step of its construction, Two concrete strategies are currently available:

      • LeveledStrategy: based on the level of the message, it defines three severity levels: low, medium, and high, as well as the breakpoints between them in terms of RFC5424 levels, and associates a sender instance with each of these levels
      • TrivialStrategy uses a single sender for all messages. Especially useful for early work client-side, where you want everything to be stored to collect as much information as possible from a limited number of clients. May also be useful for tests, to simplify test setup.
    • constructs a ClientLogger instance, passing it the strategy instance, like:

        let logger = new ClientLogger(new LeveledStrategy(
          new NullSender(),
          new ConsoleSender(),
          new MeteorClientHttpSender(Meteor.absoluteUrl('logger'))
        ));
      
    • adds processor instances to the logger instance, like:

        logger.processors.push(
          new BrowserProcessor(),
          new RoutingProcessor(),
          new MeteorUserProcessor(Meteor)
        );
      
    • is now able to log events, like:

        logger.warn("Some warning condition", { foo: "bar" });
      
    • with this configuration:

      • logger applies processors to add browser, user, and routing information to the message context
      • since warnings are considered worthy of storage (default configuration of the LeveledStrategy constructor), the logger passes the now-rich message to the Meteor server thanks to the MeteorClientHttpSender sender
      • message arrives server-side.
  • server/main.js
    • is configured in much the same way as the client, so it has its own logger, typically configured with just a MongodbSender instance for all levels.

      let sender = new MongodbSender(Mongo, 'logger');
      let logger = new ServerLogger(
        new LeveledStrategy(sender, sender, sender),
        WebApp);
      logger.processors.push(new MeteorUserProcessor());
      
    • in the example above, it receives the POSTed event, and proceed to log it

      • because it has already been processed client-side, it is logged "raw", without applying additional processors to it
      • the MongoDb sender instance stores the event.
    • the server may also initiate a logging action, for example in a publish function or Meteor method. It can use the same API:

      logger.error("Some server condition", { baz: "quux" }
      
      • since the message is server-originated, the server processors will be applied, to add the Meteor user information to the event
      • the MongoDb sender instance stores the event.

Format note

  • Whatever the enabled processors, the Logger.log() method adds a millisecond-level timestamp to each context, under the timestamp.log key.
  • Senders designed to exit the system, like the MongoDB sender, or a syslog or logstash Beats forwarder, are expected to add another millisecond-level timestamp, under the timestamp.store key.

These two timestamps are here to alleviate any issue resulting from a clock difference between clients and servers.

Any sender can add extra keys to the context, under the timestamp main key, to enable timing diagnostics.

Running tests

The module contains tests. Some of them are unit tests and need nothing special to run, while others are currently implemented as integration tests and assume you have a working harness project using Filog, exposed on http://localhost:3000.

Start by compiling the package:

  • meteor yarn run ts-compile

Then you can run :

  • just unit tests with meteor yarn run test-unit
  • just integration tests with meteor yarn run test-integration
  • both tests with meteor yarn run test
  • both tests including coverage generation with meteor yarn run cover

To run integration tests, you need to run the provided test_harness project in one terminal, and the tests in another one. Alternatively, you could also roll your own bespoke test harness, which will need to have Filog configured.

Example server-side code in the test_harness/ directory

This file is needed to allow Filog to operate on the /logger URL: otherwise, Meteor will handle it natively and return a 200 with the default application page, failing the integration tests.

Terminal 1

cd (filog_dir)/test_harness
meteor yarn
meteor run --port 3100

This example uses port 3100 to avoid conflicting with existing applications on the default Meteor port (3000). To use another port, be sure to change it in __tests__/integration/harness.ts too.

Terminal 2

cd (filog_dir)
meteor yarn
meteor yarn run compile
meteor yarn run test
meteor yarn run cover

TIP Reading the .travis.yml file can be useful too.

License

FOSSA Status

filog's People

Contributors

arggh avatar davehenton avatar fgm avatar fossabot avatar lhamrit avatar yched avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

filog's Issues

Split context by source, identify source

Introduce a SourceProcessor modifying the context handling logic to format context introduced by client processors within a context.client property, (resp. server, cordova), and introduce a top-level source property identifying the origin of the logged message, so that client-sent messages can end up structured like:

{
  "_id" : "Q2656X7y3BbdEJnHp",
  "level" : 6,
  "message" : "some message",
  "context" : {
    "message_details" : {
      "arg1" : "answer",
      "arg2" : 42,
    },
    "source": "client",
    "client" : {
      "browser" : { /*..snip.. */ },
      "routing" :  { /*..snip.. */ },
      "customprocessor" : "other data"
    },
    "server" : {
      "hostname" : "fgm-linux-figaro",
      "user" : {
        "_id" : "wEbSie2FbizumzHyf",
        "createdAt" : ISODate("2018-06-29T16:03:18.583Z"),
        "services" : {
          "customauth" : { { /*..snip */ } ,
        }
      }
    },
    "timestamp" : {
      "log" : 1531235569008,
      "store" : 1531235569008
    },
  }
}

Make syslog output more parseable

Currently, the syslog messages being sent are just v8 inspection strings, as the result of util.inspect(doc).

See SyslogSender::send():

    this.syslog.log(level, util.inspect(doc, this.formatOptions));

The reason for this is to ensure data can actually be serialized, and a more common format like JSON does not ensure serializability.

The envisioned strategy is to replace this by an exception-handling JSON stringifying in SyslogSender::send():

  • If JSON.stringify succeeds, then log the resulting JSON
  • Else create a JSON containing:
    • timestamp, priority
    • a "serialization error" information
    • possibly a message string, if available
    • the util.inspect() blob, as currently done

Allternative would be to have a user-space serialization library. Some of the envisioned ones are, in more or less decreasing relevance order:

  • Serialijse: supports classes, circular references
  • (JBB)[https://github.com/wavesoft/jbb/wiki/Introduction-to-JBB]: efficient representation, supports object types, deduplicates information, might support circular references, but "JBB is quite slow on encoding" and produces a custom binary representation, which makes it a poor fit for a logging format which needs to be fast
  • Jahoo's Serialize Javascript (née express-state): handles functions, regexps, but not circular references
  • Seri: handles Date types and custom classes, but not circular references
  • JSON2: "There is no reason to use this file unless
    fate compels you to support IE8"
  • Jsmix: "This is a WIP project,"
  • JSON.stringify() with a custom replacer, as documented at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify is just the underlying logic for these packages

In the longer run, this could mean using the Strategy pattern just like we did at the top level of the logging chain: the initial suggested solution is just a degenerate case of it.

Support a method-based sender on clients

Currently, client-to-server logs go through a traditional HTTP route, to support logging to an external destination. However for many projects, using just a method call would simplify configuration.

Add a BeaconSender

The Beacon API allows sending data without waiting for the result, which is often what is expected from front-end logging, to minimize impact on user interactions.

Possible conflict with FlowRouter

I'm currently unavailable to debug this further, but I noticed a possible issue:

I was finishing up testing FiLog (1.6), and as a final step I tried logging every route change from FlowRouter like this:

const trackRouteEntry = (context) => {
  logger.debug(`Route entry ${context.path}`, context.queryParams);
};

FlowRouter.triggers.enter([trackRouteEntry]);

I had setup FiLog like this:

// /client/main.js
const consoleSender = new ConsoleSender();
const meteorSender = new MeteorClientHttpSender(Meteor.absoluteUrl('/logger'));
const teeSender = new TeeSender([meteorSender, consoleSender]);
logger = new ClientLogger(new LeveledStrategy(
  teeSender,
  teeSender,
  teeSender
));


logger.processors.push(
  new BrowserProcessor(),
  new RoutingProcessor()
);

logger.arm();

Now, if I check what happens to FlowRouter's state, it's Router.queryParamsis populated by data from logger, depending on the processors in use:

{"browser":{"platform":"MacIntel","product":"Gecko","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36","performance":{}},"routing":{"location":{"hash":"","search":"","pathname":"/","port":"3000","hostname":"localhost","host":"localhost:3000","protocol":"http:","origin":"http://localhost:3000","href":"http://localhost:3000/","ancestorOrigins":{}}},"timestamp":{"log":1479647232743}}

And the next time I try to call

FlowRouter.setQueryParams({
    randomParam: 5,
    anotherParam: true
});

all that stuff from FiLog ends up in my path / browser address bar.

Remove user resume tokens

User "resume" tokens in the UserProcessor have no significant use case, and can make the info significantly larger than needed. They should be filtered out.

Odd issue after updating to 0.1.9

After updating a working local fork to version from npm, I get this:

TypeError: Cannot set property '/Users/arggh/Development/myapp/node_modules/filog/package.json' of undefined

Basically, I did this (having installed a local fork of filog beforehand):

$ meteor npm uninstall --save filog
$ meteor npm install --save filog
$ meteor

Luckily, I had a brand new Macbook right next to me, so I cloned my app, installed filog fresh from npm, and I get the same error when running my app.

App source code included in logs

I'm wondering, is it intentional to include app source code in the log messages?

 {
    "_id": "eLnvfTgZhHDZtLWQ6",
    "level": 3,
    "message": "t.target.closest is not a function",
    "context": {
        "message_details": {
            "mode": "stack",
            "name": "TypeError",
            "message": "t.target.closest is not a function",
            "stack": [
                {
                    "url": "https://myapp.com/66c8e63c0bcd217ad90172f6c531842452b03a9e.js?meteor_js_resource=true",
"func": "b",
                    "args": [],
                    "line": 193,
                    "column": 240691,
                    "context": [
                        "",
                        "!function(){var e=Package.meteor.Meteor,t=Package.meteor.global,i=Package.meteor.meteorEnv,n=Package.mongo.Mongo,r=Package.modules.meteorInstall,o=Package[\"babel-runtime\"].meteorBabelHelpers,a=Package.promise.Promise,s=Package[\"ecmascript-runtime-client\"].Symbol,l=Package[\"ecmascript-runtime-client\"].Map,u=Package[\"ecmascript-runtime-client\"].Set,d,c,p,f,m,h,y,g,v,b,S,I,w,k,N,E,T,B,C,U,A,x,O,z,j,F,R,P,D,M,L,G,W,V,q,H,K,Y,$,J,Z,Q,X,ee,te,ie,ne,re,oe,ae,se,le,ue,de,ce,pe,fe,me,he,ye,ge,ve,be,Se,Ie,we,ke,Ne,Ee,Te,Be,Ce,Ue,Ae,_e,xe,Oe,ze,je,Fe,Re,P............

Logger.log() does not throw on invalid levels.

Although log() is documented as behaving as per the PSR-3 spec, it accepts any value for level, but the PSR-3 spec says it should throw an InvalidArgumentException (namespaced in the case of PHP, so isolated by the module system in ES6).

Sometimes log object itself is the message

I've no idea why, but sometimes I bump into logs where the message is the log document itself, like so:

{
    "_id": "PPP2whQ6gdC9Tfry2",
    "level": 3,
    "message": "{ level: 3,\n  context: \n   { message_details: { error: 'Updating subscription failed' },\n     browser: \n      { platform: 'Linux armv8l',\n        product: 'Gecko',\n        userAgent: 'Mozilla/5.0 (Linux; Android 8.0.0; SM-G950F Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Mobile Safari/537.36',\n        performance: {} },\n     routing: { location: [Object] },\n     meteor: { platform: 'client', user: [Object] },\n     timestamp: { log: 1530510424060 } } }",
    "context": {
        "message_details": {
            "error": "Updating subscription failed"
        },
        "browser": {
            "platform": "Linux armv8l",
            "product": "Gecko",
            "userAgent": "Mozilla/5.0 (Linux; Android 8.0.0; SM-G950F Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Mobile Safari/537.36",
            "performance": {}
        },
        "routing": {
            "location": {
...

Is this by design or perhaps a bug? FiLog version 0.1.17.

About rate limiting

I have ended up customizing my fork of FiLog to have the option to limit the amount of exceptions that are logged by logger.arm(), by giving it as an argument like so: logger.arm(5).

One scenario why I implemented that: sometimes a user has a really outdated browser that might throw 1000's of syntax errors in a very short amount of time. The easy fix for scenario was to create a counter to disarm logger after the specified limit is reached.

Would this be something you'd consider merging to FiLog?

Another way to approach this would be "rate limiting" on the server side much like DDP Rate Limiter does, which would be safer and could possibly be applied to all logs, if needed.

Currently anybody can "bombard" the logger endpoint and fill the logs with garbage.

Add syslog integration

Further integrations should probably go into separate packages, but syslog is arguably a "must have" in the base package.

Documentation improvements

It would be worth mentioning in the readme, that the http meteor package is a required dependency.

Also, I would emphasize the logger.arm() and logger.disarm() methods, their purpose and effect.

Question about need for "DevSender"

I have been using FiLog with a custom sender when in development mode, "DevSender", which will log to the console, but only produces one line per log event. This way my console doesn't get obfuscated by the 30-50 lines-per-entry log messages.

It's just a minor difference compared to the ConsoleSender, but I was wondering if it's useful or different enough to be added?

Include HTTP request's headers in the log entry

In the scenario where a log entry is added from the client via HTTP, it would be nice to include the request's headers in the logs, parsed or raw (easy). This way, if properly configured, the IP (amongst other things) would be included and available for logging.

How to use global.logger inside /imports folder

So I have

client/main.js

import { ClientLogger, TrivialStrategy, MeteorClientMethodSender } from "filog";

Meteor.startup(() => {
    const sender = new MeteorClientMethodSender();
    const strategy = new TrivialStrategy(sender);
    window.logger = new ClientLogger(strategy);
}

server/main.js


import { Meteor } from "meteor/meteor";
import { WebApp } from "meteor/webapp";
import { TrivialStrategy, ServerLogger, ConsoleSender } from "filog";

Meteor.startup(() => {
  const sender = new ConsoleSender();
  const strategy = new TrivialStrategy(sender);
  global.logger = new ServerLogger(strategy, WebApp);
});`

My question is how do I use logger inside /imports/api/methods.js file?

imports/api/methods.js

global.logger.error("error"); // global.logger is undefined here..

Add a "tee" sender

Currently, strategies can only send to one sender. Sometimes it is necessary to send the same info to two different destinations at the same time. Current use case: configure a syslog sender, but until ops is ready continue logging to mongodb.

Filog should export its base classes for applications to extend

Currently, Filog only exports its implementations of the 3 main class types: Processors, Strategies, and Senders. The base Logger class is already exported, though.

Because the underlying base classes are not exported, it is not obvious to app developers that they can, or better, should, develop their own bespoke implementations of these base interfaces, namely: ProcessorBase, StrategyBase, SenderBase.

Split README to new README and separate doc.

The 0.1.x README targets both devs considering using the logger and devs actually engaged in developing with it. And fails both, being too complex for the former and insufficient for the latter.

It should be targeted firmly to adoption / discovery, and delegate actual development-oriented information to separate documentat.

Errors when running with --production flag

Reproduction here:

https://github.com/arggh/filog-prod-flag-issue

When running Meteor with --production flag, log entries triggered from the client side cause this to appear in the server log:

/sandbox/filog/prod-flag-issue$ meteor --production
Warning: The --production flag should only be used to simulate production bundling for testing purposes. Use meteor build to create a bundle for production deployment. See:
https://guide.meteor.com/deployment.html
[[[[[ /sandbox/filog/prod-flag-issue ]]]]]

=> Started proxy.                             
For compatibility, the PACKAGE_DIRS environment variable is deprecated and will be removed in a future Meteor release.
Developers should now use METEOR_PACKAGE_DIRS and Windows projects should now use a semi-colon (;) to separate paths.
=> Started MongoDB.                           
I20161111-13:26:26.964(2)? Serving logger on /logger
=> Started your app.

=> App running at: http://localhost:3000/
I20161111-13:26:32.495(2)? [SyntaxError: Unexpected end of input]
I20161111-13:26:32.495(2)? [SyntaxError: Unexpected token e]
I20161111-13:26:32.496(2)? [SyntaxError: Unexpected token l]
I20161111-13:26:32.496(2)? [SyntaxError: Unexpected token c]
I20161111-13:26:32.496(2)? [SyntaxError: Unexpected token r]
I20161111-13:26:32.496(2)? [SyntaxError: Unexpected token r]
I20161111-13:26:32.506(2)? [SyntaxError: Unexpected end of input]
I20161111-13:26:32.506(2)? [SyntaxError: Unexpected token }]
I20161111-13:26:32.506(2)? [SyntaxError: Unexpected token c]
I20161111-13:27:02.616(2)? [SyntaxError: Unexpected end of input]
I20161111-13:27:02.616(2)? [SyntaxError: Unexpected token e]
I20161111-13:27:02.617(2)? [SyntaxError: Unexpected token r]
I20161111-13:27:02.617(2)? [SyntaxError: Unexpected token /]
I20161111-13:27:02.618(2)? [SyntaxError: Unexpected token &]
I20161111-13:27:02.624(2)? [SyntaxError: Unexpected end of input]
I20161111-13:27:02.624(2)? [SyntaxError: Unexpected token }]
I20161111-13:27:02.624(2)? [SyntaxError: Unexpected token c]

The first log entry, triggered manually with logger.warn, does appear in the MongoDB collection, but the second one, triggered by calling an undefined function, does not.

Without the --production flag it works as expected. I have not yet tried it in a "real" production environment. Logging from the server side seems to work with or without the flag.

Add correlation/session ID support

In a distributed environment, which is quite frequent in Meteor apps, it is difficult to track the full path of a request.

  • Having correlation IDs (aka "tags") allows tracking the flow of a single request across multiple loggers
  • Having session IDs (not necessarily identical to login sessions) allows relating multiple requests to a user context

How to start?

I try to follow these:

constructs a ClientLogger instance, passing it the strategy instance, like:

  let logger = new ClientLogger(new LeveledStrategy(
    new NullSender(),
    new ConsoleSender(),
    new MeteorClientHttpSender(Meteor.absoluteUrl('logger'))
  ));

adds processor instances to the logger instance, like:

  logger.processors.push(
    new BrowserProcessor(),
    new RoutingProcessor(),
    new MeteorUserProcessor(Meteor)
  );

is now able to log events, like:

  logger.warn("Some warning condition", { foo: "bar" });

steps

  1. meteor npm install filog --save
  2. meteor add http
  3. add up codes in client/main.js

and I can't find log anywhere...

Web server should preserve primitive value of boxing classes

Server.Logger.objectifyContext() should convert boxed values to their primitive unboxed versions reliably. For instance, String("foo") should be objectified as "foo".

Currently, though:

  • Boxed booleans and boxed numbers are objectified as empty objects
  • Boxed strings are objectified as arrays of characters, with complex rules:
    • runes in the BMP are returned as one character
    • runes outside the BMP (e.g. emoji) are split to two characters (high surrogate / low surrogate)

logger.arm() does not work on server

Calling logger.arm() on the server does not seem to work as described in README.

Both client and server applications may capture exception stack traces and log them for plain errors (window.onerror handling) and exception reporting. This is triggers by the logger.arm() method, and can be disabled by the logger.disarm() method.

Example

/server/main.js

import { MongodbSender, ServerLogger, LeveledStrategy, MeteorUserProcessor } from 'filog';

// Initialize FiLog logger
const sender = new MongodbSender(Mongo, 'logger');
logger = new ServerLogger(
  new LeveledStrategy(sender, sender, sender),
  WebApp);
logger.processors.push(new MeteorUserProcessor());
logger.arm();

/server/some/other/directory/file.js

callNonExistentFunction();

This does not produce a log entry in the MongoDB's collection logger, instead, it just logs to console. The same procedure on the client does produce a log entry.

Remove/replace Bithound.io

Bithound analysis was recently added to the project, but the company announced immediate shutdown today 2018-05-31, so it needs to be removed and hopefully replaced by an alternative.

[windows/error] meteor npm i filog --save

Running the command below in a windows 10 machine results in error:

meteor npm i filog --save

npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"!win32","arch":"any"} (current: {"os":"win32","arch":"ia32"})
npm ERR! notsup Valid OS:    !win32
npm ERR! notsup Valid Arch:  any
npm ERR! notsup Actual OS:   win32
npm ERR! notsup Actual Arch: ia32

Depending on Meteor causes issues when creating a package

If I want to encapsulate FiLogger to a reusable Atmosphere/Meteor package, I get this when I try to import anything:

W20161119-17:35:55.303(2)? (STDERR) ReferenceError: Meteor is not defined
W20161119-17:35:55.303(2)? (STDERR)     at Object.<anonymous> (/dev/packages/logger/.npm/package/node_modules/filog/lib/index.js:64:20)
W20161119-17:35:55.303(2)? (STDERR)     at Module._compile (module.js:409:26)
W20161119-17:35:55.304(2)? (STDERR)     at Object.Module._extensions..js (module.js:416:10)
W20161119-17:35:55.304(2)? (STDERR)     at Module.load (module.js:343:32)
W20161119-17:35:55.304(2)? (STDERR)     at Function.Module._load (module.js:300:12)
W20161119-17:35:55.305(2)? (STDERR)     at Module.require (module.js:353:17)
W20161119-17:35:55.305(2)? (STDERR)     at require (internal/module.js:12:17)
W20161119-17:35:55.306(2)? (STDERR)     at npmRequire (/dev/app/.meteor/local/build/programs/server/npm-require.js:133:10)
W20161119-17:35:55.306(2)? (STDERR)     at Module.Mp.useNode (packages/modules-runtime.js:493:20)
W20161119-17:35:55.306(2)? (STDERR)     at fileEvaluate (packages/modules-runtime.js:180:20)

There is a Meteor.isServer check in /lib/index.js that's causing this.

Provide TypeScript types

At some point, FiLog will probably be converted to TypeScript. In the meantime, providing a filog.d.ts will make it easier to use on TS projects.

Support per-sender processors

Currently, all senders receive the same messages, extended by the same processors.

However, there are cases where some senders would be better off without some processors, or possibly with alternate ones. A typical case in point would be the  BrowserProcessor on the ConsoleSender in the browser: this is almost always useless, and adds unwanted noise to the console output.

It would be nice to have a way to configure processors at the Sender level too, not just at the Logger level.

Provide valid documentation

Currently the code contains a number of JSdoc comments, but they are not validated by providing an actual buildable documentation. Add one.

After checking Docco, Yuidoc, Jsdoc, Docstrap and the meteor-jsdoc fork, the standard is going to be Jsdoc with Docstrap: meteor-jsdoc is really for apps, not NPM modules. Docco basically just splits the comments from inline to side-by-side with code, Yuidoc needs too much extra jsdoc-like markup, and Docstrap is just a set of plugins on top of jsdoc, the most interesting one being search.

Logger incorrectly stores structured data

This causes problems when said data contains dotted keys, which cannot be inserted in MongoDB, and actually reduces searchability because searches need to be aware of the exact path within the searched document.

Fix multi-level message_details storing

The move to a message_details context key in #37 introduced a bug in which, depending on configuration, the message_details key may appear at multiple nested levels instead of being reduced to a single top level as expected.

image

MongoDB Sender loses information in provided context

MongoDB sender seems to be losing information on the logged error on the way to the database.

How to reproduce

// /server/index.js
try {
   throw new Error('Oops!');
} catch (e) {
   logger.error('Oops!', { error: e });
}

If we have enabled a console logger, the output would be:

W20170108-15:30:08.526(2)? (STDERR) Error Error happened { error: [Error: Oops!],
W20170108-15:30:08.527(2)? (STDERR)   meteor: 
W20170108-15:30:08.527(2)? (STDERR)    { platform: 'server',
.
.
.

but in the MongoDB, the created document for this error would look like this:

{
    "_id" : "zfrwTgNoFnNtBZ6tr",
    "level" : 3,
    "message" : "Error happened",
    "context" : {
        "error" : {}, <--- missing!
        "meteor" : {

Missing stack traces and some questions

Missing stack trace

If I catch an error on the client and attempt to log it like so:

logger.error('Something wonky happened', { error: e });

Then, the entry in DB is missing the stack trace. The error object only consists of properties line, column, sourceURL.

If I include the stack trace in the context manually like so:

logger.error('Something wonky happened', {
   stack: e.stack,
   error: e
});

-> Stack trace is present in the DB entry.

Is this intentional, if so, what is the reasoning behind it?

Logging the error objects

What about logging errors like so:

try {
   something();
} catch (e) {
   // something something if condition
   ...
   // I've no clue what happened, better just log the whole error
   logger.error(e);
}

This produces an error log entry that's not helpful at all:

{
    "level": 3,
    "message": "[object Object]",
    "context": {
        // ...whatever processors you had

Is there a reason FiLog is not producing sensible output from this type of use?

Completely swallowing the error event

Should I try to log like this:

logger.error(error.message, error);

I get nothing logged, console or DB. It just silently does nothing. This is a bit dangerous, no?

Move the user-passed context to a messageDetail sub-key

Currently, the user-passed arguments are root properties in the context object ; this makes it to get to them and only them, separated from the keys added by the default processors.

To address this while maintaining the ability of processors to add on the full context object, including the value of these arguments, they will be moved wthin a messageDetails root property.

None of the default processors modify the arguments, but custom-written processors could do so, and will need to be updated for this new format.

Allow MeteorUserProcessor to tailor its processing

The MeteorUserProcessor can produce very verbose / redundant data, which needlessly increase storage needs, bandwidth, and workload on the logs processing platform (eg BELK stack). As an example, in most cases, the values contained below user.services.resume are irrelevant to logging needs.

The customization needs for this processor are very diverse, so the simplest technique would be to add a optional post-process callback.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.