bithavoc/express-winston

express.js middleware for winstonjs

express-winston's Issues

Minimal configuration in README

In my Express.js application I had to add these settings:

In package.json,

{
  "dependencies": {
    "winston": "0.6.x",
    "underscore": "1.5.x",
    "express-winston": "0.2.x"
  }
}

In server.js:

var winston = require('winston'),
    expressWinston = require('express-winston');

So minimal configuration for express-winston does not present in README, or there is something wrong in my configuration, so I had to add these requires into server.js.

Filter authorization header by default

Currently it is not straight forward to filter the authorization header. It would be nice if there was a flag or an easy way to do this.

Perhaps this should be on by default.

Here is what i am currently doing

function customRequestFilter(req, propName){
  if(propName === "headers"){
    return Object.keys(req.headers).reduce(function(filteredHeaders, key){
      if(key !== "authorization"){
        filteredHeaders[key] = req.headers[key]
      }
      return filteredHeaders
    }, {})
  } else {
    return req[propName]
  }
}

module.exports = expressWinston.logger({
    transports: transports,
    requestFilter: customRequestFilter
 })

Does this middleware support winston.transports.File?

Does this middleware support winston.transports.File?I want to record url accesses in my log file.

response header content-type might be undefined

https://github.com/bithavoc/express-winston/blob/master/index.js#L201

res._headers['content-type'].indexOf('json') >= 0 ? JSON.parse(chunk) : chunk

res._headers['content-type'] might be undefined. indexOf throws an error.
The code should be replaced with:

(res._headers['content-type'] || '').indexOf('json') >= 0 ? JSON.parse(chunk) : chunk

Or in order to catch content-types that contain json\jsonp but not lalalajsonlalala:

/(\W|^)jsonp?(\W|$)/i.test(res._headers['content-type'] || '') ? JSON.parse(chunk) : chunk

Also, consider changing res._headers[...] with res.get(...).
Furthermore, In my opinion there is no need to wrap the JSON.parse with try-catch because I don't see a reason to use a json content-type and not returning a json object in the body...

cpu usage increase when using _routeWhitelists

After a recent release we noticed that the cpu usage on our node servers was increasing until it reached 100%, whereas before it sat around 11%. We eventually tracked it down to this line:

req._routeWhitelists.req.push('request_id');

If we comment that line out, the cpu issues disappear.

winston should be a peerDependency

Handling Uncaught Exceptions and Meanjs

I've been programming in node for a couple of months now and just a few days I found out about winstong and express-winston
So looking into winston doc I see that there is a way to handle uncaught Exceptions, so my question is how does that work here?

Using with Express 4

Does express-winston work with Express 4? I seem to almost have it working but not fully. The README says "The logger needs to be added AFTER the express router(app.router))", but in Express 4, you don't explicitly use app.router anymore. So where does express-winston need to be in the sequence of middleware?

Basically, I'm trying to use winston to log requests. I've tried it both before and after my routes. When I place it before my routes, requests are logged and it seems to work fine, but I cannot filter by log level. Here's my logger:

  app.use(expressWinston.logger({
    transports: [
      new(winston.transports.Console)({
        level: 'error',
        colorize: true
      })
    ],
    statusLevel: true,
    meta: false,
    msg: "{{res.statusCode}}\t{{req.method}}\t{{res.responseTime}}ms\t{{req.url}}"
  }));

Even though I set level: 'error', all info messages still get logged. Am I missing something? Thanks.

Why an empty callback is sent to winston.log?

I just want to have a better understanding of this:

// This is fire and forget, we don't want logging to hold up the request so don't wait for the callback
options.winstonInstance.log('error', 'middlewareError', exceptionMeta, function () {
    // Nothing to do here
});

Publish on NPM

version 0.2.0 hasn't been published on NPM, the latest is 0.1.3

expressFormat option seems to ignore colorize: false

I'm using a winston logger in my app:

var logger = new winston.Logger({
  transports: [ new winston.transports.File({
          filename: 'test.log',
          level: 'debug',
          prettyPrint: true,
          colorize: false,
          timestamp: true,
          json: false
        })
  ]
});

In addition, I use a request logger with express-winston configured to use the same winston instance:

var requestLogger = expressWinston.logger({
  winstonInstance: logger,
  expressFormat: true
});

Despite colorize: false being set the requestLogger seems to produce ANSI escape sequences for colors:

info: ESC[90mPOST /clientsESC[39m ESC[33m400ESC[39m ESC[90m1123msESC[39m
{ req:
   { url: '/clients',
...

My understanding of the option (based on the comment in the code sample:
"Use the default Express/morgan request formatting, with the same colors. Enabling this will override any msg and colorStatus if true. Will only output colors on transports with colorize set to true") was that it should not output these escape sequences for a transport with the given config. Is this correct?

Expected response time range in test.js too arbitrary

test.js#L393's arbitrary range - between 120 and 130 ms - caused tests on #78 to fail (under iojs). When I re-ran the test, it passed.

TODO: de-arbitrary this range.

Upgrade winston dependency to 1.0.0

Winston < 1.0.0 uses the colors npm package in a way that makes it monkey patch String.prototype. As a change to String.prototype will be global to the whole runtime, usage of express-winston modifies the behaviour of strings for all other code.

Any way to include optional values in message?

I have some req that come in with session information, and would like to be able to display that. But the value doesn't always exist.

Specifically, when my users are logged in there is a session token I want to display: req.user.sessionID

But the value doesn't always exist...

How would I express that in the msg?

It'd be nice if different status codes caused log messages to be logged at different levels...

It'd be nice if different status codes caused log messages to be logged at different levels...

For example, if there's a 500 status, I'd like for it to be logged at the 'error' level, but if the status is 404, I only want 'warn', whereas for a 200 status, 'info' is sufficient.

This might be customizable as a mapping provided in the options to the logger method, i.e:

app.use(expressWinston.logger({
    transports: [ foo, bar ],
        statusLevels: {
            '200>399': 'info',
            '400>499': 'warn',
            '500>': 'error'
        }
     });

Make sense?

Routes Whitelist for logging as opposed to ignoredRoutes

Very stressful listing about 20 routes not to get logged, also routes like '/api/items?q=this' don't get picked up in the blacklist.

Refactor tests

They're a bit of a mess - it's actually my fault for splitting 'v0.1.0' and 'v0.2.0' apis into sections way back - so I'll clean it up when I can - just wanted to make note of it here. Now that there's coverage checking, this can finally be done without missing anything.

• The structure of the tests - the describes and its - should better follow the structure of the module with a describe for each exports-ed item;
• All the functionality should be explicitly tested for, as some is currently just 'coincidentally' tested;
• Most of the setup code can probably be refactored and better reused;

req passed to ignoreRoute does not contain proper information

https://github.com/bithavoc/express-winston/blob/master/index.js#L173

I was trying to ignore logging based on the res.statusCode. But at this moment, the status code is somehow 200, even though in the log message it is 404

Add Test Coverage

I'm pretty sure we're missing coverage on a bunch of code - I know for sure that some of the [white|black]list items are being skipped. Probably a good idea to add some type of coverage mechanism.

I use Mocha and Blanket for most of my projects, so I'm not sure what fits with vows, but I just wanted to put it out there.

Add raw error to errorLogger?

I'd like to put the raw error/exception object on the meta data that gets logged. I'd be happy to make a PR with the feature, but I feel like it can be done in a couple different ways.

Here's my problem:

Mongoose's ValidationError stores the specifics of each individual validation error within an errors property on the exception. However, errorLogger (using winston.exception.getAllInfo) only extracts the stack trace from the exception. This makes any error logs with a ValidationError difficult to work with since it's not possible to know what field is causing the error.

So, I can see two useful ways of adding the raw error/exception to the meta data:

1. Just add a rawError (or similar) field to the meta data that gets included on every logged error
2. Accept a function on the baseMeta property (or some other one?) that the user can provide to add custom fields that would take precedence over the default meta data fields. That gives a lot of flexibility (too much? right amount?), like this:

app.use(expressWinston.errorLogger({
  transports: [
    new winston.transports.Console({})
  ],
  <newMetaPropertyHandler>: function(err) {
    return {
      errors: err.errors,
      foo: err.foo,
      stack: err.mySpecialStackTrace // This will take precedence over the default `stack` property
    };
  }
}));

Like I said, I'd be happy to add a PR (with tests!), but I didn't want to go making changes when it seems like there are other valid options. So any feedback would be appreciated!

expressFormat logging formatting error

I am using this awesome logger in my express app. But when I start my express server and hit an endpoint, the debug info which is showing up in the console looks like this:

GET /test 200 7msGET /test 200 7ms
PUT /test 200 12ms
PUT /test 200 11msGET /test 200 8ms

See the error? It looks like every GET request log is not prepended with a newline character. I looked into the source code and tried to edit line 246 of express-winston/index.js

// before
var msg = chalk.grey(req.method + " " + req.url || req.url)

// after
var msg = chalk.grey("\n" + req.method + " " + req.url || req.url)

After that edit, my output looked like this:

GET /test 200 7ms
GET /test 200 7ms

PUT /test 200 12ms

PUT /test 200 11ms
GET /test 200 8ms

So yeah.. somewhere in the code the GET request is not getting a newline.

---

I am using version 0.3.1 of express-winston, and this is the configuration I am using for the logger:

expressWinston.logger({
        transports: [
          new winston.transports.Console({
            json: false,
            colorize: true,
            showLevel: false
          })
        ],
        expressFormat: true,
        meta: false

easy way to output log like nginx's access.log

IP address in msg format string

Is there a way to include the clients ip address in the msg format string?

npm install fails for 0.2.2

npm ERR! Error: shasum check failed for /home/aoviedo/tmp/npm-7212-5ofRw203/1392314613089-0.2782984636723995/tmp.tgz
npm ERR! Expected: d7c1f685e36dee954a6a02d38e8796524ec2b86f
npm ERR! Actual:   904e49131c69cfdcd13b70cd798094dad2b4ff04
npm ERR! From:     https://registry.npmjs.org/express-winston/-/express-winston-0.2.2.tgz
npm ERR!     at /usr/lib/node_modules/npm/node_modules/sha/index.js:38:8
npm ERR!     at ReadStream.<anonymous> (/usr/lib/node_modules/npm/node_modules/sha/index.js:85:7)
npm ERR!     at ReadStream.EventEmitter.emit (events.js:117:20)
npm ERR!     at _stream_readable.js:920:16
npm ERR!     at process._tickCallback (node.js:415:13)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/isaacs/npm/issues>

npm ERR! System Linux 3.11.0-15-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install"
npm ERR! cwd /home/aoviedo/Documents/iamat/nodeserver
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.3.24

"/?%2E" causes a server crash.

The following server code with request to http://localhost:3000/?%2E crashes the server:

var express = require('express'),
    expressWinston = require('express-winston');
    winstonMongoDB = require('winston-mongodb');

var app = express();

app.use(expressWinston.logger({
  transports: [
    new (winstonMongoDB.MongoDB)({
      level: 'error',
      db: 'test',
      collection: 'test'
    })
  ]
}));

app.listen(3000);

The exception thrown:

/node_modules/winston-mongodb/node_modules/mongodb/lib/mongodb/connection/base.js:242
        throw message;
              ^
Error: key . must not contain '.'
    at Error (<anonymous>)
    at Function.checkKey (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:1444:13)
    at serializeObject (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:356:14)
    at packElement (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:855:23)
    at serializeObject (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:360:15)
    at packElement (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:855:23)
    at serializeObject (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:360:15)
    at Function.serializeWithBufferAndIndex (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:332:10)
    at BSON.serializeWithBufferAndIndex (/node_modules/winston-mongodb/node_modules/mongodb/node_modules/bson/lib/bson/bson.js:1526:15)
    at InsertCommand.toBinary (/node_modules/winston-mongodb/node_modules/mongodb/lib/mongodb/commands/insert_command.js:146:37)

Throws exception in strict mode

Hi,

running node with --use-strict I get the following error.

./node_modules/express-winston/node_modules/winston/lib/winston/common.js:367
  fs.open(options.file, 'a+', 0644, function(err, fd) {
                              ^^^^
SyntaxError: Octal literals are not allowed in strict mode

This probably happens because of an outdated version of winston.

Could you please a look at this?

Thanks,

Format error message

It whould be nice to allow formatting the error message in the same way it is possible with the log message (at least with the req object).

ignoredRoutes Doesn't Work

I know the ignoredRoutes property is undocumented, but it doesn't seem to work as currently implemented. It uses req.route, which is apparently only available after requests are handled or in route-specific middleware (according to this Stack Overflow question).

Maybe testing req.url against a regex or an array of strings would work? I'd be happy to submit a PR.

How to use this with winston-config?

I would like to use express-winston, but I'm also using winston-config

Is it possible to use them together? to somehow get expressWinston.logger(... load from winston config json)

how to suppress req & resp in message?

I wanted to specify my own format, but I am always getting the req & resp data included. Any way for me to suppress that in the output?

Broken

Hi all,

This library looked great for express logging, we are already using winston for simultaneously logging to the console and to Loggly.

After putting this library in to push express logging through winston our app slowed to a crawl when running locally.

I had a look at the code of the library and it's a bit of a disaster to be honest.

1. The express chain of middlewares gets invoked once for each transport that was passed to the logger/error logger. This will cause extremely random behaviour and effectively breaks express. (Have a look at how next() gets called).
2. Even though next() is called multiple times, it isn't invoked initially until winston fires the logging call back to say the transport has logged the message, in our case as we're using Loggly this can take a while. So every web request waits on disk or network I/O for logging before being processed. This is a disaster.
3. You're using the async library because presumably you thought winston logged things synchronously? Well it doesn't, the whole point of it is to be async. There's no need to use the async library.

Sorry for the rant but I hope others can see this message before they decide to use this library. It's not worth a pull request so I'll just make a new one from scratch. Thanks.

Blocking certain routes

I have the kue admin interface mounted, but it spams my logs because they are constantly requesting the queue state. It would be nice to have a blacklist of routes to not log.

Expose winston instance?

Preface: This is my first time using winston/express-winston but Google did not seem to help with my issue.

I would like to use winston in my express project for both the HTTP request logging via the express-winston middleware but also use it as a general logger in other modules.

I believe to do that I would need to have the instance of the winston logger exposed via express-winston?
Unless I can have multiple instances of winston writing to the same log files and not cause issues.

request logging breaks streaming file uploads

If I use express-winston in my middleware chain to log requests it breaks streaming file uploads. I'm guessing it suffers from the same pausing request issue as passport: jaredhanson/passport#106

Response body is logging only the last char: "body":"}"

Hi, I am trying to log my response body (with the latest version 2.0.0).
I think I followed the documentation (which is awesome by the way!), and got the following code:

const winston = require('winston');
const expressWinston = require('express-winston');

expressWinston.requestWhitelist.push('body');
expressWinston.responseWhitelist.push('body');

module.exports = expressWinston.logger({
  transports: [
    new winston.transports.File({
      name: 'req-res',
      filename: '/var/log/nodejs/req-res.log',
      level: 'info',
    }),
  ],
  bodyBlacklist: ['password', 'pass', 'hash'],
});

The result is:
"res":{"statusCode":401,"body":"}"}

After review the source code I think I found a bug, the chunk
res.end = function(chunk, encoding) {
parameter will get only the last char.

The following code:

function logResponseBody(req, res, next) {

    var oldWrite = res.write,
        oldEnd = res.end;

    var chunks = [];

    res.write = function (chunk) {
        chunks.push(chunk);

        oldWrite.apply(res, arguments);
    };

    res.end = function (chunk) {
        if (chunk)
            chunks.push(chunk);

        var body = Buffer.concat(chunks).toString('utf8');
        console.log(req.path, body);

        oldEnd.apply(res, arguments);
    };

    next();
}

manage to log the entire response, maybe the code should also aggregate the entire chunks?
Do you want me to try fix the code and pull request it?

Thanks a lot,
Michael.

Node version dependency

You seem to be relying on node 0.6.x which mean I get:

npm WARN engine [email protected]: wanted: {"node":"0.6.x"} (current: {"node":"v0.8.11","npm":"1.1.62"})
[email protected] node_modules/express-winston

app.router is deprecated

How do you handle exceptions like in your example section without using app.router because it's deprecated since 4.x...

Thanks !

errorHandler has become errorLogger

Hi,

I used express-winston following the documentation of the main page of this project, and I have realized that the expressHandler function doesn' t exist, but taking a look into the code it was easy to see that the name of the function is errorLogger, so it is an stupid mistyped mistake, but it can save time for the future users.

Thanks for your endeavours in this module.

Why are we using `underscore` instead of `lodash`?

Srsly.

Dear @floatingLomas: look into swapping this.

"Implement a chain of requestFilters" - Uhhh, why?

I see that there's a feature request in the README to 'implement a chain of requestFilters'; do you remember what this was about?

I can't figure out a valid use case for it, so I can't really implement it. And I'm bored, so I wanted to implement it. :P

Push logs to loggly

Can we push logs to loggly using this module please?

request body missing

I haven't had luck in getting req.body to print out in the log.

Here's what I'm getting:

{
  "req": {
    "url": "/user/invite",
    "headers": {
      "host": "192.168.1.223:3000",
      "connection": "keep-alive",
      "content-length": "164",
      "cache-control": "no-cache",
      "origin": "chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop",
      "content-type": "application/json",
      "user-agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2202.3 Safari/537.36",
      "postman-token": "d3e5d8db-4a65-d10f-02e9-2fa2c4692838",
      "accept": "*/*",
      "dnt": "1",
      "accept-encoding": "gzip, deflate",
      "accept-language": "en-US,en;q=0.8"
    },
    "method": "POST",
    "httpVersion": "1.1",
    "originalUrl": "/user/invite",
    "query": {}
  },
  "res": {
    "statusCode": 200
  },
  "responseTime": 82,
  "level": "info",
  "message": "HTTP POST /user/invite { product: 'TouchCare' }",
  "timestamp": "2014-10-28T20:49:40.212Z"
}

status code colorized

hi,
is it possible to colorize the status code for the request middleware, as morgan/logger does?
thanks

Extend requestWhitelist and requestBlacklist with nested keys

Would be great if there is ability to set Whitelist/Blacklist with key like "user.email" to filter nested properties. Sometimes i want to filter nested objects but not only top level.

Template interpolation affects all uses of underscore

https://github.com/heapsource/express-winston/blob/4472ed235541b52582f48486a0f1863211d29555/index.js#L167

You might consider resetting the template interpolation setting back to what it was afterwards as this affects all instances of underscore templates in the current process.

(I do use underscore templating with the default settings in one of my projects)

Setting requestWhitelist doesn't seem to work

I can add to requestWhitelist like the docs suggest, but if I remove from requestWhitelist, it doesn't seem to work.

I've tried setting the white list like so:

expressWinston.requestWhitelist = [
  "url"
];

But the whitelist on line 216 of index.js still has the full requestWhitelist, so I'm still getting all of the data.

Option 'statusLevels' in errorLogger

For the Error Logging an option 'statusLevels' is described as

different HTTP status codes caused log messages to be logged at different levels (info/warn/error), the default is false

The Option is not working for me and I didn't see an implementation in the code of the errorLogger. Is there a plan to implement this? This would be a very nice feature to have and I'm looking forward to use it when implemented.

Many thanks.

if res.end is called with an incomplete JSON body, server crashes

See https://gist.github.com/zigg/06e891e55f22b2d0afebaaf6fc245408

The example may seem a bit contrived, but I've observed the behavior using express-winston with Loopback, possibly due to streaming behavior from that framework.

I'd be happy to make a PR to address this, but it might be simple enough to just fix.

Dependency on Winston 0.7.x

When I install express-winston (npm install express-winston), I see the package.json file that shows a dependency on winston 0.7.x rather than the newer ~0.9.0 dependency based on the following commit:

6fd3c89#diff-b9cfc7f2cdf78a7f4b91a753d10865a2

Is the public npm registry not up-to-date with the latest, or is the issue on my end?

responseWhitelist.push('body') crashes

I'm trying to log the body of the response and apparently the lib crashes when trying to get it from the headers.

Simple example app:

var express = require('express');
var expressWinston = require('express-winston');
var winston = require('winston');
var app = express();

expressWinston.responseWhitelist.push('body');
app.use(expressWinston.logger({
    transports: [
        new winston.transports.Console({})
    ]
}));

app.get('/', function (req, res) {
    res.send('Hello World!')
});

var server = app.listen(3000, function () {

    var host = server.address().address;
    var port = server.address().port;

    console.log('Example app listening at http://%s:%s', host, port)

});

Accessing the defined route results in a crash

> node app.js
Example app listening at http://0.0.0.0:3000
TypeError: Cannot call method 'indexOf' of undefined
    at ServerResponse.res.end (/home/arnau/code/myapp/node_modules/express-winston/index.js:201:64)
    at ServerResponse.res.send (/home/arnau/code/myapp/node_modules/express/lib/response.js:154:8)
    at app.listen.host (/home/arnau/code/myapp/app.js:14:9)
    at callbacks (/home/arnau/code/myapp/node_modules/express/lib/router/index.js:164:37)
    at param (/home/arnau/code/myapp/node_modules/express/lib/router/index.js:138:11)
    at pass (/home/arnau/code/myapp/node_modules/express/lib/router/index.js:145:5)
    at Router._dispatch (/home/arnau/code/myapp/node_modules/express/lib/router/index.js:173:5)
    at Object.router (/home/arnau/code/myapp/node_modules/express/lib/router/index.js:33:10)
    at next (/home/arnau/code/myapp/node_modules/express/node_modules/connect/lib/proto.js:193:15)
    at Object.handle (/home/arnau/code/myapp/node_modules/express-winston/index.js:231:9)