Giter Site home page Giter Site logo

fast-safe-stringify's Introduction

fast-safe-stringify

Safe and fast serialization alternative to JSON.stringify.

Gracefully handles circular structures instead of throwing in most cases. It could return an error string if the circular object is too complex to analyze, e.g. in case there are proxies involved.

Provides a deterministic ("stable") version as well that will also gracefully handle circular structures. See the example below for further information.

Usage

The same as JSON.stringify.

stringify(value[, replacer[, space[, options]]])

const safeStringify = require('fast-safe-stringify')
const o = { a: 1 }
o.o = o

console.log(safeStringify(o))
// '{"a":1,"o":"[Circular]"}'
console.log(JSON.stringify(o))
// TypeError: Converting circular structure to JSON

function replacer(key, value) {
  console.log('Key:', JSON.stringify(key), 'Value:', JSON.stringify(value))
  // Remove the circular structure
  if (value === '[Circular]') {
    return
  }
  return value
}

// those are also defaults limits when no options object is passed into safeStringify
// configure it to lower the limit.
const options = {
  depthLimit: Number.MAX_SAFE_INTEGER,
  edgesLimit: Number.MAX_SAFE_INTEGER
};

const serialized = safeStringify(o, replacer, 2, options)
// Key: "" Value: {"a":1,"o":"[Circular]"}
// Key: "a" Value: 1
// Key: "o" Value: "[Circular]"
console.log(serialized)
// {
//  "a": 1
// }

Using the deterministic version also works the same:

const safeStringify = require('fast-safe-stringify')
const o = { b: 1, a: 0 }
o.o = o

console.log(safeStringify(o))
// '{"b":1,"a":0,"o":"[Circular]"}'
console.log(safeStringify.stableStringify(o))
// '{"a":0,"b":1,"o":"[Circular]"}'
console.log(JSON.stringify(o))
// TypeError: Converting circular structure to JSON

A faster and side-effect free implementation is available in the [safe-stable-stringify][] module. However it is still considered experimental due to a new and more complex implementation.

Replace strings constants

  • [Circular] - when same reference is found
  • [...] - when some limit from options object is reached

Differences to JSON.stringify

In general the behavior is identical to JSON.stringify. The replacer and space options are also available.

A few exceptions exist to JSON.stringify while using toJSON or replacer:

Regular safe stringify

  • Manipulating a circular structure of the passed in value in a toJSON or the replacer is not possible! It is possible for any other value and property.

  • In case a circular structure is detected and the replacer is used it will receive the string [Circular] as the argument instead of the circular object itself.

Deterministic ("stable") safe stringify

  • Manipulating the input object either in a toJSON or the replacer function will not have any effect on the output. The output entirely relies on the shape the input value had at the point passed to the stringify function!

  • In case a circular structure is detected and the replacer is used it will receive the string [Circular] as the argument instead of the circular object itself.

A side effect free variation without these limitations can be found as well (safe-stable-stringify). It is also faster than the current implementation. It is still considered experimental due to a new and more complex implementation.

Benchmarks

Although not JSON, the Node.js util.inspect method can be used for similar purposes (e.g. logging) and also handles circular references.

Here we compare fast-safe-stringify with some alternatives: (Lenovo T450s with a i7-5600U CPU using Node.js 8.9.4)

fast-safe-stringify:   simple object x 1,121,497 ops/sec ±0.75% (97 runs sampled)
fast-safe-stringify:   circular      x 560,126 ops/sec ±0.64% (96 runs sampled)
fast-safe-stringify:   deep          x 32,472 ops/sec ±0.57% (95 runs sampled)
fast-safe-stringify:   deep circular x 32,513 ops/sec ±0.80% (92 runs sampled)

util.inspect:          simple object x 272,837 ops/sec ±1.48% (90 runs sampled)
util.inspect:          circular      x 116,896 ops/sec ±1.19% (95 runs sampled)
util.inspect:          deep          x 19,382 ops/sec ±0.66% (92 runs sampled)
util.inspect:          deep circular x 18,717 ops/sec ±0.63% (96 runs sampled)

json-stringify-safe:   simple object x 233,621 ops/sec ±0.97% (94 runs sampled)
json-stringify-safe:   circular      x 110,409 ops/sec ±1.85% (95 runs sampled)
json-stringify-safe:   deep          x 8,705 ops/sec ±0.87% (96 runs sampled)
json-stringify-safe:   deep circular x 8,336 ops/sec ±2.20% (93 runs sampled)

For stable stringify comparisons, see the performance benchmarks in the safe-stable-stringify readme.

Protip

Whether fast-safe-stringify or alternatives are used: if the use case consists of deeply nested objects without circular references the following pattern will give best results. Shallow or one level nested objects on the other hand will slow down with it. It is entirely dependant on the use case.

const stringify = require('fast-safe-stringify')

function tryJSONStringify (obj) {
  try { return JSON.stringify(obj) } catch (_) {}
}

const serializedString = tryJSONStringify(deep) || stringify(deep)

Acknowledgements

Sponsored by nearForm

License

MIT

fast-safe-stringify's People

Contributors

bengourley avatar bridgear avatar corbinu avatar davidmarkclements avatar donutespresso avatar glesperance avatar kristofferostlund avatar mcollina avatar robinchow avatar saintedlama avatar simoneb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fast-safe-stringify's Issues

Support for Symbols

Similar to #41 we could add support for Symbols via toString and is-symbol.

> const safeStringify = require('fast-safe-stringify')
undefined
> safeStringify({ foo: 'bar', [Symbol.for('axe.silent')]: true });
'{"foo":"bar"}'
> safeStringify({ foo: 'bar', [Symbol.for('axe.silent').toString()]: true });
'{"foo":"bar","Symbol(axe.silent)":true}'

MaxStringLimit like in utils

Could you please make a feature for maxStringLength in order to trimm just the strings for a given length(like in utils), it would be a great addition to this module

`decirc` seems to fail for circular getter attributes

Hi there!

We've found an issue when calling the stringify function where there are circular references in a getter.

Example:

const fastSafeStringify = require('fast-safe-stringify')

const o = {
  a: 1,
  get self() {
    return o
  },
}

console.log(fastSafeStringify(o))

Which throws the following error:

/Users/kristofferostlund/dev/logger-node/examples/node_modules/fast-safe-stringify/index.js:11
  var res = JSON.stringify(obj, replacer, spacer)
                 ^

TypeError: Converting circular structure to JSON
    at JSON.stringify (<anonymous>)
    at stringify (/Users/kristofferostlund/dev/logger-node/examples/node_modules/fast-safe-stringify/index.js:11:18)
    at Object.<anonymous> (/Users/kristofferostlund/dev/logger-node/examples/circular-meta-example.js:57:13)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Function.Module.runMain (module.js:694:10)
    at startup (bootstrap_node.js:204:16)

Support for BigInt

Would be useful if this lib gracefully handled native bigint values.

Could simply toString() them

this.$__path is not a function error when using with Mongoose

Whenever I try to use fast-safe-stringify with a Mongoose Object, I get the error "this.$__path is not a function". I tried investigating exactly what is it in mongoose documents that cause this but couldn't find anything.

JSON.stringify() works with mongoose objects just fine.

I'm using the newest version of both packages:
fast-safe-stringify: 2.0.6,
mongoose: 8.6.8,
node: 10.16.1

Typescript definition misses undefined return type

The follow example returns undefined:

import safeStringify from "fast-safe-stringify";
safeStringify(undefined) // -> returns undefined

But the type definition only specifies string as return type, we should update it to string | undefined instead.

Copy of object

In usage, this has the potential to blow up an application with really hard to trace errors.

Because you are modifying the object itself, if that object has references to anything that needs to remain unmodified and this lib removes it from the object, you can potentially crash the application. For example, serializing the response from a server when the response object contains a reference to the server itself.

Spent a few days tracking this down in my own app.

Type module support

I have type: module in package.json

Now I am using

import safeStringify from 'fast-safe-stringify';

and

safeStringify.default(incident)

I would prefer to hide this .defaut in the library code so it should be built as esm and cjs version.

Handling Circular children with toJSON overidden

Hi there,

it seems that the library is still having issues handling circularity and objects with a toJSONmethod.

The latest version (1.1.11) does help a bit with this problem but it's still breaking when the children provides a toJSON method.

The following test provides more details.

Thanks.

const fss = require('fast-safe-stringify')
const assert = require('assert')

// Create a test object that has an overriden `toJSON` property
TestObject.prototype.toJSON = function () { return { special : 'case' } }
function TestObject (content) {}

// Creating a simple circular object structure
const parentObject = {}
parentObject.childObject = new TestObject()
parentObject.childObject.parentObject = parentObject

// Creating a simple circular object structure
const otherParentObject = new TestObject()
otherParentObject.otherChildObject = {}
otherParentObject.otherChildObject.otherParentObject = otherParentObject

// Making sure our original tests work
assert.deepEqual(parentObject, { childObject : { parentObject } })
assert.deepEqual(otherParentObject, { otherChildObject : { otherParentObject } })

// Should both be idempotent
assert.equal(fss(parentObject), '{"childObject":{"special":"case"}}')
assert.equal(fss(otherParentObject), '{"special":"case"}')

// Therefore the following assertion should be `true`
assert.deepEqual(parentObject, { childObject : { parentObject } }) // Still fails on 1.1.11
assert.deepEqual(otherParentObject, { otherChildObject : { otherParentObject } }) // Fails on < 1.1.11

global/shared arr object doesn't work well in parallel environment

Something that we've noticed is that the arr value is shared globally.

https://github.com/davidmarkclements/fast-safe-stringify/blob/master/index.js#L6

We run a bunch of low priority processes in parallel that all use safeStringify and have built in "pauses" to allow other high priority processes to use the main thread. This is leading to odd behavior since all the safeStringify's use the same arr object. There are a number of potential solutions to this that we could help to implement, but curious if you have any thoughts or preferred approaches?

The stringifier is not safe when it comes to throwing toJSON

One of our users has reported a somewhat weird issue - our app was crashing on a cross-origin frame that was contained somewhere in the object passed to the fast-safe-stringify.

After some debugging I've discovered that the issue is caused by the fact that even before calling the replacer function this window object throws on the internal obj.toJSON access. And the problem is... that we can't even easily supply a custom replacer to work around this because it won't see the value soon enough. We'd have to scan all the properties of the parent object, inspect their values and replace the window object there. This solution would drastically increase the amount of work needed to stringify all objects so I'm not super keen on doing this in our library.

The problem sort of can be showcased by this simplified version here:

JSON.stringify(
  {
    a: 1,
    b: {
      toJSON() {
        console.log('toJSON called');
        throw new Error("Oops");
      },
    },
  },
  (key, value) => {
    console.log({ key, value });
    return value;
  }
);

This results in this being logged:

{key: '', value: {…}}
{key: 'a', value: 1}
toJSON called

As we may see - the b key is never observed because toJSON is called first and it throws. In the original issue, this already throws on property access, I imagine that on logic like this 'toJSON' in obj.

And this simplified example showcases the problem quite accurately, even though it doesn't use this library. This is because at the end of the day... you are still using JSON.stringify under the hood:

if (replacerStack.length === 0) {
res = JSON.stringify(obj, replacer, spacer)
} else {
res = JSON.stringify(obj, replaceGetterValues(replacer), spacer)
}

I'm not sure if you should fix this... this would be quite nice for me and I could help in fixing this. But I'm not sure how to best approach this one - this would probably require attaching a fake toJSON to all objects (or even everything?), implementing proper try/catch in it, returning the original value so it could be handled by the replacer and then removing the fake toJSON from the mutated object 😬

I'm opening this issue to see if you have any better ideas on how this could be handled. And for posterity, since somebody might run into this too.

New Default Depth Limit Breaks Things

Just installed some dependencies that include a lenient semver string for this package. The depth limit made my app break in strange ways, wasn't really expecting [...] to start showing up in my REST calls. The depth limit is a neat feature, but probably shouldn't have a default value.

Infinite loop when called on a Sequelize object

In ladjs/superagent@2e5d6fd the Superagent library changed from using JSON.stringify to using fast-safe-stringify for it's default object serializer when sending JSON data.

We use Sequelize for our ORM. Sequelize uses their own custom object structure, which supports a .toJSON call.

When Superagent was using JSON.stringify, that called the .toJSON method on our Sequelize objects, properly converting them to JSON. However, with the change to fast-safe-stringify, .toJSON is no longer called, even if it exists.

I'm not sure how to provide a copy of the object in question since getting its JSON representation is unlikely to be helpful, since all the Sequelize specifics are stripped from that.

Is there a reason this library does not call .toJSON if it exists?

[ts] Cannot invoke an expression whose type lacks a call signature

Hello,
I got this error with tslint. The below code still work but I got the error warning with tslint.

[ts] Cannot invoke an expression whose type lacks a call signature. Type 'typeof import("../node_modules/fast-safe-stringify/index' has no compatible call signatures.

Here is my code

const stringify = require('fast-safe-stringify');
const info = { message: 'Message from John...' };
console.log(stringify(info));
  • fast-safe-stringify: v2.0.4
  • tslint: v5.10.0
  • Typescript: 2.9.2

Benchmarks

As far as I see it the current benchmarks are not ideal. It would be great to have something similar like the Node.js benchmarks where they are over and over and a significance is shown as well. Something similar is done when using https://www.npmjs.com/package/benchmark.

Should we switch to that?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.