fastparallel's Introduction

fastparallel

Zero-overhead parallel function call for node.js. Also supports each and map!

Benchmark for doing 3 calls setImmediate 1 million times:

benchSetImmediate*1000000: 1378.514ms
benchAsyncParallel*1000000: 1740.304ms
benchAsyncEach*1000000: 1566.517ms
benchAsyncMap*1000000: 1687.518ms
benchNeoParallel*1000000: 1388.223ms
benchNeoEach*1000000: 1473.006ms
benchNeoMap*1000000: 1402.986ms
benchInsyncParallel*1000000: 1957.863ms
benchInsyncEach*1000000: 1383.822ms
benchInsyncMap*1000000: 1822.954ms
benchItemsParallel*1000000: 1690.118ms
benchParallelize*1000000: 1570.064ms
benchFastParallel*1000000: 1536.692ms
benchFastParallelNoResults*1000000: 1363.145ms
benchFastParallelEachResults*1000000: 1508.134ms
benchFastParallelEach*1000000: 1325.314ms

Obtained on node 12.18.2, on a dedicated server.

If you need zero-overhead series function call, check out fastseries. If you need a fast work queue check out fastq. If you need to run fast waterfall calls, use fastfall.

The major difference between version 1.x.x and 2.x.x is the order of results, this is now ready to replace async in every case.

Example for parallel call

var parallel = require('fastparallel')({
  // this is a function that will be called
  // when a parallel completes
  released: completed,

  // if you want the results, then here you are
  results: true
})

parallel(
  {}, // what will be this in the functions
  [something, something, something], // functions to call
  42, // the first argument of the functions
  done // the function to be called when the parallel ends
)

function something (arg, cb) {
  setImmediate(cb, null, 'myresult')
}

function done (err, results) {
  console.log('parallel completed, results:', results)
}

function completed () {
  console.log('parallel completed!')
}

Example for each and map calls

var parallel = require('fastparallel')({
  // this is a function that will be called
  // when a parallel completes
  released: completed,

  // if you want the results, then here you are
  // passing false disables map
  results: true
})

parallel(
  {}, // what will be this in the functions
  something, // functions to call
  [1, 2, 3], // the first argument of the functions
  done // the function to be called when the parallel ends
)

function something (arg, cb) {
  setImmediate(cb, null, 'myresult')
}

function done (err, results) {
  console.log('parallel completed, results:', results)
}

function completed () {
  console.log('parallel completed!')
}

Caveats

The done function will be called only once, even if more than one error happen.

This library works by caching the latest used function, so that running a new parallel does not cause any memory allocations.

Why it is so fast?

This library is caching functions a lot.
V8 optimizations: thanks to caching, the functions can be optimized by V8 (if they are optimizable, and I took great care of making them so).
Don't use arrays if you just need a queue. A linked list implemented via processes is much faster if you don't need to access elements in between.
Accept passing a this for the functions. Thanks to this hack, you can extract your functions, and place them in a outer level where they are not created at every execution.

License

ISC

fastparallel's People

Contributors

Stargazers

Watchers

fastparallel's Issues

question: how come it's so fast?

Hey there, I didn't really look that closely at the code to compare it with other libraries out there. Can you perhaps add a tiny info into the README as to the reason to its fastness? 👍

New Release

Latest version available on npm is 4 years old. There have been some changes and updated deps in last years, why don't release a new package? @mcollina

Update the benchmarks README

Hey @mcollina sorry for bothering you again, I just made a tiny change to parallelize that had a big impact in performance (using sliced instead of Array.prototype.slice.

Can you please install [email protected], rerun the benchmark and update the README please?

Remove xtend use Object.assign

As mentioned here xtend is outdated in performance

Done callback function isn't called

The _count value isn't being reduced in some functions https://github.com/mcollina/fastparallel/blob/master/parallel.js#L102
So it does not end up calling the done/final callback.