Perfomance issue

[middyjs]

Perfomance issue

[lmammino]

Also, @vladgolubev, I really like the idea of having scoped package as babel. we might have the following approach:

const middy = require('@middy/middy') // or '@middy/core' or simply 'middy'
const doNotWaitForEmptyEventLoop = require('@middy/doNotWaitForEmptyEventLoop')

[DavidWells]

I was thinking about this the other day as well.

We want to include only the code required for a given lambda to run.

We can optimize in two places

1. at runtime like @NPCRUS suggests
2. the actual project size in node_modules. The size of zip effects perf as well, so even if you only include 1 middleware, the s3 zip still needs to load and keeping things lean is a "best practice" as it were

You can solve this in userland with webpack/rollup but I think we can also help users not shoot themselves in the foot forcing this with middleware published as separate packages

We could have middleware published as separate packages with a tool like https://lernajs.io/ (or the new packaging tool the lerna creator is close to finishing)

So a user would install the lean core and only the middleware they need.

npm install middy # lean core
npm install middy-validator 
npm install middy-json-body-parser
etc 

This would keep the zip as tiny as possible.

The repo can stay a monorepo, it's just a change to how the middleware would get published to npm via lerna

[vladholubiev]

Agree with @DavidWells 👍

This is a perfect case for setup much like Babel has. One monorepo and complementary packages under @babel npm org scope.

[NPCRUS]

I've done some research.

Looks like validator(especially ajv library) middleware loads about 140ms, which may be deadly amount of time for some services. And anyway since amount of prebuild middlewares is growing i think that it's not a good a good approach to load all of them. What I think will be possible to do here is get middlewares in such way

const validator = require('middy/middlewares').validator()

This will save execution time by not loading any other middlewares and still pretty handy and comfortable way to get middleware.

What do you think?
@lmammino

[lmammino]

Very good points from @NPCRUS and @DavidWells. The current architecture (and examples) are definitely helpful in terms of performance when using the bundled middlewares.

I like the options of using a mono-repo approach and separate the middlewares from the main core, so this might be something to consider for the near future, maybe targeting the first stable 1.0.0 release.

At the moment there's a very easy fix anyway, you can import middlewares directly as follows:

// imports only one middleware (doNotWaitForEmptyEventLoop)
const doNotWaitForEmptyEventLoop = require('middy/src/middlewares/doNotWaitForEmptyEventLoop')

This should do, until we figure out how to switch to a mono-repo approach.

Comments/thoughts?

[lmammino]

Hello @kevinrambaud 😉
For the moment I would suggest to use the approach mentioned above to avoid to load all the middlewares (and their dependencies) at once:

const doNotWaitForEmptyEventLoop = require('middy/src/middlewares/doNotWaitForEmptyEventLoop')

Moving towards a more stable 1.0.0 (yet to be planned), we will probably move towards a mono-repo approach where no middleware is bundled with the core and you have to install/require each and every one of them separately (similar to express or fastify).

[karlbateman]

@lmammino Will do 👍

[kevinrambaud]

Hey @lmammino any update about the approach you guys would prefer ?

[kevinrambaud]

if you need any help, I would highly be interested to contribute 😁

[lmammino]

Definitely, all help is appreciated. I'll try to lay out a roadmap for 1.0.0, meanwhile, being an open source project, every area you feel needs an improvement, is probably something that deserves a PR or a discussion, so feel free to make your own suggestions 😉

[karlbateman]

It'd be prudent to update your documentation to include this workaround.

[lmammino]

Hello @karlbateman, would you like to submit a PR for that?

[lmammino]

Awesome, i look forward to seeing your PR :)

[willfarrell]

Closing, v1-beta is now out with mono-repo support.