How is compiling to memory an advantage exactly?

[timmolendijk]

How is compiling to memory an advantage exactly?

[korya]

It is an old issue, but I just wanted to share our experience with memory-fs. Probably it is true that memory limitation isn't so important. However in non-trivial apps with multiple entry points, using require.context and having a lot of various images it does become a very important aspect, and suddenly a webpack dev server process starts spanning gigabytes of RAM. This is without mentioning the plugins!

In our setup, webpack dev server process takes 2GB of RAM. After we configure a more aggressive policy for CommonsChunkPlugin, the RAM consumption goes down to 1.2GB, which is still insane. Removing typescript loader reduces it to 1GB.

May be we can apply some more optimization and reduce it even further. But it does not makes much sense to me. In our case the ideal solution for us would be to switch to a regular FS. This will solve the memory consumption problem and also reduce the time needed to start webpack dev server. By the way as webpack's user base keeps growing a lot of other folks are reporting same problems: webpack/webpack#2157 and webpack/webpack#1914.

Webpack is a great tool and has a huge potential. But it is wrong to make assumption on user's dev environment and to limit webpack's applications. I think any controversial decisions, such as using memory-fs, should be at least debatable and there is should be a possibility to opt it out. Is it possible to make memory-fs optional and to have a flag to disable its usage?

[jchv]

I had no part in this decision, but I agree with it, so here's my position. For me, this is the kind of thing that varies on use case. On one hand, using memory storage limits you a tad since you probably want to be able to fit the output into memory. On the other hand:

• The memory limitation isn't so important because e.g. a 100 MiB Webpack app probably wouldn't be very nice to use anyways.
• Using memory means that the storage is fully ephemeral, such that there's no clean-up needed, and no possibility of accidentally reusing stale files from a previous run.
• Memory is obviously faster to write to/serve from than disk, though the difference here is probably very minuscule. May help much more if you use a remote filesystem.
• Nothing to add to .gitignore or similar, since there's no disk output. Nothing to clear from the home folder if it crashes.

The fact that it doesn't write to a build folder by default is nice. I don't accidentally want a version of the app with hot reload enabled to get deployed. (I have other safe-guards around bad deploys like that, of course, but nonetheless.)

You may find that in your case it hardly helps and perhaps it even hurts you. I think it'd be reasonable to have an option to not use an in-memory filesystem and instead use your build folder; but it does introduce a couple of filesystem related complications that in-memory work skips.

[jeffshaver]

It would be nice to have both. It is super nice to have hot module reloading, but if I reload at all, I have to re-run webpack since to get going again and that is a little frustrating.

[timmolendijk]

Yeah I agree with you that compiling to memory is a fine solution. But since you usually want to compile to disk anyway (for production environments or for doing other things with the compiled bundle), it feels as the plainer, more straight-forward solution to just compile to disk in this scenario as well. (It saves a code path.)

Now, having clarified the point I was trying to make in my opening post, the nine days that have passed in the meantime have provided me with a relevant new insight: due to hot-reload script files (with their unique names) the build target directory turns into a big mess very quickly if you don't proactively clean-up (or never write to disk at all).

So yeah, I would conclude that this is the reason why compiling to memory is worth it. It's by far the most elegant solution for preventing chaos in your build target folder.