zertosh/v8-compile-cache

Require hook for automatic V8 compile cache persistence

v8-compile-cache

v8-compile-cache attaches a require hook to use V8's code cache to speed up instantiation time. The "code cache" is the work of parsing and compiling done by V8.

The ability to tap into V8 to produce/consume this cache was introduced in Node v5.7.0.

Usage

1. Add the dependency:

$ npm install --save v8-compile-cache

2. Then, in your entry module add:

require('v8-compile-cache');

Requiring v8-compile-cache in Node <5.7.0 is a noop – but you need at least Node 4.0.0 to support the ES2015 syntax used by v8-compile-cache.

Options

Set the environment variable DISABLE_V8_COMPILE_CACHE=1 to disable the cache.

Cache directory is defined by environment variable V8_COMPILE_CACHE_CACHE_DIR or defaults to <os.tmpdir()>/v8-compile-cache-<V8_VERSION>.

Internals

Cache files are suffixed .BLOB and .MAP corresponding to the entry module that required v8-compile-cache. The cache is entry module specific because it is faster to load the entire code cache into memory at once, than it is to read it from disk on a file-by-file basis.

Benchmarks

See https://github.com/zertosh/v8-compile-cache/tree/master/bench.

Load Times:

Module	Without Cache	With Cache
babel-core	218ms	185ms
yarn	153ms	113ms
yarn (bundled)	228ms	105ms

^ Includes the overhead of loading the cache itself.

Acknowledgements

• FileSystemBlobStore and NativeCompileCache are based on Atom's implementation of their v8 compile cache:
  • https://github.com/atom/atom/blob/b0d7a8a/src/file-system-blob-store.js
  • https://github.com/atom/atom/blob/b0d7a8a/src/native-compile-cache.js
• mkdirpSync is based on:
  • https://github.com/substack/node-mkdirp/blob/f2003bb/index.js#L55-L98

v8-compile-cache's Issues

use v8.cachedDataVersionTag

nodejs/node#11515

Unable to use dynamic import() in cjs-files

Trying to use a dynamic import() statement will fail with:

TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]: A dynamic import callback was not specified.

The error originates from an insufficient options object passed to the new Script()-constructor node VM docs

Cache directory should account for architecture

The compile cache may cause Node to crash if it was generated from a Node build where os.arch() does not match.

I experienced that recently with Node 16 on an Apple M1 machine. I was debugging an issue with yarn and eventually tracked the issue to v8-compile-cache.
yarn had been used with a non-native Node build (os.arch() reported x86), then when a native M1 build of the same node version was installed (os.arch() reported arm64) Node would crash when invoking yarn.

We determined that a compile cache had been generated, but because the compile cache matched independent of the architecture it was incompatible when used across Node binaries —both valid and running on the same machine without issue— and would cause Node to crash.

We were able to resolve the issue by:

• Invoking yarn with the environment variable as follows: DISABLE_V8_COMPILE_CACHE=1 yarn
• Finding and removing the generated temporary directories that look like v8-compile-cache*

I believe an appropriate solution would be to add os.arch() to the cache directories around here:

v8-compile-cache/v8-compile-cache.js

Lines 322 to 327 in 454af0b

	const version = typeof process.versions.v8 === 'string'
	? process.versions.v8
	: typeof process.versions.chakracore === 'string'
	? 'chakracore-' + process.versions.chakracore
	: 'node-' + process.version;
	const cacheDir = path.join(os.tmpdir(), dirname, version);

Feature request : Cache hit flag

I'm trying to add telemetry and logging around if the cache has been used. This flag will be very helpful in determining if there is an issue with the cache.

Feature request: callback for save() exception

save() currently just throw the exception if save were to fail. I'm wondering if we could have a callback hook so we could handle the exception by sending telemetry and add some logging.

process.mainModule has been deprecated since v14.0.0

process.mainModule has been deprecated since v14.0.0, while it is still being used here: https://github.com/zertosh/v8-compile-cache/blob/master/v8-compile-cache.js#L176, and here is the document of Node.js: https://nodejs.org/api/process.html#process_process_mainmodule

Write to commonly ignored location in .gitignore ?

First of all, thanks for this package and continued maintenance!

To avoid problems with v8-compile-cache-0 and similar generated directories not being ignored in .gitignore and projects accidentally committing v8-compile-cache-0 and similar to version control, would it make sense to write the cache directory to a commonly-ignored location such as node_modules?

Please add a "repository" Link to the package

It took some Google gymnastics to arrive at this repo, as there's no link from NPM due to the missing repository field in package.json.

Please consider adding a security policy

Hello, I believe I have found a security issue in v8-compile-cache. I do not see any GitHub security policy nor SECURITY.md files, and no mention in the README.md, so it is hard to know if you would prefer a private report or a public issue. Please consider populating these with what you'd like people to do with potential security problems.

Thanks

V8_COMPILE_CACHE_CACHE_DIR default doesn't match README

README says it defaults to <os.tmpdir>/v8-compile-cache-<v8.version>
whereas the code defaults to <os.tmpdir>/v8-compile-cache-<UID>/<v8.version>

Aside from that, I think it'd make more sense for the variable to only specify the base dir, and for <v8.version> to always be appended,
i.e. given V8_COMPILE_CACHE_CACHE_DIR=/my/v8cache
files would go in /my/v8cache/<v8.version>/*.BLOB etc.

decompile cache into javascript code

is it possible to decompile v8-cache into javascript code?

using of v8-compile-cache breaks source-maps functionality

The following code:
index.ts:

require('v8-compile-cache');
import { testFunction } from './testFunction';

function main(): void {
	try {
		testFunction();
	} catch (err: any) {
		console.error('Caught error:', err);
	}
}

main();

testFunction.ts:

export function testFunction() {
	throw new Error('TEST ERROR');
}

prints:

Caught error: Error: TEST ERROR
    at testFunction (Q:\source-maps-test\dist\testFunction.js:5:11)
    at null.main (Q:\source-maps-test\index.ts:6:15)
    at Object. (Q:\source-maps-test\index.ts:12:1)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47

with the commented index.ts' first line (//require('v8-compile-cache');) it prints:

Caught error: Q:\source-maps-test\testFunction.ts:2
        throw new Error('TEST ERROR');
              ^

Error: TEST ERROR
    at null.testFunction (Q:\source-maps-test\testFunction.ts:2:8)
    at null.main (Q:\source-maps-test\index.ts:6:15)
    at Object. (Q:\source-maps-test\index.ts:12:1)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47

So, the source-maps stack trace translation is broken after the first level. With turned on v8-compile-cache a javascript file is displayed instead of a typescript file.

I attach the whole test project here.
source-maps-test.zip

Buffer is undefined

https://nodejs.org/api/buffer.html

The Buffer class is a global within Node.js, making it unlikely that one would need to ever use require('buffer').Buffer.

Contrary to this, it needs "require" to use.

Install / uninstall programmatically?

Is there a way to use v8-compile-cache as a library?

My use-case is to programmatically install the compile cache, then uninstall it, and for other code in node to be able to require('v8-compile-cache') and for it to install correctly.

Right now, if I temporarily override the env var to prevent installation, I can get access to __TEST__ but it will also prevent subsequent code from calling require('v8-compile-cache') because it is in require() cache already. I'm also not sure __TEST__ can be relied on as a stable API surface.

Error: Package exports for 'colorette' do not define a valid

at require (v8-compile-cache/v8-compile-cache.js:161:20) {
code: 'MODULE_NOT_FOUND'

The module v8-compile-cache is present in the node_modules folder and reinstall the package also did not help.

This problem appeared on the server after installation PHP-V8.JS

If you use node_modules, which was created before v8 and which also contains v8-compile-cache, the project calmly executes npm run dev, and if you start npm i again or add some other package like: npm i bootstrap-vue-the installation goes fine, and compilation is interrupted with the same error.

Can you tell me if there is a way to fix this?

Using module.enableCompileCache() for the underlying implementation?

First of all, thanks for creating and maintaining this package!

We recently landed a similar built-in API in Node.js core (nodejs/node#54501) in an effort to support the ESM use cases and reduce CJS loader monkey-patching in the wild. The built-in compile cache has a few architectural differences in cache storage spelled out in nodejs/node#53639. I wonder if it's acceptable to migrate this package to use module.enableCompileCache() on newer Node.js versions (this can be done by just checking if require('module').enableCompileCache exists), which helps solving some of the open issues here & reduce CJS loader monkey-patching in general in the wild (there are some plans to no longer maintaining compatibility for monkey-patchers in the CJS loader, as that has been making refactoring and any improvements to the CJS loader very challenging).

There would be some open questions in how to solve the differences in the storage design, though, since module.enableCompileCache() enables caching for all modules loaded in the thread, and the cache can be shared by code loaded from any entrypoint. The cache file also corresponds one-to-one to source files, as that allows reading cache files on-demand & cheaper invalidation (skip immediately when the source hash at the beginning doesn't match), this actually seems faster than reading one giant blob per entrypoint in my testing in nodejs/node#47472.

Add programatic cache save

A very useful project, so widely used. With one flawn - cache is stored only on the graceful project exit.

I was trying to speed up the load times of my dev server, the one I always ctrl+c in order to close, and after hooking into the cache internals found that cache was just missing. It was never dumped on the disk.

I've added one export to the package, letting myself store cache when I want to save it (after server start), and was able to utilize the power of caching for the first time.

Feature request

Expose saveCache API for applications with never-ending or self-controleld life cycles.

TypeError: require.resolve.paths is not a function

Not sure if intentional...

https://nodejs.org/api/modules.html#modules_require_resolve_paths_request

This overrides default require resolve, but does not implement full api

      require.resolve = function(request, options) {
        return Module._resolveFilename(request, mod, false, options);
      };

Newer V8 code caching features

Very interesting project. I think you can benefit from some of the newer features we added to code caching in V8.

The current API compiles the top-level code and immediately puts that compilation result into the code cache. Inner functions are compiled lazily and not included in the cache. We recently introduced a new compile option that forces V8 to compile every function in the source so that we end up with more data in the code cache.

We also introduced a new API to produce the code cache not immediately after compilation, but at a later point when some execution might already have happened, at which point we may have compiled more functions.

I know none of this are yet available in Node.js since the newest V8 has not rolled into Node.js yet, and even then you have to expose these new API in a suitable way.

But I thought this might interest you in the long run.

question: can the package be used in esm?

Folder permission on multi-user systems

v8-compile-cache uses the OS temp directory (path.join(os.tmpdir(), 'v8-compile-cache', process.versions.v8)), but on non-windows systems, that temp directory is global.

On a system shared by multiple users, this creates a directory, affected by the most common umask, that is not writable by other users. However, just calling chmod 1777 on the folder is not enough of a workaround as other users need to be able to write on the v8 version specific subfolder, which is also global.

This affects yarn 0.22 users.

LICENSE file

Hello,

Could you please add a LICENSE file to this project? Thanks :)

Please support a CACHE_DIR env var

I would like to archive and then re-use the cache between ci builds. Currently, the cache location is only chosen via os.tmpdir().

I'd like to be able to override this behavior via env var.