Rebuilding a nodejs application fails if old code contained no package.json but new code contains it

[paketo-buildpacks]

Rebuilding a nodejs application fails if old code contained no package.json but new code contains it

[SaschaSchwarze0]

Hi @SaschaSchwarze0, thanks for bringing this to our attention. For now, it seems to be a problem in the node-engine buildpack's build logic. If this is blocking you as we continue to evaluate this issue, you should be able to get around it by running the same pack command with the addition of the --clear-cache option.

Thanks for checking and confirming. Workaround is known. But given we do not always know what we rebuild, we at the moment turn caching off always - not necessarily efficient. ;-) So looking forward for a fix - also to learn how it is done to get a better understanding of that area. :-)

[thitch97]

Hi @SaschaSchwarze0, thanks for bringing this to our attention. For now, it seems to be a problem in the node-engine buildpack's build logic. If this is blocking you as we continue to evaluate this issue, you should be able to get around it by running the same pack command with the addition of the --clear-cacheoption.

[ryanmoran]

After taking a look at this today, I've isolated the issue. When building without a package.json, the order group that is applied is the following:

paketo-buildpacks/node-engine
paketo-buildpacks/node-start

In this case, the node-start buildpack is what is requiring node into the build context. However, it does this by only specifying launch = true in its build plan requirement. The node-engine buildpack thus installs node, but only marks the launch = true flag which results in a situation where the restorer doesn't repopulate the contents of the layer on subsequent builds. This is expected and aligns with the buildpack specification. For applications that never introduce a package.json file into their codebase, this is fine because the node layer is never needed until run-time.

However, when switching to using a package.json, we suddenly have a build-time requirement for the node layer that did not previously exist. In this situation, the node-engine buildpack reuses the cache, which won't include the contents of the layer (as described above), and therefore results in npm or really any of the node layer executables being reported as missing when invoked during the remainder of the build process.

The solution here is that the node-engine buildpack needs to be a bit smarter when it is choosing to reuse the cached layer. If the current build plan requires node during build, we need to check to see if the layer contents are restored and then only reuse the layer if it exists.

[c0d1ngm0nk3y]

I can still reproduce this with the latest builder using

[detector] 5 of 11 buildpacks participating
[detector] paketo-buildpacks/ca-certificates 3.5.1
[detector] paketo-buildpacks/node-engine     1.2.1
[detector] paketo-buildpacks/npm-install     1.0.3
[detector] paketo-buildpacks/node-start      1.0.3
[detector] paketo-buildpacks/npm-start       1.0.3