An orb for working with Node.js on CircleCI
Home Page: https://circleci.com/orbs/registry/orb/circleci/node
License: MIT License
An orb for working with Node.js on CircleCI.
For full usage guidelines, see the orb registry listing.
We welcome issues to and pull requests against this repository!
For further questions/comments about this or other orbs, visit CircleCI's orbs discussion forum.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Include the Shellcheck check job in the build process
3.0.1
Symmetrically to setup parameter in test job it would be handy to have teardown parameter to be able to notify about job result or publish test artifacts.
👋 I was updating a project to run integration tests with Cypress. Cypress requires browsers installed in order to run the tests, and I understand neither versions v2, v3, and v4 of this orb will help me until support for browser image variants is provided for the "newer" docker images of nodejs.
So I stuck to v1 and all was good so far. Eventually, I ran into a caching issue because cypress since v3 and up is installing itself in the .cache folder, rather than in the node_modules folder as any other node package (see doc link). This orb, and I think every version of it, doesn't allow me to specify multiple cache paths. It defaults to .npm and does allow to override it, but not to specify more than one.
Is passing a list value possible? and, would you consider that change?
The alternatives I have today are:
npm ci command.save_cache step passing the .cache folder key. Would look odd since I'm running those inside a "with-cache" named step that is supposed to take care of that for me.3.0.1
Error on node/install-packages
---
Unable to find your package.json file. Did you forget to set the app-dir parameter?
---
Current directory: /home/circleci/project
I have working_directory set in my .circleci/config.yml and I feel that app-dir should default to that.
Thoughs? I would be willing to try my hand at a PR if this is even possible. Not sure CircleCI exposes the value of working_directory to orbs.
Support for Yarn v2 (a.k.a., berry) was recently introduced to the install-packages command. Yarn v2 introduces a great new feature called zero installs that allows dependencies to be cached in version control. As this is the recommended manner of using yarn, it would be ideal if the orb supported it. The presence of .yarn/cache/ and/or .pnp.cj before yarn install is run implies use of zero installs.
yarn install should be passed --immutable-cache in addition to --immutable since the cache is tracked in version control. This will correctly start failing builds in which the package.json does not match the contents of .yarn/cache/ or .pnp.cj.save_cache since this will reduce performance in return for nothing.yarn install should be passed --check-cache, but only if a cache keyed off yarn.lock misses.--check-cache on forked builds (e.g., by inspecting CIRCLE_PR_REPONAME).cache-path parameter based on the outcome of #61.For others using zero installs, I suggest replacing the install-packages command with the following until zero installs is supported:
- restore_cache:
name: Restore cached lock file.
key: v1-yarn-lock-file-{{ checksum "yarn.lock" }}
- run:
name: Install dependencies, refetching if necessary for added security.
command: |
if diff --brief yarn.lock ~/.cache/yarn.lock 2> /dev/null
then
yarn install --immutable --immutable-cache
else
# See https://yarnpkg.com/features/zero-installs#does-it-have-security-implications
yarn install --immutable --immutable-cache --check-cache
mkdir --parents ~/.cache
ln yarn.lock ~/.cache/yarn.lock
fi
- save_cache:
name: Cache lock file.
key: v1-yarn-lock-file-{{ checksum "yarn.lock" }}
paths:
- ~/.cache/yarn.lock
If your yarn.lock is big, or you are extremely concerned with performance, you could replace the diff with a test for the presence of an empty file that you cache instead of yarn.lock. If you are unconcerned with the possibility of the cache being tampered with, then you could alternatively pass an override-ci-command of yarn install --immutable --immutable-cache and with-cache of false.
4.1.1
This configuration fails:
- node/install:
install-yarn: true
install-npm: false
node-version: 14.15.2
Because we defined a "USER" in our image, and that sudo is not available.
yarn install with the current user, without trying to sudo.
Due to this line:
https://github.com/CircleCI-Public/node-orb/blob/master/src/scripts/install-yarn.sh#L1
4.0.0
- node/install:
node-version: 12.18.1
Workaround:
I update the configuration to:
- node/install:
node-version: 12.18.1
install-npm: false
This works fine, since npm is already installed w/ node. However, I'd expect the defaults to work out of the box.
NPM install fails.
Latest version of NPM is 6.14.7
Something went wrong; the specified version of NPM could not be installed
Exited with code exit status 1
NPM install works.
Bump up the orb used for deployment to the latest orb-tools-orb version.
https://github.com/CircleCI-Public/orb-tools-orb/blob/master/MIGRATION.md
It would be nice to be able to cache based on .pnpm-lock.yaml when using pnpm as a package manager in the install-packages command.
Granted, this might amount to "npm support."
jobs:
test:
- run:
name: install pnpm
command: sudo npm i -g pnpm
- checkout
- node/install-packages:
override-ci-command: pnpm i
cache-path: ~/.pnpm-storeWe encountered a bug with machine images resulting in new Node versions not being set while using nvm. This was addressed in #73, but we had to remove the machine image tests as the nvm version was for whatever reason incompatible with the commands we're running. These will need to be re-added once the next tag is released with the updated nvm command.
In our pipeline we are installing packages as the first job in the pipeline and then restoring the cache in future jobs.
- node/install-packages
However, this does a restore cache/install/save cache
I think we only need to restore cache in our future jobs and do not need to run npm install again.
Given the cache key is generated automatically, would you advise against manually restoring the cache?
node-deps-v1-{{ branch }}-{{ checksum 'package-lock.json' }}
We would save 30s on our pipeline but not sure if that'd cause any issues.
circleci/[email protected]
When setting the executor and providing a name + tag as suggested in the docs: https://circleci.com/orbs/registry/orb/circleci/node#usage-run_tests_with_npm
jobs:
test:
executor:
name: node/default
tag: '13'it doesn't lint (in VSCode):
That is lints nicely (the code works as expected).
Other parts have nice intellisense such as when you hover over executor
1.1.6
Attempting to run node/install on a MacOS executor is failing due to missing gpg and sha256sum commands. Here is the relevant section of the CircleCI config:
build-ios:
macos:
xcode: '11.1.0'
working_directory: ~/repo
steps:
- checkout
- node/install:
node-version: '10.16.3'And the output from running the job:
Selected version of Node.js is 10.16.3
A different version of Node.js is installed (v12.10.0); removing it
Importing Node.js trusted release keys...
/bin/bash: line 155: gpg: command not found
/bin/bash: line 164: sha256sum: command not found
grep: SHASUMS256.txt: No such file or directory
The node installation should complete successfully.
The default install-packages command with cache-only-lockfile true tries to restore the cache path that looks like (simplified version)
node-deps-{{ arch }}-<<parameters.cache-version>>-{{ .Branch }}-{{ checksum "/tmp/node-project-lockfile" }}
but saves a cache with the key:
node-deps-{{ arch }}-<<parameters.cache-version>>-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-{{ checksum "/tmp/node-project-lockfile" }}
which means that restore always falls back to the version that restores without caring about the lockfile.
The save_cache cache key should match the most specific restore_cache key attempt.
install-packages should allow configuration to ignore the package.json file when constructing the cache string. Considering the package.json checksum for cache invalidation will introduce many unnecessary cache invalidations when a lock file is also present.
** I can't speak to package-lock.json, but my projects use yarn, so I can say at least this is applicable to yarn.lock.
yarn.lock file is a good metric for determining whether the dependency graph has changed and can be used alone irrespective of the package.json checksum. When it exists, it can stand alone.package.json that has nothing to do with dependencies. If a developer changes any number of things including "description", "scripts", or 3rd-party configs (e.g. eslint, prettier, etc configs within package.json), cache will be invalidated. This adds unnecessary delay (and cost) to a CI job execution.I understand that this could be a breaking change if not added as an optional parameter, so I'm fine with that. As-is, I cannot use this orb job (though I'd like to), because the unnecessary cache invalidations result in unnecessary costs via longer job times.
2.0.3
Version check fails on windows
Script should work on all platforms including windows
This line fails on windows when trying to produce a cross-platform matrix script.
Here's my config (adapted from one of the CircleCI blogposts):
version: 2.1
executors:
windows: win/default
node: node/default
orbs:
node: circleci/[email protected]
win: circleci/[email protected]
jobs:
test:
parameters:
os:
type: executor
node-version:
type: string
executor: << parameters.os >>
steps:
- checkout
- node/install:
node-version: '<< parameters.node-version >>'
install-yarn: false
install-npm: false
- run: npm install
- run: npm test
workflows:
all-tests:
jobs:
- test:
matrix:
parameters:
os: [node, windows]
node-version: ['0.10.48', '0.12.18', '6.17.1', '8.17.0', '10.20.1', 'current', 'latest']
Error messages:
Missing '(' after 'if' in if statement.
At line:13 char:7
+ if [[ latest == "" || latest == "latest" || latest == "current" ]]; ...
+ ~
Missing type name after '['.
At line:13 char:22
+ if [[ latest == "" || latest == "latest" || latest == "current" ]]; ...
+ ~~
I'm not sure if I'm doing this wrong or if I have unrealistic expectations! Let me know!
4.1.0
Trying to install Node lts on the Circle machine
- node/install:
lts: true
Fails with the following error message
npm ERR! peer dep missing: eslint@>=3.1.0, required by [email protected]
Full error
#!/bin/bash -eo pipefail
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
NVM_DIR="$HOME/.nvm"
echo "export NVM_DIR=\"$HOME/.nvm\"" >> $BASH_ENV
echo "[ -s \"$NVM_DIR/nvm.sh\" ] && \. \"$NVM_DIR/nvm.sh\"" >> $BASH_ENV
source $BASH_ENV
nvm install \
--lts
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13527 100 13527 0 0 102k 0 --:--:-- --:--:-- --:--:-- 102k
=> nvm is already installed in /opt/circleci/.nvm, trying to update using git
=> => Compressing and cleaning up git repository
=> nvm source string already in /home/circleci/.bashrc
=> Appending bash_completion source string to /home/circleci/.bashrc
npm ERR! peer dep missing: eslint@>=3.1.0, required by [email protected]
=> You currently have modules installed globally with `npm`. These will no
=> longer be linked to the active version of Node when you install a new node
=> with `nvm`; and they may (depending on how you construct your `$PATH`)
=> override the binaries of modules installed with `nvm`:
/opt/circleci/.nvm/versions/node/v6.1.0/lib
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
=> If you wish to uninstall them at a later point (or re-install them under your
=> `nvm` Nodes), you can remove them from the system Node as follows:
$ nvm use system
$ npm uninstall -g a_module
=> Close and reopen your terminal to start using nvm or run the following to use it now:
export NVM_DIR="/opt/circleci/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
Exited with code exit status 1
CircleCI received exit code 1
Install Node v12
Small description update. Should be included in next patch update.
"Easily install Node.js and its package mangers (npm, yarn). Best of all, install your packages with caching enabled by default."
mangers > managers
The orb currently supports Yarn 1, but requires these parameter overrides to work with Yarn 2:
# Yarn 2 deprecates "--frozen-lockfile" in favour of "--immutable"
override-ci-command: yarn install --immutable
# Yarn 2 uses this rather than "node_modules"
cache-path: .yarn/cacheIt would be good if the orb supports Yarn 2 natively; perhaps by adding a Yarn 2 option to the pkg-manager parameter?
2.1.0
node/install-packages command executes:
npm install by defaultnpm install with override-ci: falsenpm install with override-ci: truenode/install-packages command should execute:
npm ci by defaultnpm ci with override-ci: falsenpm install with override-ci: trueFrom the description of override-ci:
By default, modules will be installed with "npm ci" or "yarn install --frozen-lockfile", this override will use the standard "install" command.
Also the same problem occurs with yarn.
The if statement is wrong.
It should be
if [[ << parameters.override-ci >> == true ]]; thenor
if << parameters.override-ci >>; then After solving the tiny bug above, the next problem is the cache.
Currently node/install-packages command caches/restores node_modules directory, but it doesn't work for npm ci because npm ci removes node_modules first.
So it should cache ~/.npm for npm ci instead.
circleci/[email protected]
when your node project uses an npm-shrinkwrap.json file (docs), jobs fail to restore or save cache:
error computing cache key: template: cacheKey:1:30: executing "cacheKey" at <checksum "~/project/package-lock.json">: error calling checksum: open /home/circleci/project/package-lock.json: no such file or directory
cache key should be calculated using either package-lock.json (as it does today) or npm-shrinkwrap.json
@ganezasan has found that the Yarn install incorrectly installs under MacOS due to including an incorrect PATH (for MacOS).
https://github.com/CircleCI-Public/node-orb/blob/master/src/commands/install-yarn.yml#L21
First reported here: https://discuss.circleci.com/t/having-fork-resource-temporarily-unavailable-when-running-commands-on-macos/30966/4?u=jcm
The incorrect path leads to slow processes on MacOS and errors similar to the following:
^D^D/tmp/.bash_env-5d10d6f32cffde0008c4dc34-0-build: fork: Resource temporarily unavailable
Detect OS and change install path (@ganezasan has mentioned removing this line would work).
circleci/[email protected]
This is my configuration for this orb:
- node/install-packages:
pkg-manager: yarn
but always the "Installing YARN packages" step install the packages instead of using the cache.
This is the outpout of the "Restoring cache" step:
Found a cache from build 28614 at node-deps-arch1-linux-amd64-6_85-v1-update-circle-ci-orbs-5iWF2YzHP8e1G6HhzxszgwKVJ5IrCah37qVy9y7D96U=-pCvIcgffSUKf4AQQJL2DcbU1PjkyP85xag3cObI_gM8=
Size: 32 B
Cached paths:
* /home/circleci/yarn.lock
Downloading cache archive...
Validating cache...
Unarchiving cache...
I've also tried to use cache-path: ~/project/node_modules but it didn't work, in fact it wasn't used at all.
Should not the cached path be the following? * /home/circleci/project/yarn.lock
Using the version 2 of this orb (the current one in my project) the cached path used is this one: * /home/circleci/project/node_modules
It should skip installing packages if there is a cache.
run_generate:
executor: node/default
parameters:
BASE_URL:
type: string
environment:
BASE_URL: << parameters.BASE_URL >>
steps:
- checkout
- node/install:
lts: true
- node/install-npm
- run:
command: |
npm ci --prefer-offline --no-audit
npm run generate
#!/bin/bash -eo pipefail
if [[ $EUID == 0 ]]; then export SUDO=""; else export SUDO="sudo"; fi
# FUNCTIONS
get_npm_version () {
if [[ "$NODE_PARAM_NPM_VERSION" == latest ]]; then
NPM_ORB_VERSION="$(npm view npm | sed -E 's|.*-||g' | grep latest | \
perl -pe 's/.*: //g' | perl -pe "s/'|,//g")"
echo "Latest version of NPM is $NPM_ORB_VERSION"
else
NPM_ORB_VERSION="$NODE_PARAM_NPM_VERSION"
echo "Selected version of NPM is $NPM_ORB_VERSION"
fi
}
installation_check () {
if command -v npm > /dev/null 2>&1; then
if npm -v | grep "$NPM_ORB_VERSION" > /dev/null 2>&1; then
echo "NPM $NPM_ORB_VERSION is already installed"
exit 0
fi
fi
}
get_npm_version
installation_check
if [ "$NODE_PARAM_NPM_VERSION" = latest ]; then
$SUDO npm install -g npm@latest > /dev/null 2>&1 || \
npm install -g npm@latest > /dev/null 2>&1
else
$SUDO npm install -g "npm@$NPM_ORB_VERSION" > /dev/null 2>&1 || \
npm install -g "npm@$NPM_ORB_VERSION" > /dev/null 2>&1
fi
# test/verify version
if npm -v | grep "$NPM_ORB_VERSION" > /dev/null 2>&1; then
echo "Success! NPM $(npm -v) has been installed to $(which npm)"
else
echo "Something went wrong; the specified version of NPM could not be installed"
exit 1
fi
Latest version of NPM is 8.1.4
Something went wrong; the specified version of NPM could not be installed
Exited with code exit status 1
CircleCI received exit code 1
Install NPM
2.0.2
After switching from 1.x version, I lose the possibility to run custom steps while installing Node dependencies. Before I used for specifying some flags for yarn install command. I tried also to add yarn install command to package.json scripts, but that didn't work properly since it's running yarn install as usual first and then custom yarn install from package.json.
Here is an example of how I installed the deps before:
install_deps:
steps:
- node/with-cache:
cache-version: v1
cache-key: yarn.lock
dir: ~/repo/node_modules
use-strict-cache: true
steps:
- run: yarn install --pure-lockfile --ignore-scripts --no-progressAnd to install just prod deps:
install_deps:
steps:
- node/with-cache:
cache-version: v1
cache-key: yarn.lock
dir: ~/repo/node_modules
use-strict-cache: true
steps:
- run: yarn install --prod --pure-lockfile --ignore-scripts --no-progressI want to be able to run a custom command or at least flags for the package manager while installing the dependencies.
install_deps:
steps:
- node/install-packages:
app-dir: ~/repo
cache-version: v1
cache-key: yarn.lock
override-ci: true
override-ci-flags: --prod --pure-lockfile
pkg-manager: yarnOr
install_deps:
steps:
- node/install-packages:
app-dir: ~/repo
cache-version: v1
cache-key: yarn.lock
override-ci: true
override-ci-command: yarn install --prod --pure-lockfile --ignore-scripts --no-progress
pkg-manager: yarn4.1.0
I have the following configuration:
version: 2.1
orbs:
node: circleci/[email protected]
jobs:
test:
docker:
- image: cimg/node:12.18.2
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_PASSWORD
steps:
- checkout
- node/install-packages:
pkg-manager: yarn
cache-path: ~/.cache/yarn
workflows:
test:
jobs:
- test:
context:
- docker-hub-credsThere is, unfortunately, no visual way to describe what I think is happening, because yarn install's output is the same whether the dependencies are cached or not. Only the duration can be an indicator, but it's too variable between jobs to be reliable.
If I understand correctly the code, when cache-path is specified, that's the path which will be cached by save_cache.
That path is, however, not used by yarn install, unless I override the command with override-ci-command: yarn install --frozen-lockfile --cache-folder ~/.cache/yarn.
The yarn install command uses the cache directory specified by cache-path.
Alternatively, the documentation could be clarified to explain that the install command needs to be overridden with override-ci-command to specify the cache folder.
Currently the documentation for the cache-path parameter is confusing for yarn users:
By default, this orb will utilize 'npm ci' and cache the '~/.npm' directory. Override which path to cache with this parameter.
AFAIK, this is also true for npm.
I haven't used recent versions of npm and I don't know how its cache works, though.
5.0.0
We occasionally see flaky builds where node or NPM fails to install. For example, here's the output from one recent failure:
#!/bin/bash -eo pipefail
# Only install nvm if it's not already installed
if command -v nvm &> /dev/null; then
echo "nvm is already installed. Skipping nvm install.";
else
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash;
echo 'export NVM_DIR="$HOME/.nvm"' >> "$BASH_ENV";
echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" --no-use' >> "$BASH_ENV";
# shellcheck source=/dev/null
source "$BASH_ENV";
fi
# See: https://github.com/nvm-sh/nvm#usage
if [ "$NODE_PARAM_VERSION" = "latest" ]; then
# When no version is specified we default to the latest version of Node
NODE_ORB_INSTALL_VERSION=$(nvm ls-remote | tail -n1 | grep -Eo 'v[0-9]+\.[0-9]+\.[0-9]+')
nvm install "$NODE_ORB_INSTALL_VERSION" # aka nvm install node. We're being explicit here.
nvm alias default "$NODE_ORB_INSTALL_VERSION"
elif [ -n "$NODE_PARAM_VERSION" ]; then
nvm install "$NODE_PARAM_VERSION"
nvm alias default "$NODE_PARAM_VERSION"
elif [ -f ".nvmrc" ]; then
NVMRC_SPECIFIED_VERSION=$(<.nvmrc)
nvm install "$NVMRC_SPECIFIED_VERSION"
nvm alias default "$NVMRC_SPECIFIED_VERSION"
else
nvm install --lts
nvm alias default lts/*
fi
echo 'nvm use default &>/dev/null' >> "$BASH_ENV"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13527 100 13527 0 0 1016k 0 --:--:-- --:--:-- --:--:-- 1016k
=> Downloading nvm from git to '/home/circleci/.nvm'
=> Cloning into '/home/circleci/.nvm'...
ssh: connect to host github.com port 22: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Failed to clone nvm repo. Please report this!
Exited with code exit status 2
CircleCI received exit code 2
Node should reliably install in the container.
Enable support for ARM-based installation and execution for CircleCI's upcoming ARM build fleet.
CircleCI ARM build fleet: https://www2.circleci.com/arm.html
N/A
I added a new project to CCI using the config builder. I couldn't figure out why it was behaving differently from what the documentation said—in particular, it wasn't installing the version of Node listed in my .nvmrc. Turns out it had given me circleci/[email protected].
Config builder should suggest circleci/node@5 instead.
I know this isn't really a change to me made in this repo, but I'm not sure where else to file this. 😄
2.0.0
When using the node/test job with yarn the caching of dependencies failed when looking for package-lock.json.
The node/test job passes node/install-packages a cache-key parameter that matches the pkg-manager parameter: yarn.lock.
1.0.1
The install command does not install node or npm into the $PATH. Consequently, downstream scripts that expect a global node or npm command will not work.
The install command should make global node and npm commands available to downstream scripts.
4.7 and 5.01
Tried to install Node 14.19 and Yarn, it errors with gzip: stdin: not in gzip format, tar: Child returned status 1
Circle Config Command:
- node/install:
install-yarn: true
node-version: '14.19'
I was getting this error on orb version 4.7, upgraded to 5.01 and still getting the error. Guessing it's an issue with the latest Yarn build?
It just started happening today, was running multiple builds and then it just stopped working.
Build log:
#!/bin/bash -eo pipefail
if [[ $EUID == 0 ]]; then export SUDO=""; else export SUDO="sudo"; fi
# FUNCTIONS
get_yarn_version () {
if [[ "$NODE_PARAM_YARN_VERSION" == "" ]]; then
YARN_ORB_VERSION=$(curl -Ls -o /dev/null -w "%{url_effective}" \
"https://github.com/yarnpkg/yarn/releases/latest" | sed 's:.*/::' | cut -d 'v' -f 2 | cut -d 'v' -f 2)
echo "Latest version of Yarn is $YARN_ORB_VERSION"
else
YARN_ORB_VERSION="$NODE_PARAM_YARN_VERSION"
echo "Selected version of Yarn is $YARN_ORB_VERSION"
fi
}
installation_check () {
echo "Checking if YARN is already installed..."
if command -v yarn > /dev/null 2>&1; then
if yarn --version | grep "$YARN_ORB_VERSION" > /dev/null 2>&1; then
echo "Yarn $YARN_ORB_VERSION is already installed"
exit 0
else
echo "A different version of Yarn is installed ($(yarn --version)); removing it"
if uname -a | grep Darwin > /dev/null 2>&1; then
brew uninstall yarn > /dev/null 2>&1
elif grep Alpine /etc/issue > /dev/null 2>&1; then
apk del yarn > /dev/null 2>&1
elif grep Debian /etc/issue > /dev/null 2>&1; then
$SUDO apt-get remove yarn > /dev/null 2>&1 && \
$SUDO apt-get purge yarn > /dev/null 2>&1
elif grep Ubuntu /etc/issue > /dev/null 2>&1; then
$SUDO apt-get remove yarn > /dev/null 2>&1 && \
$SUDO apt-get purge yarn > /dev/null 2>&1
elif command -v yum > /dev/null 2>&1; then
yum remove yarn > /dev/null 2>&1
fi
$SUDO rm -rf "$HOME/.yarn" > /dev/null 2>&1
$SUDO rm -f /usr/local/bin/yarn /usr/local/bin/yarnpkg > /dev/null 2>&1
fi
fi
}
# cd to home so that yarn --version will not use relative installed yarn from .yarn/releases
cd ~ || echo "Cannot change directory to home directory, yarn version may be mismatched."
get_yarn_version
installation_check
# install yarn
echo "Installing YARN v$YARN_ORB_VERSION"
curl -L -o yarn.tar.gz --silent "https://yarnpkg.com/downloads/$YARN_ORB_VERSION/yarn-v$YARN_ORB_VERSION.tar.gz"
$SUDO tar -xzf yarn.tar.gz && rm yarn.tar.gz
$SUDO mkdir -p /opt/yarn
$SUDO mv yarn-v"${YARN_ORB_VERSION}"/* /opt/yarn
$SUDO rm -rf "yarn-v${YARN_ORB_VERSION}"
$SUDO chmod 777 "/opt/yarn"
$SUDO ln -s /opt/yarn/bin/yarn /usr/local/bin/yarn
$SUDO ln -s /opt/yarn/bin/yarnpkg /usr/local/bin/yarnpkg
$SUDO ln -s /opt/yarn/bin/yarn.js /usr/local/bin/yarn.js
$SUDO mkdir -p ~/.config
if uname -a | grep Darwin; then
$SUDO chown -R "$USER:$GROUP" ~/.config
$SUDO chown -R "$USER:$GROUP" /opt/yarn
else
$SUDO chown -R "$(whoami):$(whoami)" /opt/yarn
$SUDO chown -R "$(whoami):$(whoami)" ~/.config
fi
# test/verify version
echo "Verifying YARN install"
if yarn --version | grep "$YARN_ORB_VERSION" > /dev/null 2>&1; then
echo "Success! Yarn $(yarn --version) has been installed to $(which yarn)"
else
echo "Something went wrong; the specified version of Yarn could not be installed"
exit 1
fi
Latest version of Yarn is 1.22.18
Checking if YARN is already installed...
A different version of Yarn is installed (1.22.5); removing it
Installing YARN v1.22.18
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
mv: cannot stat 'yarn-v1.22.18/*': No such file or directory
Exited with code exit status 1
CircleCI received exit code 1
1.0.0
The orb needs usage examples. I spent a not of time trying to figure out how to use the executor.
Usage examples :)
5.0.0
I'm using Yarn policies (e.g., yarn policy set-version 1.22.11) to bundle Yarn with my app. .yarnrc looks like this:
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
lastUpdateCheck 1579039968308
yarn-path ".yarn/releases/yarn-1.22.11.js"
I'm trying to use this orb to install Node and Yarn, like so:
- node/install:
install-yarn: trueThis downloads the latest stable version of Yarn, which is 1.22.17 at the time of writing. Unfortunately, the installation process fails at the end with this error:
Verifying YARN install
Something went wrong; the specified version of Yarn could not be installed
This happens because after the Yarn installation is complete (1.22.17 is successfully installed) yarn --version returns 1.22.11, which is the version specified in .yarnrc. This mismatch causes the process to exit.
I'm not sure, to be honest! A few options are possible:
.yarnrc that would helpI could hard-code the Yarn version in the task config, but that defeats the purpose of using .yarnrc. I could also somehow parse .yarnrc myself and extract the version, but that's the yarn executable's job. Any advice here would be appreciated!
circleci/[email protected]
Neither pnpm nor yarn do not populate binaries after installing dependencies. So for example after pnpm install or yarn commands eslint is not available when invoking with either of them in package.json scripts. Only after installing those packages globally are they available to run. Below is a screenshot of the failing job.
I've also tried to SSH and checked myself. Binaries are actually not created for some reason. See the screenshot.
Also here is part of the config that defines that job:
version: 2.1
orbs:
node: circleci/[email protected]
...
jobs:
dev_eslint:
executor:
name: node/default
tag: 'lts'
steps:
- checkout
- run:
name: Node.js version
command: node --version
- run:
name: Install pnpm@6
command: |
yarn global add pnpm esbuild
pnpm config set store-dir ~/.pnpm-store
- restore_cache:
keys:
- v1-deps-{{ checksum "pnpm-lock.yaml" }}
- node/install-packages:
override-ci-command: pnpm install
with-cache: false
- save_cache:
paths:
- ~/.pnpm-store
key: v1-deps-{{ checksum "pnpm-lock.yaml" }}
- run:
name: Run ESLint project-wide
command: |
export TIMING=1 && pnpm prettier:client:base --check
pnpm eslint:client --quiet && pnpm lint:css
pnpm prettier:server:base --check
pnpm eslint:server --quiet
pnpm prettier:common:base --check
pnpm eslint:common --quietI had issues with installing pnpm the way their docs suggest that global install was not permitted so I found myself using yarn global add pnpm which actually succeeded.
Let me know if you need more details.
Create package binaries and successfully invoke from package.json scripts.
I add node/install like this:
- node/install:
install-yarn: true
install-npm: false
lts: true
But, the lts version not set as default.
This is the example evidence: https://app.circleci.com/pipelines/github/berv-uni-project/tweety-ui/34/workflows/f5060f49-1048-47d3-b6d4-8d73b897278f/jobs/34
The lts version will be default.
2.1.0
- node/install:
install-yarn: true
node-version: latest
results in the Installing NodeJS Latest job
The installing run time is very unstable. The worst cases were more than 10mins and 20mins even. And it would fail sometimes due to timeout.
1.2.0
Currently node/npm/yarn versions must be set in config.yml
Heroku and other platforms install runtime version specified in package.json
https://docs.npmjs.com/files/package.json#engines
It would be default behavior, or options to support this would be added
It is very common, and actually recommended in the CircleCI documentation, that we specify several keys in restore_cache, so that the cache can be reused and primed even if the underlying lock files change.
For example, Here is a real-life configuration from my project:
Saving is:
- save_cache:
paths:
- node_modules
key: v2-dependencies-{{ checksum "package.json" }}-{{ checksum "yarn.lock" }}
Restoring is:
- restore_cache:
keys:
- v2-dependencies-{{ checksum "package.json" }}-{{ checksum "yarn.lock" }}
- v2-dependencies-{{ checksum "package.json" }}-
- v2-dependencies-
But this isn't what the orb is doing:
node-orb/src/commands/install-packages.yml
Lines 62 to 67 in a8e4186
node-orb/src/commands/install-packages.yml
Lines 100 to 105 in a8e4186
Is there a drawback to always doing this? Or should we make this a configuration option?
I'd be happy looking at this if there is consensus about it.
Thanks!
4.7.0
When setting up the build environment on cimg/ruby:2.7.4-browsers, this orb fails when checking npm. It succeeds in installing node:
#!/bin/bash -eo pipefail
# Only install nvm if it's not already installed
if command -v nvm &> /dev/null; then
echo "nvm is already installed. Skipping nvm install.";
else
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash;
echo 'export NVM_DIR="$HOME/.nvm"' >> $BASH_ENV;
echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" --no-use' >> $BASH_ENV;
source $BASH_ENV;
fi
if [ "$NODE_PARAM_LTS" = "1" ]; then
nvm install --lts
nvm alias default lts/*
else
nvm install "$NODE_PARAM_VERSION"
nvm alias default "$NODE_PARAM_VERSION"
fi
echo 'nvm use default &>/dev/null' >> $BASH_ENV
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13527 100 13527 0 0 1016k 0 --:--:-- --:--:-- --:--:-- 1016k
=> Downloading nvm from git to '/home/circleci/.nvm'
=> Cloning into '/home/circleci/.nvm'...
Warning: Permanently added the ECDSA host key for IP address '140.82.112.3' to the list of known hosts.
remote: Enumerating objects: 333, done.
remote: Counting objects: 100% (333/333), done.
remote: Compressing objects: 100% (283/283), done.
Receiving objects: 100% (333/333), 177.15 KiB | 8.05 MiB/s, done.
remote: Total 333 (delta 38), reused 150 (delta 25), pack-reused 0
Resolving deltas: 100% (38/38), done.
=> Compressing and cleaning up git repository
=> Appending nvm source string to /home/circleci/.bashrc
=> Appending bash_completion source string to /home/circleci/.bashrc
=> Close and reopen your terminal to start using nvm or run the following to use it now:
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
Downloading and installing node v14.18.1...
Downloading https://nodejs.org/dist/v14.18.1/node-v14.18.1-linux-x64.tar.xz...
######################################################################## 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v14.18.1 (npm v6.14.15)
Creating default alias: default -> 14.18.1 (-> v14.18.1 *)
default -> 14.18.1 (-> v14.18.1 *)
CircleCI received exit code 0
But then it fails when setting up NPM:
#!/bin/bash -eo pipefail
if [[ $EUID == 0 ]]; then export SUDO=""; else export SUDO="sudo"; fi
# FUNCTIONS
get_npm_version () {
if [[ "$NODE_PARAM_NPM_VERSION" == latest ]]; then
NPM_ORB_VERSION="$(npm view npm | sed -E 's|.*-||g' | grep latest | \
perl -pe 's/.*: //g' | perl -pe "s/'|,//g")"
echo "Latest version of NPM is $NPM_ORB_VERSION"
else
NPM_ORB_VERSION="$NODE_PARAM_NPM_VERSION"
echo "Selected version of NPM is $NPM_ORB_VERSION"
fi
}
installation_check () {
if command -v npm > /dev/null 2>&1; then
if npm -v | grep "$NPM_ORB_VERSION" > /dev/null 2>&1; then
echo "NPM $NPM_ORB_VERSION is already installed"
exit 0
fi
fi
}
get_npm_version
installation_check
if [ "$NODE_PARAM_NPM_VERSION" = latest ]; then
$SUDO npm install -g npm@latest > /dev/null 2>&1 || \
npm install -g npm@latest > /dev/null 2>&1
else
$SUDO npm install -g "npm@$NPM_ORB_VERSION" > /dev/null 2>&1 || \
npm install -g "npm@$NPM_ORB_VERSION" > /dev/null 2>&1
fi
# test/verify version
if npm -v | grep "$NPM_ORB_VERSION" > /dev/null 2>&1; then
echo "Success! NPM $(npm -v) has been installed to $(which npm)"
else
echo "Something went wrong; the specified version of NPM could not be installed"
exit 1
fi
Latest version of NPM is 8.1.4
Something went wrong; the specified version of NPM could not be installed
Exited with code exit status 1
CircleCI received exit code 1
It should successfully install NPM.
This works fine on the legacy circleci/ruby:2.7.4-node-browsers image.
v1.1.6
I have the following circle config:
version: 2.1
orbs:
aws-cli: circleci/[email protected]
jq: circleci/[email protected]
node: circleci/[email protected]
docker:
- image: alpine:3.7
jobs:
build_and_deploy_lambda:
parameters:
deploy_env:
type: string
description: "The environment that we're deploying to. Either 'dev', 'stage', or 'prod'."
executor: aws-cli/default
steps:
- checkout
- aws-cli/setup
- node/install:
install-yarn: false
node-version: "10.17.0"
- run: ./deploy_function -e << parameters.deploy_env >>
The majority of the time this works with no issues, but a small portion of the time we're seeing the following from the node/install step:
Selected version of Node.js is 10.17.0
Importing Node.js trusted release keys...
Too long with no output (exceeded 10m0s): context deadline exceeded
We would like a way to address this, even if that is the ability to bump the timeout for whatever internal orb request is failing.
The orb does not time out on node/install and if that is unavoidable (since you folks don't control node.js servers) then we have the ability to bump the install timeout so that this issue does not happen as often 🤞.
The docs state:
Download and install any version of Node.js. Optionally installs Yarn with a parameter. You may use the default executor of this orb for a Node execution environment as well.
Yet no YML directive or API is documented for how to do this. This could use clarification as I don't quite understand what the docs are trying to say.
Cannot run node/install-packages running using a Ramdisk (CircleCI doc).
Extract from config.yml:
working_directory: /mnt/ramdisk
steps:
- checkout
- node/install-packages
Error:
ln: failed to create hard link '/tmp/node-project-lockfile' => 'package-lock.json': Invalid cross-device link
Running in a Ramdisk should still allow to install packages.
Full error:
#!/bin/bash -eo pipefail
if [ -f "package-lock.json" ]; then
echo "Found package-lock.json file, assuming lockfile"
ln package-lock.json /tmp/node-project-lockfile
elif [ -f "npm-shrinkwrap.json" ]; then
echo "Found npm-shrinkwrap.json file, assuming lockfile"
ln npm-shrinkwrap.json /tmp/node-project-lockfile
elif [ -f "yarn.lock" ]; then
echo "Found yarn.lock file, assuming lockfile"
ln yarn.lock /tmp/node-project-lockfile
fi
ln package.json /tmp/node-project-package.json
Found package-lock.json file, assuming lockfile
ln: failed to create hard link '/tmp/node-project-lockfile' => 'package-lock.json': Invalid cross-device link
Exited with code exit status 1
CircleCI received exit code 1
config.yml:
version: 2.1
orbs:
node: circleci/node@5
jobs:
build:
docker:
- image: cimg/node:14.18.1
working_directory: /mnt/ramdisk
steps:
- checkout
- node/install-packages:
include-branch-in-cache-key: false
Edit: upgraded to circleci/node@5
1.1.6
Installation of version "12" with version "13" already installed yields:
Selected version of Node.js is v12.13.0
A different version of Node.js is installed (v13.0.1); removing it
Importing Node.js trusted release keys...
curl: (22) The requested URL returned error: 404
Exited with code 22
CircleCI received exit code 22
run with step:
...
- node/install:
node-version: "12"
...
I believe this is caused by the VERSION variable already being prefixed with v yielding an invalid URL here: https://github.com/CircleCI-Public/node-orb/blob/master/src/commands/install-node.yml#L195-L197
It should remove node version 13 and install node version 12.
In our node/install-packages step, I notice the restore_cache output to be something like this:
No cache is found for key: node-deps-arch1-linux-amd64-6_85-v1-WEBPLAT-1424-otawg9WNM0SS_vutk39ZUPxQPgxUCFt5g4TXo9whdYs=
Found a cache from build 188495 at node-deps-arch1-linux-amd64-6_85-v1-WEBPLAT-1424-3zVE04aLnLzo5ttkKIxeRB9NIhoXVIiqLiwkGMKBONA=
After which npm ci runs for 5 minutes or so. After that is done, I notice that it is skipping cache generation with such a message
Skipping cache generation, cache already exists for key: node-deps-arch1-linux-amd64-6_85-v1-WEBPLAT-1424-3zVE04aLnLzo5ttkKIxeRB9NIhoXVIiqLiwkGMKBONA=-otawg9WNM0SS_vutk39ZUPxQPgxUCFt5g4TXo9whdYs=
Based on the source code as I see here:
node-orb/src/commands/install-packages.yml
Lines 68 to 72 in ccf5f1a
npm ci step.
Additional context from our config.yaml:
node: circleci/[email protected]
We're running node/install-packages with no custom parameters.
5.0.0
Installing YARN Packages job failed when using yarn-berry without zero installs - the else statement to execute yarn install --immutable is unreachable.
Internal Error: Cache path does not exist.
at Nt.setup (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:430:1519)
at async Function.find (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:430:728)
at async Cm.execute (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:501:12109)
at async Cm.validateAndExecute (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:349:673)
at async Is.run (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:363:2087)
at async Is.runExit (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:363:2271)
at async i (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:448:12669)
at async r (/home/circleci/project/.yarn/releases/yarn-3.2.0.cjs:448:10887)
Yarn should not use the --immutable-cache flag when the cache location doesn't exist
The negation in this expression is a bug - the zero installs is only executing when the cache directory doesn't exist which is incorrect:
Temporary workaround: override-ci-command: yarn install --immutable
1.1.0
Running yarn config set cache-folder ~/.cache/yarn now needs sudo. It's weird for it to ask for sudo since it should be installed for the current user.
This started happening after upgrading from 1.0.1 to 1.1.0 to get the fixes from #5
Should run without sudo(?)
If that is expected please let me know.
This failure was observed using 4.0.1 and 4.1.0. The failure went away under 3.0.1 as that version does not use nvm, so looks like a nvm error. I did not investigate further so as to not waste my mac minutes and that 3.0.1 worked.
#!/bin/bash --login -eo pipefail
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
NVM_DIR="$HOME/.nvm"
echo "export NVM_DIR=\"$HOME/.nvm\"" >> $BASH_ENV
echo "[ -s \"$NVM_DIR/nvm.sh\" ] && \. \"$NVM_DIR/nvm.sh\"" >> $BASH_ENV
source $BASH_ENV
nvm install \
12.13.1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13527 100 13527 0 0 209k 0 --:--:-- --:--:-- --:--:-- 213k
=> Downloading nvm from git to '/Users/distiller/.nvm'
=> Cloning into '/Users/distiller/.nvm'...
remote: Enumerating objects: 290, done.
remote: Counting objects: 100% (290/290), done.
remote: Compressing objects: 100% (257/257), done.
remote: Total 290 (delta 34), reused 110 (delta 20), pack-reused 0
Receiving objects: 100% (290/290), 163.37 KiB | 4.54 MiB/s, done.
Resolving deltas: 100% (34/34), done.
=> Compressing and cleaning up git repository
=> Appending nvm source string to /Users/distiller/.bashrc
=> Appending bash_completion source string to /Users/distiller/.bashrc
N/A: version "12.13.0 -> N/A" is not yet installed.
You need to run "nvm install 12.13.0" to install it before using it.
=> Close and reopen your terminal to start using nvm or run the following to use it now:
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
N/A: version "12.13.0 -> N/A" is not yet installed.
You need to run "nvm install 12.13.0" to install it before using it.
Exited with code exit status 3
CircleCI received exit code 3
Node would be installed, and build would not fail
Stripped down build config:
version: 2.1
orbs:
node: circleci/[email protected]
jobs:
build:
macos:
xcode: '11.4'
environment:
HOMEBREW_NO_AUTO_UPDATE: 1
steps:
- checkout
- run:
name: Setup dependencies
command: brew install unixodbc gnupg coreutils
- node/install:
install-yarn: true
node-version: "12.13.1"
workflows:
version: 2
build:
jobs:
- build
circleci/[email protected]
- node/install:
install-yarn: true
install-npm: false
node-version: 16.13.0
yarn-version: 1.22.17
led to:
Selected version of Yarn is 1.22.17
Checking if YARN is already installed...
Installing YARN v1.22.17
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
mv: cannot stat 'yarn-v1.22.17/*': No such file or directory
an installed yarn.
I found a workaround and a possible explanation for why this failed to install
https://discuss.circleci.com/t/yarn-install-failing-node-orb/41477
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.