Comments (7)
I found your issue and had a look this morning, because I was curious. I added a lot of logs to my build to try and troubleshoot this.
I found that yarn uses ~/.cache/yarn
by default, as it should.
However... it won't populate it if node_modules
is present and uptodate, because it won't try to update it. This means that if your cache with the same cache key already contains node_modules
, then no new file will be added to .cache/yarn
.
Here is the output of restore_cache
:
Found a cache at node-deps-v1-circleci-project-setup-oEx3_vcLhzBBgJRwMsY+dClTUDNYYPpwBWXBvzrAH4g=
Size: 20 MiB
Cached paths:
* /home/circleci/project/node_modules
Downloading cache archive...
Validating cache...
Unarchiving cache...
Maybe what isn't obvious is that cache-path
is used for the save_cache
command, not the restore_cache
command.
Moreover, trying to save to an existing cache key doesn't work either: this is skipped instead of being replaced.
Initially I got the same problem as you when I tried on my project. I could fix it by adding cache-version: v2
to invalidate the cache.
Therefore I think that everything works like it should :-) and that this bug could be closed.
Hope this helps!
from node-orb.
Thanks for taking the time to look at the issue @julienw.
I found that yarn uses ~/.cache/yarn by default, as it should.
I didn't realize that was the default. I don't remember how I chose this value, but that wasn't by looking the default up.
In that particular case, indeed, I don't need to override the install command.
If I specify a cache-path
different from yarn's default cache dir (~/.other-cache/yarn
, for instance), it won't work anymore, however, since the default install command will use yarn's default cache location (~/.cache/yarn
).
That was the point of this ticket. The default yarn install command, when cache-path
is specified, should be yarn install --frozen-lockfile --cache-folder << parameters.cache-path >>
In that regard, I think the issue is still opened. At least if the maintainers think the use case is worth fixing.
Regarding the node_modules
comment: indeed, if it's already present in the job cache (by not having specified cache-path
in a previous job, for instance), and is up-to-date with the lock file, the yarn cache will not be used and save_cache
will be skipped. Updating the cache version is the solution here, as you mentioned.
That wasn't the problem I had, though, but at least now I understand why save_cache
caches node_modules
when cache-path
is not specified.
One thing I just noticed and which could indicate this Orb's purpose doesn't match my use case, is that restore_cache
will only restore on the exact same cache key. The circleci doc (and my previous "manual" implementation) use multiple cache keys for sharing caches, at least partially, between jobs on different branches, in addition to jobs on the same branch.
This is a whole different issue from this one (and deserve its own ticket), though.
from node-orb.
If I specify a cache-path different from yarn's default cache dir (
~/.other-cache/yarn
, for instance), it won't work anymore, however, since the default install command will use yarn's default cache location (~/.cache/yarn
).
Yeah that's right. I wonder what is the use case for this though :-)
Regarding the node_modules comment: indeed, if it's already present in the job cache (by not having specified
cache-path
in a previous job, for instance), and is up-to-date with the lock file, the yarn cache will not be used andsave_cache
will be skipped.
This is even a bit more subtle: if you try to save with the same key as an existing cache (because same lock / same branch), then the save will be skipped altogether, even if there are files to be saved.
So there were actually 2 problems here, if the cache already exists: 1/ no new file in .cache/yarn
because node_modules
is uptodate, and 2/ even with some files, we'd need a different key name anyway.
One thing I just noticed and which could indicate this Orb's purpose doesn't match my use case, is that
restore_cache
will only restore on the exact same cache key. The circleci doc (and my previous "manual" implementation) use multiple cache keys for sharing caches, at least partially, between jobs on different branches, in addition to jobs on the same branch.
This is a whole different issue from this one (and deserve its own ticket), though.
Yeah, actually this is the issue I wanted to file initially and that's why I looked at other issues and found this one first :D I was planning to file this issue indeed.
Update: filed #63
from node-orb.
Yeah that's right. I wonder what is the use case for this though :-)
I don't have any specific use case in mind 🤔 , but since the Orb allows changing the cache path, it should use it in the yarn install
. Otherwise it's inconsistent IMO.
If the intent of the cache-path
parameter was just to cache the default cache path instead of the node_modules
directory, then a boolean parameter would have been better.
Update: filed #63
Thank you.
from node-orb.
I also found it counter-intuitive that cache-path
only works gracefully with npm rather than yarn. I would say it should at least be documented that cache-path
should not be used with yarn unless you override the install command as well.
from node-orb.
Hi! Thank you for your patience. I'll look at getting this into the next major version if possible :)
from node-orb.
Implemented in #94
Please feel free to review the changes and see if they work for your use-case 😄
from node-orb.
Related Issues (20)
- Support pnpm HOT 3
- Change log
- "max-workers" as a parameter or a step to check the size of resource class
- Upgrade Yarn Install Command to Berry HOT 3
- Install Job Fails - Out of sync HOT 1
- Node has issues installing. HOT 1
- Prevent nvm from falling back to building from source HOT 2
- Inability to install requested NodeJS version does not break the build
- May need to be able to pass http session type to the curl commands HOT 1
- Support Bun
- Latest container no longer includes `node-gyp`
- Node fails to download HOT 2
- Change `install-packages` `include-branch-in-cache-key` default to `false` HOT 1
- Caching fails if no lockfile exists HOT 3
- Cache created but ignored by default HOT 2
- install-yarn fails stating sudo command not found
- In yarn-berrry, node_modules are not cached causing the native binaries to be rebuild every time
- install-yarn does not work on machine executors
- Installing yarn-berry does not seem to work HOT 1
- https://hub.docker.com/r/cimg/node
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from node-orb.