Comments (30)
We ran into a related issue, which I believe might be the underlying cause of some of the symptoms described in this thread. I just wanted to document it here.
We're creating zip files on Windows and unzipping them on Linux. (Actually, AWS Elastic Beanstalk is doing the latter for us.) The code that creates the zip files used to be fairly trivial:
return gulp.src('build/**/*')
.pipe(zip('package.zip'))
.pipe(gulp.dest('dist'))
The thing is that the glob also matches directories, which makes gulp-zip create explicit directory entries in the zip file, copying their permissions. Unlike Linux, Windows doesn't require that directories be executable in order to allow access, meaning that the directory entries in the zip file won't be marked as executable. Consequently, unzipping package.zip
on Linux will render every subdirectory inaccessible. We verified this by opening the archive in 7-Zip File Manager and checking the Attributes column, but there are of course many other ways.
The easiest fix we could think of is passing {nodir: true}
to gulp.src
:
return gulp.src('dist/**/*', {nodir: true})
This will only pass files to gulp-zip, keeping it from creating explicit directory entries, and instead creating the directories implicitly upon unzipping files contained inside them. Of course, this means that directory permissions won't be preserved, which may not be what you're after. (I didn't check, but I'm pretty sure the default directory permissions from umask
will be applied in this case.) Also, empty directories won't be created, but you can easily work around that with an empty file.
Of course, as Sindre already mentioned, gulp-chmod and/or gulp-chown will work as well, but you'll need to combined it with gulp-filter to only modify directories, which could become tedious. Alternatively, I guess you could also pipe the files through an object stream yourself, check if they're directories, and change their permissions.
from gulp-zip.
just as a followup for people finding this... this was my workaround: using the gulp-tap:
gulp.task('zip', ['npm', 'subfolders'], function() {
return gulp.src([distributionFolder + '**/*'])
.pipe(tap(function(file) {
if (file.isDirectory()) {
file.stat.mode = parseInt('40777', 8);
}
}))
.pipe(zip(distroZip))
.pipe(gulp.dest(distributionFolder));
});
from gulp-zip.
@jjmartin 3.5 hours of struggling with this today and BOOM your gulp-tap solution works. I REALLY need to get a linux machine so I don't have to deal with windows-related issues...
from gulp-zip.
just in case anyone else is finding this - i think what i actually want is your other project https://github.com/sindresorhus/gulp-chmod
from gulp-zip.
@jjmartin, your tap workaround saved my day!
from gulp-zip.
It was just changed to correctly preserve directory permissions: 864a8fd
from gulp-zip.
Are you saying you don't believe this to be a bug?
Edit: I see what's happening. Before this update, the bad permissions just happened to work for our project. After the update, the permissions were being set correctly and those correct permissions were breaking our app.
from gulp-zip.
@jjmartin @pe8ter I just tested on a Fedora Linux with various sets of permissions and multiple nested files and folders, and I can't reproduce the problem. All the permissions are kept in the zip file as they were before using the plugin.
from gulp-zip.
@pe8ter so, to you, this issue may be closed?
from gulp-zip.
Yes, but it's not my Issue to say so. Thanks for timely responses!
from gulp-zip.
@pe8ter you're welcome! Sure, the issue is not yours, but yet, 1 of 2 issuers are OK with the module's behavior!
from gulp-zip.
maybe it wasn't the permissions on ours that was causing the problem - all the issues we had with our deployment were coming from windows - they all work correctly with 3.0.2 but failed with 3.1.0...
we aren't setting any special permissions on the folders, its a newly created folder that gets zipped. and its hard to tell exactly what the problem is from the AWS lambda side, we have re-downloaded the zip from amazon and not had any problems opening it or running the node scripts inside of them.
from gulp-zip.
@jjmartin We were building on a Windows box too. Time to update!
from gulp-zip.
time to update what tho - i am not sure what i need to be doing differently... i can accept that we are doing something wrong but its pretty default behavior...
gulp.task('zip', ['npm', 'subfolders'], function () {
return gulp.src([distributionFolder + '**/*'])
.pipe(zip(butils.getZipName()))
.pipe(gulp.dest(distributionFolder));
});
from gulp-zip.
This update means it's time for us to not build on a Windows box and deploy to a Linix machine. Hard to say much else without details of your CI.
from gulp-zip.
well that is unacceptable... one of the whole points of my migrating my deployment to gulp was so that it would be cross platform. (was previously using a powershell script).
My build server actually is linux and i believe i still get the issue.
from gulp-zip.
@jjmartin After a few new tests, I could reproduce your problem.
I assume your variable distributionFolder
is a folder path, like this one my-folder/my-sub
, without any slash at the end, so the glob you're having looks like this: my-folder/my-sub**/*
.
Using this glob, you'll get permission errors (I don't know if it's intended, or if it's a bug).
Try to add a slash after the folder name, like this one my-folder/my-sub/**/*
, and you'll get a ZIP without any permission errors (but you will lose the top-level folder my-sub
in that ZIP).
from gulp-zip.
ok thanks - i will give that a try ...
that seems like a workaround or fix that i can live with.
from gulp-zip.
actually after getting a chance to look back at my code that distribution folder does have a trailing slash so it looks somthing like '../myproject-dist/' and the final glob looks like '../myproject-dist/*/' which i think is about what you are saying it should look like, but we did get the issue of Lambda not being able to read the node_modules folder.
is there some windows setting i can put on the folders as i create them that would make the gulp-zip 3.1.0 make the folders 775 again? (or appear a such to the linux container that lambda runs in? )
from gulp-zip.
ok after watching a debug and reading a bunch - i think i understand the underlying problem. Windows is ALWAYS going to return 16822 for its stat.mode on a directory. when translated to octal we get 40666 - (directory)read/write for owner, group and others (no execute)
I'm assuming that the node_modules of my deployment should have execute permissions (otherwise i wouldn't be having this problem)
would you accept a merge request that included an option to force set a mode on the zipped files/folders? (or left them unset)
i think thats the only solution that i would be able to continue to use gulp-zip to do this - i think i'd rather move to my own use of yazl or another library rather than remained pinned to a specific version.
from gulp-zip.
@jjmartin yazl
seems to default to 040775
for directories, so not sure why gulp differs. This should really be brought up on gulp issue tracker, and not here. All this plugin does is pass the values from gulp to yazl. Not interested in having an option. You can use https://github.com/sindresorhus/gulp-chown for that.
from gulp-zip.
ok - i'll look into the gulp-chown - that is totally a fine solution -
yazl did do 775 by default (this is what was set in 3.0.2), but since you added your own mode setting - you aren't using the default anymore - you are using what is coming off of the node file.stat.mode which in this case of folders is the 16822.
from gulp-zip.
Open an issue/PR on https://github.com/gulpjs/vinyl-fs. This plugin is just passing the permissions it get's from there.
from gulp-zip.
@sindresorhus Do you think that's relevant though? I don't think vinyl-fs is doing something wrong per se. It's just that the permissions on Windows and Linux don't align, so it kinda makes sense that you have to do some additional work if you really want to preserve them.
from gulp-zip.
By which I don't mean that gulp-zip should take care of that for you. An option like autoMakeDirectoriesExecutable
would probably save a lot of people a bit of code, but it's far from generic and it opens a can of woms.
from gulp-zip.
As another follow-up for those finding this: Thank you, @timdp! Your idea to use the {nodir: true}
option to gulp.src
worked for me in my situation of using gulp-zip to create a PhoneGap Build package on Windows and having the upload to PGB fail in weird, nondeterministic ways.
from gulp-zip.
This was a great suggestion @jjmartin !! Many thanks for the workaround.
from gulp-zip.
@jjmartin not sure what your tap workaround does but it saved me from hours of figuring it out! thanks
from gulp-zip.
@timdp suggestion to add {nodir: true}
also works when deploying a zip from Windows to AWS Lambda. Without it, Lambda has a problem unzipping zips created by gulp-zip on Windows.
from gulp-zip.
The suggestion from @timdp to add {nodir: true} worked for me as well. To prevent future complaints/"me toos" would you accept a PR to note this in the documentation?
from gulp-zip.
Related Issues (20)
- Replace gulp util
- Changelog HOT 1
- Feature Request: support Yazl's `options.mtime` to enable creating byte-for-byte identical zip archives across rebuilds HOT 2
- readable-stream throws error when using gulp-zip HOT 1
- Can't open zip on Mac OS HOT 7
- Unable to zip Windows "Junction" folders.
- yazl 2.5.0 broke gulp-zip HOT 1
- How to use the options configuration object HOT 1
- gulp-util removal request HOT 1
- gulp-zip change folder permission, from 755 to 666 HOT 6
- Can't install w/ npm HOT 1
- Remove gulp peer dependency? HOT 3
- Files without name are ignored during zip (e.g. ".txt") HOT 2
- yazl TypeError: jsDate.getDate is not a function when setting modfiedTime HOT 1
- How tooutput zip into source directory?
- Request: Add more examples to documentation HOT 3
- Won't zip if folder name contains [square brackets] HOT 2
- FEATURE REQUEST: zip files recursively HOT 1
- Append files to existing zip HOT 1
- Deprecation of Gulp-util HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gulp-zip.