Comments (24)
As a hacky workaround, I've broken it up into smaller sets of files on each task. That seems to solve the problem.
function makeSubTasks(dir) {
grunt.file.expand(path.join(dir, '*')).forEach(function(dir) {
grunt.config('s3.' + dir + '.upload', uploadsOfDir(dir));
});
}
makeSubTasks(devRoot);
// it's kinda shitty to have to do this on grunt.initConfig(), because
// scanning all the files is slow and putting it there means that
// we will do it every time the user types `grunt`
function uploadsOfDir(dir) {
var files = grunt.file.expand(path.join(dir, '**', '*.*'));
return files.map(function(file) {
return {
src: file,
dest: path.sep + file.split(path.sep).slice(1).join(path.sep)
}
});
}
from grunt-s3.
Yeah, I've witnessed this too. I imagine it's just s3 being finicky. I've considered adding automatic retry to everything.
from grunt-s3.
Interesting. Is it possibly related to knox? s3 is supposed to be able to handle huge uploads, right? How can it be failing for < 500 files?
Automatic retrying sounds reasonable.
from grunt-s3.
hmm, good point about knox. There is this issue: Automattic/knox#116
Have you tried grunt-s3 0.2.0-alpha.2
? It uses a version of knox
that has the fix from that issue.
from grunt-s3.
I have been using that version of grunt-s3, and I'm still getting the same issue.
from grunt-s3.
I think that this problem exists for node 0.10.6
but not 0.10.10
.
from grunt-s3.
I can repeat the problem with grunt-s3 0.2.0-alpha.2
and Node 0.10.10
. It doesn't happen all the time, just sometimes.
from grunt-s3.
This issue is happening constantly for me as well with a large amount of files.
from grunt-s3.
+1 Had not seen this error before a month ago, now it's happening on every upload.
from grunt-s3.
+1 happens intermittently but frequently with grunt-s3 0.2.0-alpha.2
and Node 0.10.11
from grunt-s3.
I think (theoretically) this is due to not calling .end() on the knox calls. I'm playing with fixing this too.
from grunt-s3.
Any updates/fixes for this one? It's happening consistently on uploading to S3 for me.
from grunt-s3.
+1 same problem. only uploading a couple dozen files
from grunt-s3.
+1 this intermittent error happens with node v0.10.22
and grunt [email protected]
, [email protected]
(on mac os x maveriks).
from grunt-s3.
+1 still happening
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
+1
from grunt-s3.
Related Issues (20)
- alpha.3 looses headers on upload HOT 1
- Fatal error: write EPIPE HOT 9
- Sync fails for files that don't exist in bucket HOT 4
- Clean up branches HOT 5
- Tests should run against an actual S3 bucket HOT 1
- Consider setting up readthedocs? HOT 1
- Delete Remote Folders
- Task runs without failing or uploading anything HOT 4
- Require templates from s3 HOT 1
- gzip option breaks s3 task with warning HOT 1
- Doesn't do anything HOT 6
- Whoops. HOT 1
- Anyone had the error Fatal error: write ECONNABORTED? HOT 2
- Files not being written into bucket folders HOT 4
- Random ETIMEOUT and hangups after uploading files HOT 3
- After upload invalidate file when it was changed
- Setting meta data HOT 2
- Support for AWS S3 CLI default secrets location
- No compatible version found: grunt-s3@~0.2.0
- Large file downloads fail with length > kMaxLength
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grunt-s3.