Comments (10)
This has happened to me before when I forget to immediately pipe a stream (or call putStream) after getting it. If you don't do that, you'll miss data events and Amazon will get confused, giving you a 400. This often only shows up in strange cases, like larger files or different uploading clients.
I don't know if that's your exact problem though. It could be our bug, certainly. If you can get a small comparison test case of code that works vs some that doesn't I'll gladly look into it.
from knox.
exports.awsPut = function(req, res) {
var headers = {
'Content-Length' : req.headers['content-length']
};
var amazonReq = knoxClient.putStream(req, '/slideDigest/' + req.query["fileName"], headers, function(err, amazonRes){
var stringData = "";
amazonRes.on("data", function(rData){
stringData += rData;
});
amazonRes.on("end", function(){
console.log(amazonRes.statusCode);
});
});
amazonReq.on("progress", function(e){
console.info(e)
});
};
This doesn't works.
In production I have something like that:
var amazonRequest = client.request('PUT', '/' + dest
+ '?partNumber=' + params.query.partNumber + '&uploadId='
+ params.query.uploadId, {
'Content-Length' : req.headers['content-length']
});
req.pipe(amazonRequest);
And it works
from knox.
The strange thing is that on the progress event all files ( the ones that give me a 200 or 400 ) achieve 100%
from knox.
What is happening to req before you pass it in to awsPut (if anything)?
from knox.
Nothing. This is just a simple router from the expressjs
from knox.
I wonder if it has to do with how client.put(...)
(which is called by client.putStream
) adds Expect: '100-continue'
. Note that request('PUT', ...)
does not add that. Still not in a position to test this properly for a day or two, but you could try adding Expect: undefined
or Expect: ""
to your headers hash and see if that fixes things. Very interesting...
from knox.
Strangely when I've arrived to the office today the problem is gone. I'm able to upload larger files to the Amazon Bucket without any problems. The stranger thing is that my production code doesn't behave like that ( and we have a bunch of clients uploading files larger than 500mb simultaneous ).
Anyway, Thanks for the help and if this happens again I'll keep you posted.
from knox.
I also encountered this problem, my code below
var wstream = client.put(dest, {
'Content-Length': stat.size,
'Content-Type': 'image/jpeg',
'x-amz-acl': 'public-read'
})
wstream.on('error', function(err) {
cb(err)
})
wstream.on('response', function(res) {
cb(null, res)
})
wstream.end()
when i delete wstream.end()
, it's ok.
from knox.
That code just immediately closes the stream it tries to put to. That will of course cause a timeout since the connection is waiting for stat.size
bytes but you sent it zero bytes.
from knox.
@domenic you are right, thanks.
from knox.
Related Issues (20)
- Documentation regarding progress
- Option to use x-amz-date header instead of Date
- getFile callback called more than once HOT 1
- Anyone help about Create Bucket with knox? i dont find it HOT 4
- How to debug S3 connection? HOT 3
- Is this repo still maintained? HOT 6
- Allow head requests for S3 buckets
- 403 on getFile on bucket with CRR and versioning on
- Error uploading a file HOT 2
- How is putFile() supposed to work? HOT 2
- Typescript definitions HOT 1
- No access denied error and also missing other errors
- + in secret_key can cause problems
- Download .zip (binary) files HOT 1
- putStream with transfer-encoding chunked HOT 2
- mime dependency breaking changes HOT 11
- Support for deprecated lookup API HOT 3
- Update the readme with deprecation message
- XML parsing Dependency migration HOT 1
- Incorrect sign in function signedUrl when filename contains special chars: !'()#*+? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from knox.