Comments (6)
In your code you don't need subscribe record_parsed event nor call
converter.end explicitly.
Remove the event listener and try again. Also can you share the csv file
you have problem with using something like fileota.com ?
On Tuesday, 3 November 2015, Sean [email protected] wrote:
Thanks for adding the error event Listener from 0.4.3 to 0.4.4
However, my code stopped working when I upgraded to this new version. What
I mean by that is whenever I upload a csv, my code gets stuck now.Current code, for reference:
var form = new formidable.IncomingForm();
form.parse(req, function(err, fields, files) {
console.log(files.file.type);
if (!files || !files.file || files.file.size == '0') {
res.send("CSV must not be empty.");
}else if( files.file.type == 'text/csv'){
var Converter = require("csvtojson").Converter;var csvFileName = files.file.path;
var fileStream = fs.createReadStream(csvFileName);
//new converter instancevar csvConverter = new Converter({constructResult:true});
csvConverter.on("record_parsed",function(resultRow,rawRow,rowIndex) {
//console.log(resultRow); //here is your result json object
fileStream.unpipe(csvConverter);
console.log('manually close the file stream2');
csvConverter.end();
});//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed",function(jsonObj) {
res.json(200, jsonObj);
});//read from file
fileStream.pipe(csvConverter, { end: false });
} else {
res.send("Not a valid type! Must be CSV format");
}
});—
Reply to this email directly or view it on GitHub
#59.
from node-csvtojson.
Here is my csv: http://fileota.com/s/UWzUO
This is off topic, but... I close the stream early because I only need the headers of this csv. If it happens to be a very large csv... I don't want to parse the whole thing, just get the headers in this case. This is so the user on the client can match headers with database values before the actual import.
Here is my revised code with the same problem... I've also taken out the record_parsed event call.
var formidable = require('formidable');
var fs = require("fs");
exports.import = function(req, res){
var form = new formidable.IncomingForm();
form.parse(req, function(err, fields, files) {
if (!files || !files.file || files.file.size == '0') {
res.send("CSV must not be empty.");
}else if( files.file.type == 'text/csv'){
var Converter = require("csvtojson").Converter;
var fileStream = fs.createReadStream(files.file.path);
var csvConverter = new Converter({constructResult:false});
//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed",function(jsonObj) {
return res.json(200, {});
});
//read from file
fileStream.pipe(csvConverter);
} else {
res.send("Not a valid type! Must be CSV format");
}
});
}
Any idea's why node is choking? Thanks in advance
from node-csvtojson.
Hi,
I cannot reproduce this problem with the sample data. All entries have gone
through.
Also converter.end() will not stop emitting "record_parsed" event if there
are more than one line csv row in buffer (memory). You would better close
read stream as well.
Are you running the node process under debugging mode? If true, I l
accidentally left a "debugger;" statement in the code base and I have
removed it in 0.4.5
~Keyang
On 3 November 2015 at 14:48, Sean [email protected] wrote:
Here is my csv: http://fileota.com/s/UWzUO
This is off topic, but... I close the stream early because I only need the
headers of this csv. If it happens to be a very large csv... I don't want
to parse the whole thing, just get the headers in this case. This is so the
user on the client can match headers with database values before the actual
import.Here is my revised code with the same problem... I've also taken out the
record_parsed event call.exports.import = function(req, res){
var form = new formidable.IncomingForm();
form.parse(req, function(err, fields, files) {
if (!files || !files.file || files.file.size == '0') {
res.send("CSV must not be empty.");
}else if( files.file.type == 'text/csv'){
var Converter = require("csvtojson").Converter;
var fileStream = fs.createReadStream(files.file.path);
var csvConverter = new Converter({constructResult:false});//end_parsed will be emitted once parsing finished csvConverter.on("end_parsed",function(jsonObj) { return res.json(200, {}); }); //read from file fileStream.pipe(csvConverter);
} else {
res.send("Not a valid type! Must be CSV format");
}
});
}Any idea's why node is choking? Thanks in advance
—
Reply to this email directly or view it on GitHub
#59 (comment)
.
from node-csvtojson.
Thanks for the tip on closing the read stream... I had forgotten about that.
I will create a new repo and share with you on runnable.io.
I really like this plugin you've created... and the error reporting. Very
powerful.
On Tue, Nov 3, 2015 at 8:47 AM, Keyang Xiang [email protected]
wrote:
Hi,
I cannot reproduce this problem with the sample data. All entries have gone
through.
Also converter.end() will not stop emitting "record_parsed" event if there
are more than one line csv row in buffer (memory). You would better close
read stream as well.
Are you running the node process under debugging mode? If true, I l
accidentally left a "debugger;" statement in the code base and I have
removed it in 0.4.5~Keyang
On 3 November 2015 at 14:48, Sean [email protected] wrote:
Here is my csv: http://fileota.com/s/UWzUO
This is off topic, but... I close the stream early because I only need
the
headers of this csv. If it happens to be a very large csv... I don't want
to parse the whole thing, just get the headers in this case. This is so
the
user on the client can match headers with database values before the
actual
import.Here is my revised code with the same problem... I've also taken out the
record_parsed event call.exports.import = function(req, res){
var form = new formidable.IncomingForm();
form.parse(req, function(err, fields, files) {
if (!files || !files.file || files.file.size == '0') {
res.send("CSV must not be empty.");
}else if( files.file.type == 'text/csv'){
var Converter = require("csvtojson").Converter;
var fileStream = fs.createReadStream(files.file.path);
var csvConverter = new Converter({constructResult:false});//end_parsed will be emitted once parsing finished
csvConverter.on("end_parsed",function(jsonObj) {
return res.json(200, {});
});
//read from file
fileStream.pipe(csvConverter);
} else {
res.send("Not a valid type! Must be CSV format");
}
});
}Any idea's why node is choking? Thanks in advance
—
Reply to this email directly or view it on GitHub
<
https://github.com/Keyang/node-csvtojson/issues/59#issuecomment-153376602>.
—
Reply to this email directly or view it on GitHub
#59 (comment)
.
from node-csvtojson.
No Problem.
Please let me know if you have created something re-producible.
~Keyang
from node-csvtojson.
Close due to lack of activity.
from node-csvtojson.
Related Issues (20)
- How to get a single record?
- json
- ignoreEmpty: true and checkType: true does not work if set output is csv format HOT 2
- It does not create an array of objects when using a ReadStream of CSV HOT 1
- String types which contain a number prefixed by zero are incorrectly detected
- When header has ".", it creates inner objects
- Node's types should not be imported for module to be used in browser HOT 2
- Several comma's prevent csv() identify fields in text
- Vulnerable dependencies (indirect security issues)
- colParser giving error by passing undefined.
- Reduce published npm package size
- feat: stream iterator
- Empty CSV or CSV with just Headers creates an invalid JSON
- api to get only headers
- How can I create nested objects from a CSV line without renaming my CSV headers? HOT 1
- csvtojson : The term 'csvtojson' is not recognized as the name of a cmdlet, function, script file, or operable program.
- New tagged version 2.0.11 is not available on the npmjs.com server
- bluebird
- Bluebird causes js/incomplete-sanitization high security vulnerability
- How to auto-detect encoding of csv file?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from node-csvtojson.