Comments (9)
Just to check, I upgraded node to v6 and I see still the issue. Example variable output below:
[root@git: /node/classifier] $ nodejs --version
v6.3.1
[root@git: /node/classifier] $ nodejs duplicates.js
19
[root@git: /node/classifier] $ nodejs duplicates.js
18
from feedparser-promised.
Hi @dsl101
Thank you for bring this issue. I was able to reproduce this weird behaviour as well.
I will take a look on this and I should give you a feedback shortly.
Best!
from feedparser-promised.
Hey @dsl101
At the moment feedparser-promised
can't handle request
object directly. Since there is no type checking javascript will fire the request twice, adding duplicated responses during the parsing.
Please, try to use this code below and see if it works for now:
var FeedParser = require('feedparser-promised');
FeedParser.parse('http://www.europlanet-eu.org/feed/').then(function (items) {
console.log(items.length);
});
In the meanwhile I'll add the request
object support (or an object that responds to uri
and timeout
), sounds good?
Best
from feedparser-promised.
Ah - that's good news. Yes, I need to be able to specify the timeout which is why I switched to a request object. I thought I'd seen that in the docs somewhere, but maybe I just imagined it :). I will test without the timeout on Friday.
from feedparser-promised.
Since you need to specify the timeout, I was wondering of having an API like this:
var FeedParser = require('feedparser-promised');
var options = { timeout: 3000 };
FeedParser.parse('http://www.europlanet-eu.org/feed/', options).then(function (items) {
console.log(items.length);
});
What do you think? I would like to avoid an explicitly dependency of request
package, but I would like to know your thoughts on that.
Best
from feedparser-promised.
My only reason for putting it in the request object was in the admittedly
edge case of wanting different options for different URLs. It seemed to
make sense to keep the URL and its options together in the same object. But
I'd much rather have what you put below then the duplicates :)
On Thursday, 28 July 2016, Alabê Duarte [email protected] wrote:
@dsl101 https://github.com/dsl101
Since you need to specify the timeout, I was wondering of having an API
like this:var FeedParser = require('feedparser-promised');
var options = { timeout: 3000 };
FeedParser.parse('http://www.europlanet-eu.org/feed/', options).then(function (items) {
console.log(items.length);
});What do you think? I would like to avoid an explicitly dependency of
request package, but I would like to know your thoughts on that.Best
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#4 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA29vvlu73QgvvNtytfrBdPf8c6Ep-xgks5qaCpFgaJpZM4JV-kv
.
from feedparser-promised.
Just added some extra unit tests but it's also possible to use request options right away, like:
var FeedParser = require('./lib/feedParserPromised');
const tryout = (options) => {
FeedParser.parse(options).then( (items) => {
console.log(items.length);
const allTitles = items.map( (item) => { return item.title; });
console.log(allTitles);
}).catch( (err) => {
console.log('feedParserPromised error caught: ', err);
});
};
const uri = 'http://www.europlanet-eu.org/feed/'
tryout({ uri: uri, timeout: 1 });
/*
expected output:
feedParserPromised error caught: { [Error: ETIMEDOUT] code: 'ETIMEDOUT', connect: true }
*/
tryout({ uri: uri, timeout: 3000 });
/*
expected output:
10
[ 'ESOF 2016 – What do you think a comet smells like?',
'Juno public event: Tuesday, 5 July 2016, Athens',
'Liquid water in Ceres’s past',
'Counting Down to Jupiter',
'Mission Juno : University of Liège goes into orbit at Jupiter',
'European involvement in the Juno mission',
'Scientists come to Schloss Seggau to discuss Rosetta’s comet',
'Coming soon: Jupiter and its Icy Moons',
'DPS-EPSC Joint Meeting, 16-21 October 2016',
'Jupiter blasted by 6.5 fireball impacts per year on average' ]
*/
tryout({ uri: uri });
/*
expected output:
10
[ 'ESOF 2016 – What do you think a comet smells like?',
'Juno public event: Tuesday, 5 July 2016, Athens',
'Liquid water in Ceres’s past',
'Counting Down to Jupiter',
'Mission Juno : University of Liège goes into orbit at Jupiter',
'European involvement in the Juno mission',
'Scientists come to Schloss Seggau to discuss Rosetta’s comet',
'Coming soon: Jupiter and its Icy Moons',
'DPS-EPSC Joint Meeting, 16-21 October 2016',
'Jupiter blasted by 6.5 fireball impacts per year on average' ]
*/
tryout(uri);
/*
expected output:
10
[ 'ESOF 2016 – What do you think a comet smells like?',
'Juno public event: Tuesday, 5 July 2016, Athens',
'Liquid water in Ceres’s past',
'Counting Down to Jupiter',
'Mission Juno : University of Liège goes into orbit at Jupiter',
'European involvement in the Juno mission',
'Scientists come to Schloss Seggau to discuss Rosetta’s comet',
'Coming soon: Jupiter and its Icy Moons',
'DPS-EPSC Joint Meeting, 16-21 October 2016',
'Jupiter blasted by 6.5 fireball impacts per year on average' ]
*/
Commit changes: 553be3a
from feedparser-promised.
So, just because I'm curious to understand... It looks like I should have just passed the object directly to feedparser-promised all along, rather than creating the request object myself - is that right? Event though the parameter was named uri
, it would have worked? Or did I miss some other changes you made?
from feedparser-promised.
Yeah, you right. You should have just passed the object directly (without calling request
function). The only change that I made was on the tests. I'll update the README as well with that.
from feedparser-promised.
Related Issues (18)
- Errors aren't handled in the `catch` HOT 2
- Can't load feedparser-promised - broken package? HOT 5
- Recent update broke passing options to underlying feedparser HOT 7
- Don't include Babel as dependency
- "use strict"; HOT 3
- encoding problems HOT 7
- Parsing XML attributes HOT 3
- App crashes in IE 11 HOT 6
- Parser crashes on this valid RSS feed HOT 3
- The ES6 class syntax makes this unusable in ES5 projects HOT 1
- Update the @types/request = 0.0.45 HOT 4
- Issue with parsing some feeds HOT 2
- Unicode issue with Chinese Language. HOT 4
- How to avoid import stopping on error? HOT 2
- Getting ETIMEDOUT on Amazon AWS Lambda Node 8.10 HOT 4
- Is it possible to remove unwanted characters? HOT 3
- Timeout Error Error: ESOCKETTIMEDOUT parsing https://www.nasdaq.com/feed/rssoutbound?category=Stocks HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from feedparser-promised.