n1k0 / readable-proxy Goto Github PK
View Code? Open in Web Editor NEWNode proxy server attempting to fetch readable contents from any provided URL.
Home Page: http://readable-proxy.herokuapp.com/
Node proxy server attempting to fetch readable contents from any provided URL.
Home Page: http://readable-proxy.herokuapp.com/
I was running into the same "Unable to access" errors other users have reported. I eventually realized that the 1000 ms timeout value in phantom-scrape.js was just too low for any page I was hitting and my internet connection.
I'd like to bump the timeout value to at least 5000 ms, and add a handler so the app returns a more useful message.
Follow-up to #15 (comments)
This would help with debugging during development.
For obvious performance reasons.
The URL is https://streams.spec.whatwg.org/
It fails at
readable-proxy/phantom-scrape.js
Line 83 in 4aa814e
Maybe Phantom can't cope with modern HTTPS setups?
I tried creating node js application on OpenShift and provided the link to this repo as source but it doesn't work. The application won't start. What can I do?
Admittedly I don't know too much about the differences in JSDOM and PhantomJS. But I see the app runs really slowly. So I made a version that uses JSDOM and it runs a lot faster and doesn't make my Macbook fan kick in. Mine doesn't support most of the options right now cause I just wanted to shoehorn in JSDOM to see if there was a performance improvement or not and there is. Here is the repo:
https://github.com/OKNoah/readable-proxy-jsdom
Is there some reason this uses PhantomJS and not JSDOM?
stdout maxBuffer exceeded
Sorry to bother you,
but I have no idea if that is coming form the proxy or the Readable.js
lib,
or if I can raise that buffer limit somehow.
The website I tried: http://www.equestriadaily.com/2016/10/music-intersekt-twilight-says-bass-house.html
What can I do?
And the server should be responsible of constructing a JSON error response out of it.
This would be useful for testing how things will work on Android vs. desktop Firefox.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.