Comments (2)
Seems like whatever the page is responding with isn't compressed properly. I'd love to know what exactly it is, other pages with the same headers work using zlib
. I can replicate the same behaviour using curl
and gunzip
on the CLI. E.g.
curl -H 'Accept-Encoding: gzip,deflate' https://github.com | gunzip -dc # works
curl -H 'Accept-Encoding: gzip, deflate' http://www.concordmonitor.com/on-new-hampshire-s-antique-alley-it-s-out-with-the-old-in-with-the-new-9234789 | gunzip -dc # errors with "unknown compression format"
Note: One page is gzip and the other is deflate. Not sure if there's a good page using deflate
available to test?
According to https://nodejs.org/api/zlib.html#zlib_compressing_http_requests_and_responses both types of responses should be handled by node.js properly and I haven't seen it before. Anyone else have ideas to fix it? I can try splitting out to the specific methods zlib methods used instead of the combined method. Someone else want to test that fixes it first though? I might not get around to it for a while.
from metascraper.
Closing that because 2.0.0 move HTTP calls out of the library, stay stunning!
from metascraper.
Related Issues (20)
- Is there a package for grabbing just the domain? HOT 1
- Wrong Typescript declared types HOT 5
- Not scrapping the correct date HOT 1
- JSDOM dependency is not compatible with node v14 HOT 7
- Missing 'logo' in Metadata interface when using metascraper-logo plugin HOT 1
- Instagram Scraping Give me null at some field HOT 1
- Crash on `metascraper-audio` HOT 4
- Incorrect metadata types HOT 4
- Specific pages lead to crash of nodejs HOT 1
- Specific pages crash NodeJS with the usage of metascraper-audio or metascraper-video HOT 3
- New version causes breaks in Definitely Typed packages HOT 9
- Cannot use in Defer because of re2 HOT 1
- Add types for metascraper packages
- metascraper-date typing errors HOT 2
- [metascraper-clearbit] Typescript build error from `keyvOpts` HOT 1
- Any issues with a memory leak, particularly metascraperDescription() HOT 1
- High memory usage HOT 9
- `author` is not recognized for YouTube HOT 1
- Make JSDOM opt-in HOT 4
- [metascraper-image] ability to pick desired image HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from metascraper.