nasa8x/html-metadata-parser

Html metadata scraper and parser for Node.js

Html Metadata Parser

Html Metadata scraper and parser for Node.js

Install

npm install html-metadata-parser

yarn install html-metadata-parser

const { parser } = require('html-metadata-parser');

parser('https://www.youtube.com/watch?v=eSzNNYk7nVU').then(result=>{
   console.log(JSON.stringify(result, null, 3));
})

// async 
const { parser } = require('html-metadata-parser');
(async () => {
    var result = await parse('https://www.youtube.com/watch?v=eSzNNYk7nVU');
    console.log(JSON.stringify(result, null, 3));
})();

// result

{
   "og": {
      "site_name": "YouTube",
      "url": "https://www.youtube.com/watch?v=eSzNNYk7nVU",
      "title": "Rebuilding iOS 15 with Tailwind CSS",
      "image": "https://i.ytimg.com/vi/eSzNNYk7nVU/maxresdefault.jpg",
      "description": "In this video, I'll show you how to rebuild the new Notification Summary UI from iOS 15 using Tailwind CSS.Source code: https://play.tailwindcss.com/kY4LYXwsNZ",
      "type": "video.other"
   },
   "meta": {
      "title": "Rebuilding iOS 15 with Tailwind CSS",
      "description": "In this video, I'll show you how to rebuild the new Notification Summary UI from iOS 15 using Tailwind CSS.Source code: https://play.tailwindcss.com/kY4LYXwsNZ"
   },
   "images": []
}

☞ 15 Best Visual Studio Code Themes of 2021

☞ What is the difference between PM2 Cluster vs Fork Mode

☞ Lightbox Photo Grid and Slideshow component for Vue.JS?

Contributing

Please refer to each project's style and contribution guidelines for submitting patches and additions. In general, we follow the "fork-and-pull" Git workflow.

1. Fork the repo on GitHub
2. Clone the project to your own machine
3. Commit changes to your own branch
4. Push your work back up to your fork
5. Submit a Pull request so that we can review your changes

NOTE: Be sure to merge the latest from "upstream" before making a pull request!

Community

Stay up to date on the development of Morioh UI and reach out to the community with these helpful resources.

Follow @codek_tv and @im_a_developer on Twitter.

Follow Morioh and Javascript Developers on FaceBook.

Join the official Discord room: https://discord.gg/sqxU6un.

html-metadata-parser's Issues

Error when fetching meta data

I am building a service that requires me to fetch metadata from any given URL.
This isn't at the center of what I am working on, it's an addon feature and after some Google search I found this package (cool stuff)

When working with most URLs and fetching data everything seems to work fine.
But whenever I try to retrieve metadata from http://web.com, it throws an error.

Initially thought it was an issue from my setup, but I tried it on Runkit and the same issue occurs.
Is there a Good reason for this?

Attached is a link to what happens
https://scrnli.com/uqKqh195WJa3b8

Also attached is a link to the expected output
https://scrnli.com/WFfk2P1WPjdA7v

TypeScript: metadata.images incorrectly reported as string[]

When using html-metadata-parser in VS Code I get Metadata.images?: string[] | undefined whereas Metadata.images really is an Array of { src: String } objects.

I tried to look in https://github.com/nasa8x/html-metadata-parser/blob/master/src/index.ts but couldn't find the error.

My workaround:

interface MetadataImage {
  src: string
}

const metadata = await htmlMetadataParser(url)
const metadataImages = metadata.images as unknown as MetadataImage[]

Parsing existing HTML?

Are you open to PRs? If so, I'd like to add the utility to parse existing HTML returned other HTTP libraries to get meta tag information

const Meta = require('html-metadata-parser');

Meta.parseHtml(`<meta name="author" content="John Doe">`, (err, res) => {
    // handle r es
})

Add data-src support and fix url regex check

Some IMG tags only have data-src attributes when axios returns the HTML data. It will be nice to also have data-src support.

Also, the regex check for URL returns false for https://c. and it is supposed to be true.

Looking to create a PR

Support for Twitter

I was trying to parse a link but I only get a response of the link that was passed.

parser(link).then((result: unknown) => {
console.log("RESULT: ", result);
})

"RESULT": { "meta": { "url": "https://twitter.com/foo/status/bar" }, "og": { "site_name": "Twitter" }, "images": [] }

convert to a json file

Hi, what part should i change if i want to convert it to a json file?
I want to host it on Heroku and call it like.
http://myscrapperapp.heroky.com?url=http://websitetoscrap.com/artile2.html
And returning a json respond file.