Companion crawl server for the ScrappyDoo Chrome Extension. Given a URL and a set of CSS selectors, the server goes through the markup, parses for and returns the values found at those selectors.
- Node.js (>= 6.9.0)
- yarn (
npm i yarn --global
)
$ yarn install
TO start up the server (on localhost
and default port 6969
), use the command -
$ npm start
{
"Content-Type": "application/json"
}
Type: String
(Page URL)
Required: true
Fully-qualified URL to crawl.
Type: Array
(Selector information)
Required: true
Array of selectors.
Type: String
(Unique name for the selector)
Required: true
Unique name for the selector. Will be used in the response to identify selector in results.
Type: String
(CSS selector to select from page)
Required: true
CSS selector for element to select.
Type: String
(HTML attribute to pick from selected element)
Required: true
HTML attribute to pick from the selected element.
{
"url": "https://www.reddit.com",
"data": [
{
"name": "header_logo",
"selector": "#header-img",
"attribute": "href"
},
{
"name": "sidebar_donate_link",
"selector": "html>body>div:eq(2)>div:eq(7)>div>div>a",
"attribute": "href"
}
]
}
{
"header_logo": "/",
"sidebar_donate_link": "/gold?goldtype=code&source=progressbar"
}
WTFPL © GP