unlight/nightmare-xpath

Nightmare plugin to select nodes by xpath expression.

nightmare-xpath

Nightmare plugin to select nodes by xpath expression.

INSTALL

This will install plugin nightmare-xpath for Nightmare 3.X:

npm install --save-dev nightmare-xpath

Note: If you are looking for plugin for Nightmare 1.X, then use:

npm install nightmare-xpath@1

USAGE

const Nightmare = require('nightmare');
require('nightmare-xpath')(Nightmare);

const links = await Nightmare()
    .goto('http://example.com/')
    .xpath('//a[@href]', function (node) {
        // We cannot return DOM element to nodejs,
        // we must return serializable object or primitive.
        // If function is omitted, node.toString() will be used.
        return node.href;
    })
    .then();

console.log(links); // ['https://www.iana.org/domains/example']

API

xpath(selector: string, handler?: (Node: node) => any): Array<any>

selector

Type: string

XPath expression.

handler

Type: function
Optional: Yes
Signature: (Node: node) => any

This function be called on each result of XPathResult. We cannot return DOM element to nodejs, we must return serializable object or primitive. If function is omitted, node.toString() will be used.

RESOURCES

• https://developer.mozilla.org/ru/docs/Web/API/Document/evaluate

nightmare-xpath's Issues

click() with xpath

Hi!

Is there any way I can .click() with xpath instead css selector?

xpath does not support promises

I am using Javascript ES6 and I use the promises quite a lot, they've been integrated to Nightmare and it would be nice that they are integrated here too.

Error: TypeError: self.goto(...).xpath(...).then is not a function

Suggested Change to how Nightmare prototype is changed?

Hey I love this package and that I can select off the xpath of an element. However, I happen to use a lot of different modules that modify the Nightmare instance that I use to scrape, and it's hard to remember how all the different packages change the Nightmare class. instead of doing the following:

var Nightmare = require("nightmare");
// all the function definitions
Nightmare.prototype.xpath = function(selector, handler) {}

Maybe it could be changed to the following:

module.exports = exports = function(Nightmare) {
  // all the function definitions
  Nightmare.prototype.xpath = function(selector, handler) {}
}

Without the change, my require statements look like the following:

/** Require in all my node modules and npm modules **/
// Most of the work I do for my clients is scraping portals, downloading files, and other sundry task automation that they need but don't want to do so I use packages like csvtojson, url parsers, path, promisify, etc. so the require statements are usually long

/** Build the Nightmare Instance that I want to Use **/
let Nightmare = require('Nightmare');
Nightmare = require('some-other-nightmare-pkg')(Nightmare);
require('nightmare-inline-download')(Nightmare);
require('nightmare-xpath');

It is frustrating to have to go and research how each particular module modifies Nightmare each time I am going to write a new scrape. If nightmare-xpath included the change above though then my require statements could be simplified a lot:

/** Require in my node and npm modules **/

/** Build the custom Nightmare for this particular scrape **/
const Nightmare = require('Nightmare');
require('some-other-nightmare-pkg')(Nightmare);
require('nightmare-inline-download')(Nightmare);
require('nightmare-xpath')(Nightmare);

// do scraping and automation task

Just a thought. Would you like me to put it in a pull request??

this._evaluate is not a function

Hello,

Came across your library while looking for a way to use XPATH with nightmarejs. Got the above error when trying to use nightmare-xpath 2.0.1 with nightmare 2.8.1. Looks to be a missing method on the nightmare prototype.

After some poking around in the nightmarejs code, I found a method called evaluate_now. Changed line 26 of nightmare-xpath's index.js from '_evaluate' to 'evaluate_now' on my local and everything's working again.

Don't know if nightmarejs changed - but this may help someone else. Let me know if you want to go this way and I can submit a pull request after re-running tests

Thanks!

Latest version of nightmare.js for ref:
https://github.com/segmentio/nightmare/blob/master/lib/nightmare.js