calebsander/html-soup

A Node.JS package to do some basic HTML parsing and CSS selection

html-soup's Issues

Support `nth-child(x)`

Sadly, one of the sites has an element which I unfortunately need to target with #page-submission table table table tr:nth-child(2). As such, I'd appreciate support for nth-child as well :). I might have to dig in to fix it, though I'm not familiar with Typescript (had a bad experience, dug into the philosophy, realized it didn't give runtime checks, and moved on with life).

Support `tag[attr=foo]`

More of a "nice-to-have" - I'd written input[name="validate_token"] and it took me a few attempts to figure out why it wasn't matching. Now using [name=validate_token], but it might be simple to support :).

Use a set that gets passed to recursive calls of findMatches()

This will fix htmlSoup.select(htmlSoup.parse('<a/><a/><b/>'), ':root ~ :last-child') returning <b/> twice.

Support direct attribute access, innerText, and dataset properties

I used something like this to get the attributes I needed (getInnerText redacted as it's currently returning wrong values):

function proxifyNode(node) {
	let wrappedChildren = false;
	return new Proxy(node, {
		get: function(target, property, receiver) {
			// children must be wrapped!
			if (property === 'children' && !wrappedChildren) {
				wrappedChildren = true;
				target.children = target.children.map(child => {
					if (child.constructor.name === 'HtmlTag')
						return proxifyNode(child);
					return child;
				});
			}

			// prefer normal properties first
			if (property in target)
				return target[property];

			if (property === 'innerText')
				return getInnerText(target);

			if (property === 'dataset') {
				const dataset = new Proxy(target, {
					get: function(target, property, receiver) {
						return target.attributes['data-' + property.replace(/([A-Z])/g, (match, p1) => '-' + p1.toLowerCase())];
					}
				});
				// set as property so I only build the dataset Proxy once
				target[property] = dataset;
				return dataset;
			}

			// check for attributes after the special properties
			if (property in target.attributes)
				return target.attributes[property];
		},
	});
}

And call it as:

const dom = proxifyNode(htmlSoup.parse(body));

Having support for this on the library level would be great. My usecase - I'm sharing scraping code between the server (node.js) and a browser extension - I've also wrapped htmlSoup.select to behave like document.querySelector and document.querySelectorAll as my scrapper code has been using those two functions.