Custom html entity defs

[rspeer]

Custom html entity defs

[rspeer]

Sorry for letting this suggestion lapse.

I think it would be argument bloat to make the entities customizable, but on the other hand, my design goal is to make the right thing happen as often as possible, so I could just code in some extra entities. Is this the entire list of sometimes-accepted but non-standard HTML entities, or are there more?

[chbrown]

That's not the entire list by any means, just the entity-looking things that showed up in my particular messily-encoded file. :)

I don't know if there's a good standard list somewhere, since those characters are outside the W3 spec, but here's some relevant Chromium source code and some from WebKit (using "Sacute" as a relatively unambiguous search term). Or maybe it'd be easier to incorporate the already-Python constants from html5lib?

That full html5lib list might be overboard, though? There are quite a few characters in there!

In [1]: import html5lib; len(html5lib.constants.entities)
Out[1]: 2231

[rspeer]

A desirable outcome for me is a list of entities that I don't have to curate myself, and html5lib looks like a good choice there. If it's important to decode these entities in an HTML5 parser, it's probably also important in ftfy.

(The best thing would be to replace the whole function with html.unescape from the standard library, but that's a Python 3.4+ thing, and, y'know, long before I drop support for Python 3.3, I have to drop support for 2.6 and 2.7 and see how mad people get.)

[chbrown]

html5lib is a pretty sizable project to depend on, but its dependencies, six and webencodings, seem reasonable. I agree with your judgment here and the solution in c960156.