Auto charset conversión for getPlainText()?

[kukko]

Auto charset conversión for getPlainText()?

[GoogleCodeExporter]

I also noticed ganon has trouble handling GB2312 (Simplified Chinese). I ended 
up having to use iconv to convert to GBK before parsing, which is pretty slow 
for larger DOMs. Rules for charset conversions can be tricky.

Original comment by [email protected] on 18 Oct 2012 at 1:52

[GoogleCodeExporter]

I don't think it's a good idea to alter getPlainText, but maybe an extra method 
called getPlainTextUTF8? Perhaps might be better to just use a local solution, 
though.

Original comment by [email protected] on 19 Oct 2012 at 4:34

• Added labels: Priority-Low, Type-Enhancement
• Removed labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

The problem is... sometimes I dont know (or I dont want to know) in wich 
charset is the input webpage... so any kind of autodetection would be great so 
I can use my code always in the same charset

Original comment by [email protected] on 19 Oct 2012 at 4:39

[GoogleCodeExporter]

Added a simple version of getPlainTextUTF8 in rev #76.

Original comment by [email protected] on 20 Oct 2012 at 10:45

• Changed state: Done