unicode support

[farvardin]

unicode support

[JohnEarnest]

This is an intentional limitation.

Unicode, in its entirety, is staggering in scope. While supporting some subset of Unicode, someday, is not impossible, it would introduce a daunting amount of complexity. For pixel-perfect cross-platform results, Decker uses custom text layout routines and its own collection of bitmapped fonts, which would both need to be extended to include new glyphs for most non-English languages, and I'd also need to adapt the touch keyboard to permit entering those new glyphs.

Supporting UTF-8 in Lil itself isn't as bad, but it would introduce impedance mismatches between Lil and Decker. For example, it would no longer necessarily be the case that a Lil string or a Lil identifier is displayable in Decker. Lil can presently manipulate UTF-8 data as raw bytes using the "array" interface, but in many ways this is less convenient than working with ordinary strings.

For displaying text in Decker, some workarounds are possible; Decks can include custom fonts, and "rich text" fields can include inline bitmaps and reference different fonts for a given span, but copying and pasting "blended" text would reveal the underlying ASCII. There is some precedent for "normalizing" Unicode characters to ASCII equivalents on input (presently Decker "straightens" curly-quote characters, for example), but I doubt that applying this approach to letters would make users of languages with diacritics very happy.

[farvardin]

What we don't want is Decker would become bloated, so I'm fine with this limitation!

[JohnEarnest]

No. Including UniFont would increase the baseline size of web-decker by multiple megabytes for a single font, and as previously stated I do not wish to drag in all the text layout and string manipulation complications of Unicode.

If future versions of Decker support non US-ASCII characters, I'm much more inclined to pursue an equivalent of MacRoman's extended code page and convert to and from UTF-8 at the periphery of the system; it would be possible to support French, German, Spanish, Portuguese, and a handful of other major western languages with relatively sparing changes to text layout and string representations.

[farvardin]

sure, I understand, it quite beyond the scope of Decker. We'll deal with it :)

[JohnEarnest]

Thanks for your understanding. I do apologize for the accessibility limitations of this constraint.

[easrng]

@JohnEarnest would you accept a PR for this? I think UniFont might be a good fit for decker.