Attribute values with UTF-8 characters crash

[jakeri]

Attribute values with UTF-8 characters crash

[cowtowncoder]

Quick question: is that a with umlauts above correctly encoded using UTF-8, as declared (although using non-standard 'utf-8'; should be recognized nonetheless)? Not sure how github issues handles encoding...

If possible, can you add snippet as attachment?

[jakeri]

Yes, it is an ä http://en.wikipedia.org/wiki/Swedish_alphabet (second last character in the swedish alphabet).
It is encoded as a double byte character in utf-8.

https://gist.github.com/2338902 (xml)
https://gist.github.com/2338944 (junit)

It feels like it has to do with double byte character split in the middle. I does not seems to handle it when it is in an attribute value.

I forgot the stack trace in prior post.

java.lang.IllegalStateException: Internal error: should never execute this code path
    at com.fasterxml.aalto.async.AsyncByteScanner.throwInternal(AsyncByteScanner.java:2936)
    at com.fasterxml.aalto.async.AsyncUtfScanner.handleAttrValuePending(AsyncUtfScanner.java:1350)
    at com.fasterxml.aalto.async.AsyncUtfScanner.handleAttrValue(AsyncUtfScanner.java:1136)
    at com.fasterxml.aalto.async.AsyncByteScanner.handleStartElement(AsyncByteScanner.java:2149)
    at com.fasterxml.aalto.async.AsyncByteScanner.nextFromTree(AsyncByteScanner.java:656)
    at com.fasterxml.aalto.stax.StreamReaderImpl.next(StreamReaderImpl.java:720)
    at net.jakeri.appstore.xml.AsyncXmlTest.testAsyncXml(AsyncXmlTest.java:39)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

[cowtowncoder]

Thanks. Yes, I am familiar with 'ä' as character, and know it gets split, just wanted to verify it was due to multi-byte UTF-8, and not as (incorrect) reaction to decoding problem. I'll have a look; you are probably correct in that this is one unhandled edge case. Async parses has to store state correctly and this must be missing currentyl.

[cowtowncoder]

Yup, can reproduce the issue.

[cowtowncoder]

Fixed -- handling was indeed missing; now should work. I will also push out 0.9.8 release, as there are couple of other bug fixes.

[jakeri]

Nice one!

Thanks.

[cowtowncoder]

Thank you for reporting this -- there are unfortunately some edge cases, but they are usually relatively easy to fix.