Cant read UIC 918-9 Deutschland Ticket

[lewinpauli]

Cant read UIC 918-9 Deutschland Ticket

[user4223]

i will check the muster tickets you provided later on.
but i've checked my personal Deutschland ticket in a quick debugging session with ticket-analyzer by placing it in front of the camera. and I've seen the U_FLEX record of my ticket is of version "03", but my current implementation supports "13" only. so I guess when the problem is not related to failing aztec decoding (this can be solved sometimes by rotating the image slightly, you can try easily in ticket-analyzer and pressing "r" or "R" when your tickets are available in image path), the issue is most probably because of an currently unsupported U_FLEX version (RecordU_FLEX.cpp).

[lewinpauli]

thank you for your quick response

the ticket-analyzer crashes when it can read the "Deutschland Ticket" qr code (looks like U_FLEX 03 is the problem as you mentioned)

I zoomed out via 0 "reset" with the following output:

[2024-04-06T18:19:49 INFO Loader] Loaded 2 image(s) from directory asynchronously: "images"
[2024-04-06T18:19:51 INFO KeyMapper] Handling key '0': reset:
terminate called after throwing an instance of 'std::runtime_error'
what(): Unsupported header: U_FLEX, 03
Aborted (core dumped)

And I used 4 a couple of times to see the full qrcode and rotated the image but when the full qrcode is visible the program crashes:

[2024-04-06T18:34:10 INFO Loader] Loaded 1 image(s) from directory asynchronously: "images"
[2024-04-06T18:34:14 INFO KeyMapper] Handling key '4': split 4: 4/3
[2024-04-06T18:34:14 INFO KeyMapper] Handling key '4': split 4: 4/4
[2024-04-06T18:34:15 INFO KeyMapper] Handling key '4': split 4: 2/0
[2024-04-06T18:34:17 INFO KeyMapper] Handling key 'r': rotate: 5
[2024-04-06T18:34:18 INFO KeyMapper] Handling key 'r': rotate: 6
[2024-04-06T18:34:19 INFO KeyMapper] Handling key 'R': ROTATE: 5
[2024-04-06T18:34:19 INFO KeyMapper] Handling key 'R': ROTATE: 4
[2024-04-06T18:34:20 INFO KeyMapper] Handling key 'R': ROTATE: 3
[2024-04-06T18:34:20 INFO KeyMapper] Handling key 'R': ROTATE: 2
[2024-04-06T18:34:20 INFO KeyMapper] Handling key 'R': ROTATE: 1
[2024-04-06T18:34:21 INFO KeyMapper] Handling key 'R': ROTATE: 0
terminate called after throwing an instance of 'std::runtime_error'
what(): Unsupported header: U_FLEX, 03
Aborted (core dumped)

[lewinpauli]

I now checked all 22 Muster files with ticket-decoder from https://www.bahn.de/angebot/regio/barcode and

only the 2 "Deutschland Tickets" and additionally the "Muster 918-9 BahnCard 25.png"

only output {}

the ticket-analyzer has the following output for the "Muster 918-9 BahnCard 25.png" file:

./build/Release/bin/ticket-analyzer
[2024-04-06T19:51:43 INFO Loader] Loaded 1 image(s) from directory asynchronously: "images"
[2024-04-06T19:51:45 INFO KeyMapper] Handling key '0': reset:
[2024-04-06T19:51:45 WARNING RecordU_FLEX] Unimplemented transport document data type: 6
[2024-04-06T19:51:46 WARNING RecordU_FLEX] Unimplemented transport document data type: 6
[2024-04-06T19:51:46 WARNING RecordU_FLEX] Unimplemented transport document data type: 6

looks to me like U_Flex 6 could also be needed

but BahnCard 25 is probably a different format anyway

[user4223]

i've checked the tickets crashing with unsupported U_FLEX record or unimplemented transport document data type. I think there are some overlapping issues right now.

1. default rotation is set to 4. so each image is rotated 4 degree ccw by default. but when the border around the aztec code is that small as it is for the mobile phone ticket screenshots (Deutschland ticket and Deutschland job ticket), aztec decoding fails because of larger parts of the image are outside the visible area.
2. default split is set to 2 of 4. so each image is split into 4 parts and the second (upper right) is chosen to find the aztec code. this is faster and works for most paper tickets but not for the screenshots, they would work better with 1 of 2 but also with 1 of 1, I guess.
3. when "0" is pressed, the reset leads to rotation 0 and split 1 of 1. this brings the aztec code decoding to work, so zxing-cpp is able to work properly. but....
4. in turn, the UIC decoding throws an exception due to unsupported U_FLEX version. I think, one of the versions listed below could do the job. at the moment, I'm not sure if "03" means "3.0.0" or something different. i hope I have some time in the next days to read the docs and to try some stuff related to this: https://github.com/UnionInternationalCheminsdeFer/UIC-barcode/tree/master/misc
5. unfortunately, the exceptions are not caught and properly logged in decoder and analyzer-loop as well to keep the app running on error. until now it was not that important and I just wanted to fail fast to see unexpected behavior directly. but now, I think I should add a proper catch and should log errors/warning instead of crashing with unhandled exception.

anyway, I tried also just to add "03" to supported versions in RecordU_FLEX.cpp in ctor to see what happens. there are follow-up errors but the app keeps running at least and the other records are decoded. so when it is enough right now, this could be a hacky hot-fix.

[lewinpauli]

Thank you again for you time, effort and your quick response

I tried the current "13-cant-read-uic-918-9-deutschland-ticket" branch with the following result:

I can only compile the branch "13-cant-read-uic-918-9-deutschland-ticket" with conan version 1.64.0 because of some wayland dependency

when I compile 8 tests are failing:

[==========] 140 tests from 43 test suites ran. (1291 ms total)
[  PASSED  ] 125 tests.
[  SKIPPED ] 7 tests, listed below:
[  SKIPPED ] EUR9_Ticket.Metadata
[  SKIPPED ] EUR9_Ticket.Record_U_HEAD
[  SKIPPED ] EUR9_Ticket.Record_0080BL
[  SKIPPED ] EUR9_Ticket.Record_U_TLAY
[  SKIPPED ] EUR9_Ticket.Record_0080VU
[  SKIPPED ] BVG_4Fahrtenkarte.Metadata
[  SKIPPED ] Unknown_Ticket1.Metadata
[  FAILED  ] 8 tests, listed below:
[  FAILED  ] Base64EncodedRawInResult.Metadata
[  FAILED  ] UIC918_9_Laenderticket_Rheinland_Pfalz.Metadata
[  FAILED  ] UIC918_9_Laenderticket_Sachsen_Anhalt.Metadata
[  FAILED  ] UIC918_9_Laenderticket_Sachsen_Anhalt.Record_U_HEAD
[  FAILED  ] UIC918_9_Laenderticket_Sachsen_Anhalt.Record_U_TLAY
[  FAILED  ] UIC918_9_Laenderticket_Sachsen_Anhalt.Record_U_FLEX
[  FAILED  ] UIC918_9_Laenderticket_Sachsen_Anhalt.Record_0080VU
[  FAILED  ] UIC918_9_Laenderticket_Schleswig_Holstein.decode

 8 FAILED TESTS

the 2 deutschland muster tickets still output {} but the other muster tickets are working

and when I want to run the
the ticket-decoder with a bash script it outputs:

Fontconfig error: Cannot load default config file: No such file: (null)

[user4223]

some note related to building the applications, maybe some are not relevant for you, so just skip if so:

• please be careful to not confuse debug and release builds. ideally, just build the release config via "./setup.Release.sh -j" and build debug versions only in case you really need it for debugging
• in my experience, when conan itself is updated or some library updates are done, it might be good to clean all dependencies properly by using just "rm -rf ~/.conan/data build/" before and start from scratch
• I guess you are building on linux: when opencv gets build from source because it cannot be loaded in a matching configuration from conan-center, it requires some dev-dependencies for highgui for visualization in ticket-analyzer. you can configure conan to install those dependencies automatically with "conf.tools.system.package_manager:mode=install" and
  "conf.tools.system.package_manager:sudo_askpass=True" or you install the required dev-packages in advance. see .github/workflows/c-cpp.yml for a list of required packages.
• to avoid all those packages on your host system, you can create the build-environment with all dependencies inside a container. executables should be usable on host-system as well after build. just start docker and execute setup.ubuntuJammy.gcc11.Release.sh
• I tried building head of this branch on macos sonoma (intel and amd64 silicon) and in ubuntu docker containers with gcc11 and clang15 and on real ubuntu jammy with native gcc11, all builds worked for me from scratch with conan 1.64.0.
• when there are still build-problems, please give more more information about your os, architecture, compiler and explicit error messages

notes related to testing:

• please ensure you downloaded the key file via "wget 'https://railpublickey.uic.org/download.php' -O cert/UIC_PublicKeys.xml" before execution since some tests are checking if the signature-verfication is working and they expect the file in "cert/UIC_PublicKeys.xml"
• please ensure you execute the tests from workspace root path to enable related tests to load the key file properly, otherwise they are failing when they check the verify flag
• when they still fail, please provide detailed information about the failing expectation

notes on ticket decoding:

• did you try the following as well? "./build/Release/bin/ticket-decoder --rotate-image 0 --split-image 11 -i images/Muster\ 918-9\ Deutschland-Ticket.png"
• when it still fails, please provide the log-messages when you add the -v flag to the command above but be careful to not add sensitive information here

notes on fontfonfig:

• fontconfig is a dependency from libpoppler used for rendering of pdf documents, usually you can ignore this error, it might lead to missing text due to missing fonts on internally rendered images. in my experience, aztec code decoding worked anyway for my test cases