hunternet93/oneshot-dialog-transcript

A datamined transcript of the dialog of the game OneShot

oneshot-dialog-transcript's Issues

Hey I know this is probably really old, but are some bits of text missing?

I know this is probably really old, but I'm not really seeing things like the part where Niko ask's about the name of your planet in this transcript. That and some others might be missing?

\\xE2 from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)

I've done the best I can to track down and attempt to fix the problem, but I'm not very well versed with hashes and working with JSON in ruby, so I'll just post my findings here.

This error is caused by line 73 at JSON.dump(output). The JSON library is trying to parse the hash, but is incorrectly assuming the encoding is ASCII-8, and complaining about finding non-ascii characters along the way. You can't really blame the JSON gem anyway, since the string encoding is set to ASCII-8BIT.
Now, the strings inside the hash are actually encoded in UTF-8, but are incorrectly marked to be ASCII-8. Forcing it manually doesn't appear to work as every level of hash forces their encoding to be ASCII-8.

Checking puts output[:maps][0][:events][0][:name].encoding outputs ASCII-8BIT but checking the exact same string at line 48 with puts e.name.encoding outputs UTF-8.

There are a couple of workarounds I have found and tried.

1. Swapping JSON for Oj
   Oj has a better json dump that actually doesn't throw an error when dumping the json. However, Oj spits out malformed JSON
   with the pesky \xE2 character anyway and running process.py makes that evident, as it immediately crashes and complains
   about the encoding. I know nothing about python, so I can't move on and fix the problem there. Anyway, it's not a fix and is
   more of just ignoring the problem outright. This is the solution that
   JoshCheek/seeing_is_believing#46 (comment) brings up.
2. Enabling "quirky mode"
   JoshCheek/seeing_is_believing#46 (comment) also brings this up, and it has the
   same issues as with Oj.
3. Writing to the file in binary mode
   This can be done simply by changing "w" to "wb+", but doesn't actually fix anything.
4. Calling .to_s on the strings
   This has no effect.
5. Setting the string encoding manually
   This has no effect. Setting the encoding to UTF-8 or ASCII-8 has no effect, my guess is it has to do with my lack of hash
   experience and me not figuring out how to set it for all the hash instead of individual elements. Solution from
   https://stackoverflow.com/questions/56737362/render-json-encodingundefinedconversionerror-xc3-from-ascii-8bit-to-utf-8
   and https://www.justinweiss.com/articles/3-steps-to-fix-encoding-problems-in-ruby/.
6. Iterating through hash elements and forcing encoding
   This actually stops the error, which is good. However it seems to output malformed JSON like with Oj. (Distinction from 5), in
   5) I set the string encoding before they get added into the hash.)

Most of the workarounds/solutions I have found either aren't applicable or just don't work. Note that this isn't actually a vanilla oneshot issue, but an issue with mods. (More specifically OSAM and OSFM.) It doesn't need to be fixed per se, but it is an actual issue nonetheless, even though it doesn't effect "intended" behavior/usage.

How does one create 'extracted.json'?

Would be useful to know as I'd like to make an up-to-date version of this.