retroachievements/rcheevos

Library to parse and evaluate achievements and leaderboards for RetroAchievements

rcheevos's Issues

running other programs on achievement trigger.

Would it be possible to implement a way to trigger some other program/function whenever an achievement is earned?

I built an arcade cabinet with WS2812 LEDs and it would be cool to make them run an animation whenever an achievement triggers

Credentials invalid (0)

I'm trying to get achievements working in duckstation. I can log in successfully but anything that uses the login token fails.

fatalis@shiro ~ $ curl -X POST 'https://retroachievements.org/dorequest.php?r=login&u=fatalis12&p=CENSORED_PASSWORD'
{"Success":true,"User":"fatalis12","Token":"CENSORED_TOKEN","Score":0,"Messages":0}%                                                                                              

fatalis@shiro ~ $ curl -X POST 'https://retroachievements.org/dorequest.php?r=patch&u=fatalis12&t=CENSORED_TOKEN&g=11283'
{"Success":false,"Error":"Credentials invalid (0)"}%                                                                                                                                                                                                                                                  

fatalis@shiro ~ $ curl -X POST 'https://retroachievements.org/dorequest.php?r=unlocks&u=fatalis12&t=CENSORED_TOKEN&g=11283&h=0'
{"Success":false,"Error":"Credentials invalid (0)"}%    

PSP homebrew hashing

On the PSP, homebrew games (which often are ports/variations of games for other platforms) are quite popular and it seems people are quite excited to make achievements for them. Hashing them is a real problem though.

Homebrew games are extracted to a directory under /PSP/GAME on the memory stick, like say /PSP/GAME/DOOM. They contain an executable container called EBOOT.PBP which contains the game binary, its PARAM.SFO and icons and stuff, so effectively simply hashing this file might seem to be enough. Unfortunately homebrew games have more files than that which may be easiliy modified, and some are simply an engine running different data files, like the luaengine, so we really have to hash other files as well. But homebrew also often writes their own configuration and save data to the same directory, so simply hashing all files won't work reliably.

It might be necessary to do something like hash the eboot, then look up that hash in an official table to figure out what other files to hash as well to make a solid identification. I realize there's probably no infrastructure for that in retroachievements at this time, and it would be real tricky to maintain.

Of course, it's also a complication that there's often no "canonical" version of a particular homebrew game or engine, so achievement makers will just have to pick one.

Anyway, I'm not sure how practical it is, but I think documenting the problem here is at least a start.

Address core memory swizzling (somehow)

Sega Saturn has a particular issue where memory appears to be byteswapped, and pointers are 4-bytes large, resulting in difficulties for creating achievements.

This is rooted in a core issue of libretro memory handling, which cheevo sets ends up expecting. libretro will just obtain pointers to core memory. Memory will proceed to be read, byte by byte, to construct memory values.

In theory, this all should work ok. In practice, some cores utilize a trick known as swizzling. This is so an emulated system with an endianess opposite of the host system can directly do common read/writes (assuming the width for common read/writes is greater than 1 byte) under the host system's endianess. When it comes time to do a 1 byte read, the address will be xored with a constant before being used to index a memory buffer (for 2 byte swizzling, this constant is 1, for 4 byte swizzling, this constant is 3).

This knowledge is even worse when you consider this is only going to possibly occur when the host system's endianess is opposite of the emulate system's endianess. If you change the host system so something which matches the emulated system's endianess, you will now read memory "correctly" as it's no longer being swizzled. Of course, this ends up resulting in a mismatch compared to the swizzled memory, and sets would only be made one way or the other.

In practice of course, this trick is limited to emulated big endian systems, as most host systems will be little endian. Not all emulated big endian systems will utilize this trick anyways, some like the Jaguar and GameCube/Wii have big endian memory, but emulators for those systems do not swizzle memory.

The selection of consoles which this trick actually occurs is also very limited. Sega Mega Drive / CD / 32X all utilize this (and of course those are more or less all under the same emulators), Sega Saturn also utilizes this. N64 utilizes this, but instead of simple byteswapping, it does the swizzling by 4 bytes (so ^ 3 instead of ^ 1 for 1 byte reads). (There could be some other systems I'm missing here)

Also in practice, the issues of a big endian host system typically won't actually occur. This will mean the user is running RetroArch on a Wii, Wii U, or old PowerPC Mac, and achievement hunting, and probably doing so with the Genesis Plus GX core. A combination of factors so unlikely it apparently has never actually resulted in some ticket being filed, as I'm told.

As far as I can tell, the libretro API does not offer any way to know if memory is being swizzled or not. It seems to have some kind of flags for retro_memory_map's, so perhaps it'd need a flag added?

Even if this is "fixed" you may still end up needing to fix a ton of existing sets, which all rely on the existing memory behavior, and users will be forced to update their cores / clients, which would create a ton of updating pain.

BizHawk actually normally fixes swizzling for its memory APIs, but for cheevo purposes it has to swizzle it back, due to the mentioned existing sets relying on memory swizzling.

Add restriction agaisnt Debug Dips (FB Neo)

With the newest FB Neo version, it is possible to enable Debug Dips 1_1 to 2_8 inside Neo-Geo games without the need of using UNIBIOS. A restriction is needed to avoid cheats, since some dips includes invincibility.

OBS: On a unrelated note, a few hours after posting this issue I've also found that "Development Kit" might need to be restricted from the BIOS option, just like it is other Universal Bios. I was able to enter the debug menu after changing to it for both KOF 2002 (hold P2 Pause (L3) and press the P2 C button (X button) on main menu) and Metal Slug 3 (press Pause (L3) 2 times in-game), but that wasn't the case on "AES Asia" or others.

KOF 2002 Debug Menu:

Add memory map for C64 and rom hashing

The c64 is avaliable to capture memory and allow rom hashing of entire roms.
https://www.c64-wiki.com/wiki/Memory_Map

Achievements never load in RetroArch Android AARCH64, close content with achievements enabled causes it to get stuck.

It seems the new implementation has different issues on AARCH64.
The achievement list never finishes loading (the logging message never pops-up) and this causes RA to stop responding on close content.

To verify I disabled achievements entirely and close content then works just fine.
This was tested on a Shield Android TV, running RetroArch AARCH64 libretro/RetroArch@77fba6d

No logs on RetroArch

Seems there is absolutely zero logging with the new implementation?
Makes it hard to tell when things are not working correctly.

Tested on windows, commit libretro/RetroArch@ead5b2c

Continue parsing Rich Presence LUTs after encountering commented lines

This is a request for a behavioral change.

When defining large look-up tables, I sometimes want to document gaps, or sets of values, as such:

Lookup:CharacterTable
// Uppercase letters
0x00=A
0x01=B
(...)
0x19=Z
// Gap -> lowercase letters
0x20=a
(...)

Currently, this is not feasible without workarounds, because parsing of a LUT is aborted at the first line with no defined element, or perhaps because the comments are stripped before being parsed and the parser sees only a blank line, and thus the end of the LUT.

Since the syntax is relatively limited there is no perfect solution that I can think of, but I would suggest continuing parsing tables until their scope is ended explicitly (i.e. until the next Lookup, Format, or Display keyword), and/or until a non-blank, non-comment line that cannot be parsed into a table entry is reached.

Add DTCM to DS/DSi Memory Maps

https://problemkaputt.de/gbatek.htm#dsmemorycontrolcacheandtcm
https://problemkaputt.de/gbatek.htm#armcp15tightlycoupledmemorytcm

TCM is high speed memory on the ARM9 CPU itself. There are two types of TCM, one meant for Instructions (ITCM) and one meant for Data (DTCM). ITCM is probably unlikely to be useful for achievements, but DTCM seems to be useful. Okamiden for the DS appears to store all important savefile flags in this region rather than in Main RAM.

There is a question however where exactly should it be put. Should it be placed right after the Main RAM for each console (which would result in a odd memory map when comparing the two, and could be potentially problematic if you want to support running DSi enhanced games in either DS or DSi mode), or should it be placed after what would be the Main RAM for DSi, and leave a 12MiB hole for DS?

Add Scratchpad memory to PS1

Not sure if this is the correct place to request this, but here goes:

The Scratchpad is a cache-like memory region that is 1KB in size and starts at 1F800000 in the console memory. It is used occasionally for certain speed optimizations and other things.
See this link for more details:
https://psx-spx.consoledev.net/memorymap/

Currently, no libretro core, nor DuckStation's RA components can access this region, they can only access the 2048KB of main memory (console memory 00000000-001FFFFF). I propose that this region be mapped in RA memory right after the main memory, at 0x200000-0x2003FF.

Here's an example of a game that actively uses this region:
https://retroachievements.org/game/5103
This has values noted in the code notes for crucial things like lives, E-tanks, progression flags and the like, and all of them cannot be accessed in any libretro core. I found them by digging in Duckstation's own memory finder, which can search this region with no issues. Same applies to the PS1 ports of all 6 OG Mega Man games, and most likely some others - if you ever search for cheatcodes and find one in a format like "1F800XXX XXXXXXXX", that means it's manipulating this memory region.

Also, it'll be a massive help if anyone ever decides to dev these two, I did some memory digging in advance and let's just say this region does a lot of heavy lifting when in gameplay.
https://retroachievements.org/game/8295
https://retroachievements.org/game/3784

Slow login on RetroArch

Just tried SNES9x on Windows with the latest RA commit, tried super mario all-stars and super mario world.
Tested on commit libretro/RetroArch@ead5b2c

SMAS takes roughly a minute to get the achievements.
SMW takes less, around 20 seconds.

[Feature Request] New runtime rcheevos interpreter (Python or custom)

Hello, while this is mostly just an idea at this point I wanted to start a public discussion on this topic. Currently, of course, achievements/leaderboards/rich presence only support being coded in the custom rcheevos syntax. rcheevos, as it stands, struggles with many titles on more modern consoles and unfortunately its problems are fundamental and can only really be mitigated at most by future extensions. Alternative means of coding assets parallel to rcheevos will allow developers more freedom in developing not only for titles hostile towards rcheevos, but for developing more expressive and better achievements. These means would be implemented in a similar fashion to how FFXI's achievements are, where there is dummy rcheevos logic which is always false, but an alternate mechanism unlocks the achievement (in this case, the toolkit portion of a Python API).

Here are a list of problems with the current solution I believe that a Python-based (or other, alternative) solution may be able to completely (or conditionally, as discussed later) solve:

Problems (rcheevos):

Poorly supported arithmetic

Basic arithmetic and bitwise operations are provided. Thus the developer can calculate the result of byte(1) + byte(2), etc. Issues arise when the developer wishes to do a more complex operation which requires intermediate values. For example, (byte(1) + byte(2)) / (byte(3) + byte(4)) is impossible to represent in rcheevos. As there is one accumulator, the result of byte(3) + byte(4) cannot be remembered to then be used to divide byte(1) + byte(2) by. This is addressed somewhat by TheMysticalOne's recent pull request to support 2 accumulators, though it's difficult to say if that would completely solve that issue. Other arithmetic tools, such as sqrt, pow, sin, cos, tan, along with rounding methods, are also not available to the programmer but they may prove beneficial to some which wish to employ 2d or 3d geometry in their achievements (some developers already use 3D bounding boxes, which along with some planes are the only shapes which can be used at the moment).

Poorly supported pointer arithmetic / beginner-friendly memory map hinders intermediate and above developers

Similar to the above section -- more than 1 address cannot be used to construct a pointer; this also means index-based pointer math is also impossible if a pointer has already been followed at least 1 level deep. Remember may help alleviate this issue. However, RA also employs a custom virtual memory mapping for games which while it works great for beginner developers it quickly becomes a hinderance when pointers are necessary. When reading a pointer, it's necessary to transform it to conform to RA's existing memory map. This has several issues:

1. A specific pointer may point to one of several different memory regions. While not impossible to work around, handling the pointer for each memory region it may point to is necessary within achievement logic.
2. Memory mappings are woefully documented. How exactly someone should transform a read pointer to conform to RA's memory mapping is largely passed around via word of mouth (e.g. someone saying you should use & 0x1fffff for pointers on PS1). If you didn't know how you should transform a pointer to make it conform to RA's memory, how would you find out? The page on Add Address (https://docs.retroachievements.org/AddAddress-Flag/) barely discusses this necessity - it utilizes a bitwise & in an example, but doesn't particularly draw a clear distinction between what separates the value of the pointer and the RetroAchievements address, especially for each platform. Even if you conducted your own investigation and possibly ended up at https://github.com/libretro/RetroArch/blob/fcd546b74c0a97d4027667237b8b9c29aa3aa6ab/deps/rcheevos/src/rcheevos/consoleinfo.c#L263 -- which lists the regions of each console which is explicitly defined, you still wouldn't know how the memory is mapped out for many consoles which aren't explicitly defined (and may actually then go on to break existing Add Address flags if a region is ever explicitly defined for them). For example, how should a pointer be transformed in order to be utilized for the Sega NAOMI 2 Arcade board?
3. Developers are only able to access regions which are explicitly defined and added to the RA memory map. In both GBA games I've worked on (Smashing Drive and Stuntman), being able to follow a pointer to ROM and evaluate the data at that address would have helped to simplify the existing logic, but the ROM is currently completely inaccessible.
4. The ability to even transform existing game pointers to match RA's memory map is only possible due to each memory having a clearly defined constant starting address and length in virtual memory. This particularly concerns me that Add Address may be impossible or exceedingly difficult to use for a potential DOS rollout. If in virtual memory the starts of these regions shifted around, it would be incredibly difficult to map them to conform them to RA's memory mapping (see the "Memory Regions" section here for the proposed DOS layout: #304).

Lack of control flow / guaranteed halting

rcheevos allows for no runtime logic, instead requiring that all possible scenarios are already accounted for and evaluated inside the achievement logic. This is probably the most fundamental flaw of the design listed, as this makes many data structures such as hashtables or linked lists impossible to access (or very difficult, with some extreme workarounds). Titles which utilize some sort of higher-level script interpreting will have that data difficult to access and the number of titles which utilize higher-level languages will only increase as more consoles are supported. DOS titles are also probably more likely to not have rewritten C++'s STL like console developers have, so they would probably be more inclined to use structures like std::list or std::map.

However, rcheevos' current design means that it is guaranteed to halt. This is a boon because even if an asset may be incredibly complicated and long it is guaranteed to complete every update.

Lack of variables

A simple one -- hits replace variables, and have a very limited use. Allowing data to be saved for later in the program's lifetime would allow for better assets. A very simple and not very inconsequential example is that for my Air Ranger set (https://retroachievements.org/game/27495) the leaderboards submit their scores earlier than I would like. Instead of submitting on the mission result screen, they submit when the mission completes. I could find a means to determine the player is on a certain part of the results screen pretty easily, but by that time the value I'd like to use would be gone. When the mission completes, the game converts the in-game frame timer used for the mission into seconds and overwrites the frame timer with that. A few frames later, the game unconditionally increments the second count by 1, also wiping out the prior value. This means I had to compromise by submitting the leaderboard early at what was unfortunately the last possible time in order to keep the frame precision on player times.

Performance implications

This probably deserves a bit more analysis and admittedly I'm going to introduce some assumptions. rcheevos is a lean solution but its performance is betrayed by 2 major issues:

1. The need for extreme workarounds to navigate some of its issues. Due to the lack of runtime logic support, all possibilities for memory that indicates an achievement has been satisfied must be evaluated each update. For example, https://retroachievements.org/achievement/262250 evaluates 37 linked list nodes in 37 factorial comparisons instead of the 37 comparisons another solution would need.
2. Since all assets operate independently, a lot of unnecessary duplicate code has to be run. An obvious example is something like level-based progression for an older game -- you may have 10 achievements with exactly the same logic, but are differentiated by a level ID that is checked. Of course, in some instances there could be many more than 10 duplicates. One of my sets, Firefighter F.D.18 (https://retroachievements.org/game/27515) has 360 leaderboards which are constantly running and are all independently evaluated. The only difference between them are 3 values: the stage ID, difficulty, and character ID. A smarter solution (such as a Python script) could simply check if a level has been completed once, and then determine which of 360 leaderboards should be submitted using those 3 values instead of checking 360 times. Other sets, such as rhythm games, may yet be more egregious still with some rhythm games having over 1000 leaderboards.

Difficulty to maintain assets created with rcheevos

Originally, as indicated by scott, rcheevos basically just began as a way to check if a memory value was a specific value at a certain time -- for example, to check if Sonic had a ring count of 100 in Sonic the Hedgehog. Since then rcheevos has been extended as needed and has become a house of cards of difficult-to-understand nuances. Combine that with other issues such as logic only showing addresses (i.e. not showing any sort of variable name -- instead notes have to be constantly referenced back and forth), games becoming more complicated, some developers using RATools and some employing the workarounds mentioned previously it becomes nearly impossible to tell what some logic is doing when viewing it on the website.

rcheevos is specialized knowledge

This ties in with the above point -- people have to learn the rcheevos toolkit. They can't use the rcheevos toolkit anywhere else, and they probably won't have any reason whatsoever to learn it before applying to become a developer. Using Python, Lua, or really any other common solution would give new developers programming skills which could be used elsewhere and also potentially speed up the junior developer pipeline for new developers which already know those languages.

Where do we go from here?

Before discussing potential solutions to the above problems, I want to outline a few requirements for a new solution (as written up by Biendeo (https://github.com/Biendeo)):

• Requirements
  • Variables including values that can persist across frames (no session persistence required)
    • Basic primitives such as int8, int16, int32, float, string, bool
  • Conventional programming arithmetic and bitwise operators (+, -, *, /, %, &, |, ^, <<, >>)
  • Arrays
  • Dictionaries
  • Strings (including formatting and general manipulation)
  • Functions (no need for multiple returns)
  • Structures (doesn't have to be classes)
    • Must support type definitions including arrays and pointers (probably helps to match C-style)
  • Shared execution context between achievements/leaderboards if the script allows it (e.g. a dev can compute a score value and reuse that computation across multiple achievements)
  • A common library that allows achievement scripts to commonly access memory and emit RA events (includes achievements, leaderboards, RP, design TBD)
  • Inline comments (this may supersede code notes?)
  • Runtime control flow
• Really nice to haves
  • Lambdas/predicates and functional features (such as maps, filters)
  • Prioritize dereferencing addresses using their original virtual address instead of first mapping to RA map
  • Some way to automatically associate code to IDs (can be a script output or an inbuilt language/library feature)
  • Unit tests? Would help accidental breakages, but how practical would they be?
• Limitations
  • Zero IO (no files, web requests, this should be as pure as possible)
  • No external compilation or processing to work in this framework
  • Needs to still allow offline (i.e. without the presence of a game or emulator) analysis of what achievements/leaderboards are utilised by the script

There are two (and a half, I guess, if you count Lua or another language) solutions I'd like to propose which will address the above requirements and many of the above problems: a Python-based toolkit, and a toolkit based on a custom specialized scripting language (such as perhaps an extension of the already existing RAScript, although perhaps with some changes so that achievements can be arbitrarily unlocked to address point 2 in Performance Implications).

Problems (Either solution)

Developer culture

RetroAchievements developers aren't taught to be scripters -- they're taught, primarily, to utilize the toolkit in the provided editor in order to make achievement sets. This isn't meant to be disparaging, this is just the reality of how developers are taught at the moment. Some developers may not be able to adapt, or refuse to adapt and simply stop developing for a different toolkit. This is why I think that a new solution should run in a similar fashion to FFXI's standalone, so that the old and new toolkits exist in parallel, at least for the time being.

Emulator integration

Not only would emulators have to integrate a new toolkit, if it ran in parallel to the old toolkit they would have to integrate and maintain both means of developing achievements -- this will likely prove a difficult issue to tackle but for the time being I would propose using PCSX2 as a testing ground for a new toolkit. PCSX2 is already standalone and faces probably the greatest incidence of issues which may be solved with a new toolkit.

Runtime logic may not halt

This is a big disadvantage a new toolkit with runtime logic capabilities would have over the existing toolkit: it may never stop running, or even just run for too long. Research would need to be conducted into how to best handle this, and while Python is pretty fast a custom RAScript-interpreter may offer more control over handling the performance of scripts.

Runtime logic may halt too soon

The other side of the coin -- how should runtime errors be handled? A custom RAScript-like language could try to avoid runtime errors at any cost, but that also runs the risk of causing hard-to-debug unexpected behaviors.

Problems (Python)

Requires heavy Python installation

Lua may be a more lightweight solution here.

Newer versions require newer versions of Windows

Difficulty of sandboxing/security issues

Arbitrary Python code can of course easily damage one's PC, so using an already existing language such as Lua or Python would require that scripts run in a sandboxed environment and ideally with no access to network or file I/O.

Conclusion

A new toolkit and its exact implementation is something Biendeo (https://github.com/Biendeo) and I (and hopefully other developers here) would like to investigate, but it is currently mostly in a brainstorming phrase. I wanted to open up a discussion here to hopefully identify some potential pitfalls and perhaps discuss finer details.

[Hashing] Decrypt Nintendo DS header during hashing if encrypted

Nintendo DS games are equally playable both encrypted and decrypted. Only about 2KiB of data is affected by the encryption, but unfortunately that section is hashed. Given that systems capable of emulating NDS are easily strong enough to handle the decryption quickly, I suggest we decrypt as part of the hashing process for encrypted roms.

Reference code:
https://github.com/SabreTools/NDecrypt/tree/master/NDecrypt/NDS
https://github.com/TuxSH/wooddumper/tree/master/woodsec

VFS support

Feature description

I'm bringing up this issue because there's a feature I've started working on. It'll require rcheevos patches, and I'm wondering if it would be acceptable for merge.

The problem to solve is that fopen() is called in this library, but often times a frontend will be loading a game from a SAMBA share or from within a zip file.

The solution follows the path of RetroArch, which added a VFS API so that cores can access games on a VFS (first proposed by me in 2016).

Requirements and drawbacks

The downside of a VFS API is that they seem to be proliferating. RetroArch has an extensive one with 19 functions. Kodi has the equivalent, and I've started exposing kodi's vfs to libretro cores (kodi-game/game.libretro#58). It'll add a lot of code.

However, I think we only need a subset of a full VFS API, which would be these seven functions:

• get_path()
• open()
• close()
• size()
• tell()
• seek()
• read()

Motivation

The motivation comes from my GSoC student, who finished his project early and went on to add a unique feature: machine captioning of savestates (garbear/xbmc#120).

He even kept going and managed to connect live presence to discord (garbear/xbmc#123).

Being able to caption savestates of games on SAMBA and within zips would be cool.

Thoughts?

Support lua51.

I was trying to work on building RetroArch with a system version of lua as requested in issue libretro/RetroArch#8153.

However this is made impossible by two things.

1. rcheevos only builds with lua52 or lua53.
2. Upstream lua doesn't provide their own shared library or pkgconfig files and this makes checking for the versioned lua libraries or pkgconfig files not portable since every distro reinvents the wheel to accomplish this...

If rcheevos built and worked with any lua version this would be a lot easier to support.

The current build problem is.

deps/rcheevos/src/rcheevos/operand.c: In function ‘rc_evaluate_operand’:
deps/rcheevos/src/rcheevos/operand.c:348:38: error: ‘LUA_OK’ undeclared (first use in this function); did you mean ‘LUA_QL’?
         if (lua_pcall(L, 2, 1, 0) == LUA_OK) {
                                      ^~~~~~
                                      LUA_QL
deps/rcheevos/src/rcheevos/operand.c:348:38: note: each undeclared identifier is reported only once for each function it appears in
make: *** [Makefile:201: obj-unix/release/deps/rcheevos/src/rcheevos/operand.o] Error 1

This feature is apparently not supported by lua51.

RetroAchievement related crashes on android

Hey @leiradel I started testing RA with the new implementation and I found it quite crashy so far
All these tests were on ARMv7e RA commit 4f2c108
libretro/RetroArch@4f2c108

Example 1:
Snes9x with Super Mario All-Stars

• achievements on
• leaderboards off
• hardcore mode on
• badges off
• unofficial off
• verbose on
• screenshot off

Crashes right after logging in

09-30 23:03:55.018  1795  1848 I RetroArch: [autoconf]: selected configuration: /storage/emulated/0/RetroArch/userdata/profiles/android/NVIDIA_SHIELD_Portable_X1_Custom.cfg
09-30 23:03:57.763  1795  1823 F libc    : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 1823 (Thread-2)
09-30 23:03:57.763   234   234 W         : debuggerd: handling request: pid=1795 uid=10074 gid=10074 tid=1823
09-30 23:03:57.763  1861  1861 W debuggerd: type=1400 audit(0.0:774): avc: denied { search } for name="com.retroarch" dev="mmcblk0p29" ino=509467 scontext=u:r:debuggerd:s0 tcontext=u:object_r:app_data_file:s0:c512,c768 tclass=dir permissive=0
09-30 23:03:57.801   317   455 W nvaudio_hw: UNDERRUN DETECTED for music-playback ret -1 avail 0
09-30 23:03:57.828  1861  1861 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
09-30 23:03:57.828  1861  1861 F DEBUG   : LineageOS Version: '14.1-20180328-NIGHTLY-foster'
09-30 23:03:57.828  1861  1861 F DEBUG   : Build fingerprint: 'NVIDIA/loki_e_wifi/foster:7.0/NRD90M/2427173_1038.2788:user/release-keys'
09-30 23:03:57.828  1861  1861 F DEBUG   : Revision: '0'
09-30 23:03:57.828  1861  1861 F DEBUG   : ABI: 'arm'
09-30 23:03:57.828  1861  1861 F DEBUG   : pid: 1795, tid: 1823, name: Thread-2  >>> com.retroarch <<<
09-30 23:03:57.828  1861  1861 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
09-30 23:03:57.828  1861  1861 F DEBUG   :     r0 00000000  r1 00000000  r2 66c6db00  r3 00007ff2
09-30 23:03:57.828  1861  1861 F DEBUG   :     r4 00000001  r5 00000005  r6 00000000  r7 520ba1b8
09-30 23:03:57.828  1861  1861 F DEBUG   :     r8 00000000  r9 00000000  sl 6605084c  fp 4af832d0
09-30 23:03:57.828  1861  1861 F DEBUG   :     ip 6656b828  sp 4af832c8  lr 65e99898  pc 660508a4  cpsr 600e0010
09-30 23:03:57.832  1861  1861 F DEBUG   :
09-30 23:03:57.832  1861  1861 F DEBUG   : backtrace:
09-30 23:03:57.833  1861  1861 F DEBUG   :     #00 pc 0031d8a4  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so
09-30 23:03:57.833  1861  1861 F DEBUG   :     #01 pc 00167f44  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so (rc_evaluate_operand+344)
09-30 23:03:57.833  1861  1861 F DEBUG   :     #02 pc 00168364  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so
09-30 23:03:57.833  1861  1861 F DEBUG   :     #03 pc 0016961c  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so (rc_test_trigger+136)
09-30 23:03:57.833  1861  1861 F DEBUG   :     #04 pc 00162144  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so
09-30 23:03:57.833  1861  1861 F DEBUG   :     #05 pc 001618d4  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so (cheevos_test+24)
09-30 23:03:57.833  1861  1861 F DEBUG   :     #06 pc 00207438  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so (runloop_iterate+10968)
09-30 23:03:57.833  1861  1861 F DEBUG   :     #07 pc 002047ec  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so (rarch_main+296)
09-30 23:03:57.833  1861  1861 F DEBUG   :     #08 pc 00346044  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so
09-30 23:03:57.833  1861  1861 F DEBUG   :     #09 pc 0024ff70  /data/app/com.retroarch-2/lib/arm/libretroarch-activity.so
09-30 23:03:57.833  1861  1861 F DEBUG   :     #10 pc 00047bf3  /system/lib/libc.so (_ZL15__pthread_startPv+22)
09-30 23:03:57.833  1861  1861 F DEBUG   :     #11 pc 00019f1d  /system/lib/libc.so (__start_thread+6)
09-30 23:03:57.860   317   455 W nvaudio_hw: UNDERRUN DETECTED for music-playback ret -1 avail 0
09-30 23:03:57.823  1861  1861 W debuggerd: type=1400 audit(0.0:775): avc: denied { search } for name="com.retroarch" dev="mmcblk0p29" ino=509467 scontext=u:r:debuggerd:s0 tcontext=u:object_r:app_data_file:s0:c512,c768 tclass=dir permissive=0
09-30 23:03:58.041   699  1868 W ActivityManager:   Force finishing activity com.retroarch/.browser.retroactivity.RetroActivityFuture
09-30 23:03:58.044   699  1868 I WindowManager: setFocusedApp token: Token{da101d5 ActivityRecord{298e68c u0 com.awe.dev.pro.tv/.Launcher t1846}} requestedOrientation: 6
09-30 23:03:58.056  1554  1554 I CmaSystemUpdateService: ActiveReceiver: Intent { act=android.os.UpdateLock.UPDATE_LOCK_CHANGED flg=0x4000010 cmp=com.google.android.gms/.update.SystemUpdateServiceActiveReceiver (has extras) }
09-30 23:03:58.056  1554  1554 I CmaSystemUpdateService: receiver: Intent { act=android.os.UpdateLock.UPDATE_LOCK_CHANGED flg=0x4000

Add Extended RAM / Save RAM for SMS/GG

https://retroachievements.org/game/12638

This reportedly has memory exposure issues, with character stats somehow unable to be found in RAM. This is due to the game actually storing a lot of info in Save RAM rather than System RAM, and rcheevos' memory map only has System RAM exposed.

GG and SMS would both only have System RAM exposed, but cartridges may offer Extended (volatile) RAM, battery backed SRAM, or EEPROM (although this last one is probably not going to store many useful things considering it isn't random accessible).

Extended RAM would be limited to GG with Ernie Els Golf (E) (En,Fr,De,Es,It) with a variant of the CodeMasters mapper? Presumably the mapper would also work on SMS, although no retail games would use it (and it seems most emulators just emulate these as the same mapper, giving Extended RAM to other CodeMasters mapper games which don't actually use it). This Extended RAM would be 8KiB, and would be mapped to 0x6000-0x7FFF and mirrored at 0xA000-0xBFFF, although this would also overlap with the ROM (these regions are banked).

EEPROM seems to be limited GG too, with only a few games having it (presumably, it would work on SMS, although no retail games would use it). This would just be 128 bytes of non-volatile memory, which can't be randomly accessed by the game, requiring a slow read/write process, bit-banging at 0x8000-0xBFFF (exact address doesn't matter here, they all map to EEPROM control).

Most games would use the standard Sega mapper which features battery backed SRAM. This can go as high as 32KiB, and is mapped at 0x8000-0xBFFF. This would also overlap with ROM (banking done again), and only half of the total possible SRAM is mappable here, so only 16KiB may be visible at one time.

Extended RAM, battery backed SRAM, and EEPROM are all mutually exclusive.

https://www.smspower.org/Development/Mappers seems to have some docs on the CodeMasters and Sega mappers.

Multiple Leaderboard Trackers Overlap

Hey not sure if this is the right repo to log this in, but I am using PCSX2 1.7.5243 and when there are multiple leaderboards active at once, the trackers in the bottom right overlap and are not really readable. I have noticed this in Sly 2, which has a permanent leaderboard pretty much for number of coins, then individual missions can have other leaderboards (the one in this screenshot is a "count" one as opposed to time):

Ideally, the leaderboard trackers should be stacked IMO. Thanks!

.rvz management for gamecube games

Hello,

Would it be possible to implement .rvz decompression in order to be able to hash game files which are this format ?

[Hashing] Monster Rancher 1 / Monster Rancher 2 need supported ways to switch discs

The two games in question here:

• https://retroachievements.org/game/16343 - Monster Rancher / Monster Farm
• https://retroachievements.org/game/11287 - Monster Rancher 2 / Monster Farm 2

These games are both single disc games and have a single supported hash. This works for the core mechanics of the game. However, one of the core gimmicks with these games is the ability to hot-swap arbitrary discs, read data from said discs, and generate usable monsters from them. This was done on console by getting to an in-game location called the Shrine, which would prompt you to insert another disc and move forward like the following image indicates.

To fully support these games in a way that allows for achievements, we would need some common hash that can be used as a basis for generating disc image data that contains different monsters. Other tools out there in the Monster Rancher community support generating discs already, and as of writing none of them write the data needed for RetroAchievements to hash them. Specifically, they don't write the contents of a SYSTEM.CNF file and any associated data from that needed.

To reiterate the core issue: Monster Rancher and Monster Rancher 2 are single-disc PSX games that support and encourage hot-swapping discs to get different monsters. Currently RetroAchievements recognizes a single hash for each of these games. What can be done to try and get a common hash setup for either of these games to allow for the hot-swapping disc mechanic?

Helpful resources from the Monster Rancher community around current monster generation tools:

• https://legendcup.com/make-a-monster-mr1.php - this is a front-end for generating a BIN+CUE pair for Monster Rancher.
• https://github.com/OmnigamerSDA/MonsterMaker - repo for building a .exe that can generate a BIN+CUE pair for Monster Rancher.
• https://legendcup.com/make-a-monster-mr2.php - this is a front-end for generating a BIN+CUE pair for Monster Rancher 2.

RC_FORMAT_FRAMES is inaccurate/a misnomer depending on platform/region

RC_FORMAT_FRAMES assumes 60fps, but this is not true for PAL-optimized titles (50Hz) or some platforms like Wonderswan that runs at 75Hz. It should be renamed or parameterized to avoid confusion.

Problems hashing PS2 games (differences with PCSX2 hashing implementation?)

LaunchBox recently integrated rcheevos library for hashing of files in their frontend, but it's failing quite often when identifying PS2 CHD games (returning an error, not returning the wrong hash).

I opened an issue with them here: https://bitbucket.org/jasondavidcarr/launchbox/issues/7624/retroachievements-scan-inconsistent-spotty

Looking at the PCSX2 implementation for comparison, looks like they're not using ra_hash_ps2 directly? Wonder if there's any inconsistencies between the two.

PCSX2: https://github.com/PCSX2/pcsx2/blob/master/pcsx2/Frontend/Achievements.cpp#L1380