n64recomp/n64recomp

Tool to statically recompile N64 games into native executables

N64: Recompiled

N64: Recompiled is a tool to statically recompile N64 binaries into C code that can be compiled for any platform. This can be used for ports or tools as well as for simulating behaviors significantly faster than interpreters or dynamic recompilation can. More widely, it can be used in any context where you want to run some part of an N64 binary in a standalone environment.

This is not the first project that uses static recompilation on game console binaries. A well known example is jamulator, which targets NES binaries. Additionally, this is not even the first project to apply static recompilation to N64-related projects: the IDO static recompilation recompiles the SGI IRIX IDO compiler on modern systems to faciliate matching decompilation of N64 games. This project works similarly to the IDO static recomp project in some ways, and that project was my main inspiration for making this.

Table of Contents

• How it Works
• Overlays
• How to Use
• Single File Output Mode
• RSP Microcode Support
• Planned Features
• Building

How it Works

The recompiler works by accepting a list of symbols and metadata alongside the binary with the goal of splitting the input binary into functions that are each individually recompiled into a C function, named according to the metadata.

Instructions are processed one-by-one and corresponding C code is emitted as each one gets processed. This translation is very literal in order to keep complexity low. For example, the instruction addiu $r4, $r4, 0x20, which adds 0x20 to the 32-bit value in the low bytes of register $r4 and stores the sign extended 64-bit result in $r4, gets recompiled into ctx->r4 = ADD32(ctx->r4, 0X20); The jal (jump-and-link) instruction is recompiled directly into a function call, and j or b instructions (unconditional jumps and branches) that can be identified as tail-call optimizations are also recompiled into function calls as well. Branch delay slots are handled by duplicating instructions as necessary. There are other specific behaviors for certain instructions, such as the recompiler attempting to turn a jr instruction into a switch-case statement if it can tell that it's being used with a jump table. The recompiler has mostly been tested on binaries built with old MIPS compilers (e.g. mips gcc 2.7.2 and IDO) as well as modern clang targeting mips. Modern mips gcc may trip up the recompiler due to certain optimizations it can do, but those cases can probably be avoided by setting specific compilation flags.

Every output function created by the recompiler is currently emitted into its own file. An option may be provided in the future to group functions together into output files, which should help improve build times of the recompiler output by reducing file I/O in the build process.

Recompiler output can be compiled with any C compiler (tested with msvc, gcc and clang). The output is expected to be used with a runtime that can provide the necessary functionality and macro implementations to run it. An example of most of the required macro implementations can be found in the Zelda 64: Recompiled project here, with the project also containing accompanying code for implementing the rest of the required runtime.

Overlays

Statically linked and relocatable overlays can both be handled by this tool. In both cases, the tool emits function lookups for jump-and-link-register (i.e. function pointers or virtual functions) which the provided runtime can implement using any sort of lookup table. For example, the instruction jalr $25 would get recompiled as LOOKUP_FUNC(ctx->r25)(rdram, ctx); The runtime can then maintain a list of which program sections are loaded and at what address they are at in order to determine which function to run whenever a lookup is triggered during runtime.

For relocatable overlays, the tool will modify supported instructions possessing relocation data (lui, addiu, load and store instructions) by emitting an extra macro that enables the runtime to relocate the instruction's immediate value field. For example, the instruction lui $24, 0x80C0 in a section beginning at address 0x80BFA100 with a relocation against a symbol with an address of 0x80BFA730 will get recompiled as ctx->r24 = S32(RELOC_HI16(1754, 0X630) << 16);, where 1754 is the index of this section. The runtime can then implement the RELOC_HI16 and RELOC_LO16 macros in order to handle modifying the immediate based on the current loaded address of the section.

Support for relocations for TLB mapping is coming in the future, which will add the ability to provide a list of MIPS32 relocations so that the runtime can relocate them on load. Combining this with the functionality used for relocatable overlays should allow running most TLB mapped code without incurring a performance penalty on every RAM access.

How to Use

The recompiler is configured by providing a toml file in order to configure the recompiler behavior, which is the only argument provided to the recompiler. The toml is where you specify input and output file paths, as well as optionally stub out specific functions, skip recompilation of specific functions, and patch single instructions in the target binary. There is also planned functionality to be able to emit hooks in the recompiler output by adding them to the toml (the [[patches.func]] and [[patches.hook]] sections of the linked toml below), but this is currently unimplemented. Documentation on every option that the recompiler provides is not currently available, but an example toml can be found in the Zelda 64: Recompiled project here.

Currently, the only way to provide the required metadata is by passing an elf file to this tool. The easiest way to get such an elf is to set up a disassembly or decompilation of the target binary, but there will be support for providing the metadata via a custom format to bypass the need to do so in the future.

Single File Output Mode (for Patches)

This tool can also be configured to recompile in "single file output" mode via an option in the configuration toml. This will emit all of the functions in the provided elf into a single output file. The purpose of this mode is to be able to compile patched versions of functions from the target binary.

This mode can be combined with the functionality provided by almost all linkers (ld, lld, MSVC's link.exe, etc.) to replace functions from the original recompiler output with modified versions. Those linkers only look for symbols in a static library if they weren't already found in a previous input file, so providing the recompiled patches to the linker before providing the original recompiler output will result in the patches taking priority over functions with the same names from the original recompiler output.

This saves a tremendous amount of time while iterating on patches for the target binary, as you can bypass rerunning the recompiler on the target binary as well as compiling the original recompiler output. An example of using this single file output mode for that purpose can be found in the Zelda 64: Recompiled project here, with the corresponding Makefile that gets used to build the elf for those patches here.

RSP Microcode Support

RSP microcode can also be recompiled with this tool. Currently there is no support for recompiling RSP overlays, but it may be added in the future if desired. Documentation on how to use this functionality will be coming soon.

Planned Features

• Custom metadata format to provide symbol names, relocations, and any other necessary data in order to operate without an elf
• Emitting multiple functions per output file to speed up compilation
• Support for recording MIPS32 relocations to allow runtimes to relocate them for TLB mapping
• Ability to recompile into a dynamic language (such as Lua) to be able to load code at runtime for mod support

Building

This project can be built with CMake 3.20 or above and a C++ compiler that supports C++20. This repo uses git submodules, so be sure to clone recursively (git clone --recurse-submodules) or initialize submodules recursively after cloning (git submodule update --init --recursive). From there, building is identical to any other cmake project, e.g. run cmake in the target build folder and point it at the root of this repo, then run cmake --build . from that target folder.

Libraries Used

• rabbitizer for instruction decoding/analysis
• ELFIO for elf parsing
• toml11 for toml parsing
• fmtlib

n64recomp's Issues

Autosave feature doesn't work in majors mask 4th and 0th day

Not sure if this is a bug or not but the built-in autosave feature doesn't work on any glitched days like 4th day or 0th day

Use the section's sh_info field to determine the target section for a reloc section instead of name

Tutorial?

Tutorial please

Build N64Recomp to be actually bootable?

Heyhey!

After quite some testing and trying, I finally was able to get a working .elf from a game of my choice that actually works with N64Recomp!

So I finally was able to run "./N64Recomp siliconvalley.toml"!
But... what now?
n64comp@cellenser:/mnt/N64Recomp-Project/SiliconValley# ./N64Recomp siliconvalley.toml Sections Num symbols: 1170 Found entrypoint, original name: func_80125900 Function count: 1030 Working dir: /mnt/N64Recomp-Project/SiliconValley

This is my toml-file:
[input] entrypoint = 0x80125900 elf_path = "siliconvalley.elf" directory = "output" output_func_path = "output/functions"

it created the output which contains four files:

• funcs.h
• lookup.cpp
• recomp_entrypoint.c
• recomp_overlays.inl

And another file that has been generated by another tool is the siliconvalley.map

Does anyone have an Idea how to finalize it?
There's no Guide or Explaination for that yet. What to do with the Files that have been created?

Compile an N64 game with existing source to run on PC

Is it possible to use the system developed here to compile a game with known source as an N64 derived PC game?

By known source I mean something like this: https://github.com/meeq/FlappyBird-N64

Would you be able to compile against the known source or would you have to compile it into a ROM and decompile it to be used? Or would you have to rebase it against RT64 manually?

Sorry if this isn't detailed enough or in the wrong place, I just wanted to know the answer to this and noone else seems to have asked as far as I could see.

I feel like it would be really useful to be able to develop an N64 game with a PC version in this way, as RT64 claims to behave in the same way as N64 hardware for rendering so debugging might be easier. Also being able to provide a PC version using the same code base with additional features would be nice.

Build error involving rabbitizer

When building N64 recomp with the latest available version including the default versions of it's dependencies, I get the following error:

~/N64Recomp/build$ cmake --build .
[ 79%] Built target rabbitizer
[ 85%] Built target fmt
[ 87%] Building CXX object CMakeFiles/N64Recomp.dir/src/recompilation.cpp.o
/home/user/N64Recomp/src/recompilation.cpp: In function ‘bool process_instruction(const RecompPort::Context&, const RecompPort::Config&, const RecompPort::Function&, const RecompPort::FunctionStats&, const std::unordered_set&, size_t, const std::vectorrabbitizer::InstructionCpu&, std::ofstream&, bool, bool, int, size_t, bool&, bool&, std::span<std::vector >)’:
/home/user/N64Recomp/src/recompilation.cpp:831:19: error: ‘cpu_c_deq_d’ is not a member of ‘InstrId’ {aka ‘rabbitizer::InstrId::UniqueId’}; did you mean ‘cpu_c_eq_d’?
831 | case InstrId::cpu_c_deq_d: // TODO rename to c_seq_d when fixed in rabbitizer
| ^~~~~~~~~~~
| cpu_c_eq_d
gmake[2]: *** [CMakeFiles/N64Recomp.dir/build.make:118: CMakeFiles/N64Recomp.dir/src/recompilation.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:149: CMakeFiles/N64Recomp.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

Is there any way to fix this line such that the program compiles properly?

Build Guide

Would be helpful!

Trying to build it from the vague guide that is available

Add a mode to the recompiler where it can accept a symbol definition file instead of an elf file

This will significantly simplify the build process of recompilations, since it will bypass the requirement to build a decompilation/disassembly before building a recompilation. This can also be used in patch recompilation so that it can handle relocations automatically.

The symbol definition file will need to include section ROM offsets and base VRAM addresses, symbol parent sections and VRAM addresses, and sizes for function symbols. It could also accept file boundaries so the recompiler can emit multiple functions per file, which would speed up configure and build steps for recompilations.

Consider recompiling libgcc/libkmc functions instead of ignoring them

Currently Recomp ignores a lot of libgcc functions and expects the user to provide manual reimplementations, Ideally those could get recompiled instead.

This was brought up here: #35 (comment)

Error linking RSPRecomp

I'm currently following this procedure to build the project:

$ git clone --recurse-submodules https://github.com/Mr-Wiseguy/N64Recomp.git
$ cd N64Recomp
$ cmake .
$ cmake --build .

The build eventually fails with this error:

[ 97%] Linking CXX executable RSPRecomp
/usr/bin/ld: cannot open output file RSPRecomp: Is a directory
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/RSPRecomp.dir/build.make:99: RSPRecomp] Error 1
make[1]: *** [CMakeFiles/Makefile2:158: CMakeFiles/RSPRecomp.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

This appears to be because the target executable has the same name as a directory in the top-level of the root; just lower-casing the name of the target executable (and project)—I have a case-sensitive filesystem—results in the build succeeding.

Alternatively, renaming the source directory may be less invasive. I didn't know which would be the preferred choice, so this is an issue rather than a PR, but if one option is clearly better, I'm happy to PR it (though, it's probably just as easy for that change to be done upstream, and I don't need credit or anything ☺).

Split runtime into separate repo

The recomp runtime (libultra function implementations and other associated code) should be in a separate project. The version in this repo is out of date as there have been many upgrades to it in the MM one.

[Question] Why does this work ?

Not an issue, but I don't know where else to ask this.

By searching "static recompilation" online, you find a lot of people saying it does not work / it is a bad idea:
https://andrewkelley.me/post/jamulator.html#conclusion
https://stackoverflow.com/questions/11215689/why-does-emulation-have-to-be-done-in-real-time
https://cs.stackexchange.com/questions/155511/why-is-static-recompilation-not-possible

So I'm curious if you have insides into why this project worked so well ?

`jal` resolution/target warnings

I tested out the Paper Mario US decompilation to some success, though it threw a lot of warnings about ambiguous resolutions and being unable to find functions at a lot of jumps.

Snippet from warnings:

No function found for jal target: 0x80075110
No function found for jal target: 0x8004D9C0
No function found for jal target: 0x8004DAD8
No function found for jal target: 0x8004DF64
[Warn] Potential jal resolution ambiguity
    ._D_LFE1_0003BD40
    osSetIntMask_recomp
[Warn] Potential jal resolution ambiguity
    ._D_LFE1_0003BD40
    osSetIntMask_recomp
[Warn] Potential jal resolution ambiguity
    fx_sun_undeclared
    fx_sun  

Function replacement with secondary ROM

Built on #1, there needs to be a mechanism to replace a function with an implementation pulled from a second ROM (e.g. a modded one).

This should accept the name of the function to replace and the ROM address, VRAM address, and size of the replacement function in the second ROM to use. If any of these value are omitted, they will be assumed to be the same as that of the original function. (which would be true for classic binary patched modded ROMs).

Config file input

A config file input mechanism for the recompiler is required for implementing any modding support. The base implementation should allow it to specify a list of functions to stub out or omit from recompilation.

Generating an elf file

I've been using Ghidra to help try and get close to generating an elf file and I have found the entry point that I am looking for, however, I am unable to figure out what I need to do to generate the elf file. Any help would be greatly appreciated!

Failed to determine size of jump table

Was trying to get this working based on this decompilation of Mischief Makers (version 1.1). It managed to produce a working result from which I utilized the outputted .elf from.
(I was looking to use a more updated one but faced like a billion issues regarding library versions and assembly generation and decided to not bother...)

Reported was:

Sections                                                                                                                              
Num symbols: 5754                                                                                                                     
Function count: 3288 

Failed to determine size of jump table at 0x801B02E4 for instruction at 0x80779264                                                    
Failed to analyze func_807790F4                                                                                                       
Error recompiling func_807790F4

It had managed to output 13 C files before hitting the error. Cannot tell if this was an issue from the elf produced by the project or something N64Recomp currently isn't equipped to handle.

Also of note, I had identified that the entrypoint in the ROM header was 0x80000400 but actually using it said it could not find the entrypoint function.

Why DX12 for Windows Builds Instead of VK?

As a user, I am curious as to what considerations motivated the decision to use Direct3D 12 for builds generated for Windows & only use Vulkan w/ builds generated for GNU/Linux?

(Opposed to Direct3D 12, )Vulkan is:

1. Free/libre & open-source software (F/LOSS)
2. Even lower level & faster than Direct3D 12
3. Cross-platform w/ Windows, GNU/Linux, Android, FreeBSD, Nintendo Switch, etc.
4. Supports Windows as far back as Windows XP, where Direct3D 12 only supports Windows & its minimum supported version of it is Windows 10

Why not just maintain one low-level graphics API i.e. Vulkan to rule them all? The devs of dolphin-emu originally supported Direct3D 12 too (although they at least gave Windows users the option of Vulkan), before quickly deprecating it when they reviewed & recognised the above advantages of Vulkan & that maintaining just Vulkan created less work for themselves than also maintaining Direct3D 12 (especially in the days before VKD3D12 real-time API calls translation).

Unknown symbols with `_recomp` suffix, from ignored and reimplemented funcs

Wondering what I'm doing wrong / missing here:

I see that https://github.com/N64Recomp/N64Recomp/blob/main/src/main.cpp#L693-L700 adds this suffix. When I run the recomp tool on the current elf/rom that I'm working with, it generates functions with unknown symbols for each reimplemented/ignored function.

If I dump the context, I see something different from what https://github.com/Zelda64Recomp/Zelda64Recomp/blob/dev/us.rev1.syms.toml has. For example, I see a lot of this in my recompilation:

# Autogenerated from an ELF via N64Recomp
[[section]]
// snip

functions = [
    { name = "recomp_entrypoint", vram = 0x80025C00, size = 0x50 },
]

# Autogenerated from an ELF via N64Recomp
[[section]]
// snip

functions = [
  // lots of good stuff here, but also:
  { name = "__osCheckId_recomp", vram = 0x8010AA24, size = 0xB8 },
  { name = "__osPfsRWInode_recomp", vram = 0x8010AADC, size = 0x344 },
  // etc
]

But these functions are not defined anywhere. I think I would expect that? Since they're explicitly ignored. However, I do see their counterparts in the MM symbols toml, e.g. __osCheckId and __osPfsRWInode.

So what am I supposed to do with these functions in order to compile with a runtime? Should they be getting emitted with my RecompiledFunctions? For example, should I be seeing a __osCheckId_recomp.c file?

It looks like this is the full list of fns that are undefined:

osPfsRepairId_recomp
__osCheckId_recomp
__osPfsSelectBank_recomp
__osContRamRead_recomp
__osPfsRWInode_recomp
__osContRamWrite_recomp
__osViGetCurrentContext_recomp
__osViSwapContext_recomp
__osTimerInterrupt_recomp

Cross-Compiling for ARM Based Computers?

Is it possible if we can get a Cross-Compiling feature for ARM based computers?

Add a temp for jalr jump target

Currently, if the delay slot instruction for a jalr modifies the jump target register then the jalr will lookup the wrong address for the function call. Add a temp to record the jalr target that records the register value before the delay slot is run to prevent this from happening.

AKI games/

could this work with the AKI wcw/wwf games?

compilation failure with clang++ on macOS

I'm not sure exactly what's going on here because I'm no C++ wizard, so apologies for dumping this on you; but compilation seems to fail on macOS, it looks like due to some sort of template-related failure. macOS implements a subset of C++20, so it may be that something here is not fully supported. Not sure if you intend to allow building with macOS clang++ so I wanted to post in order to find out. Let me know if I can provide more information.

The following applies to all the latest macOS/Xcode: 14.4.1, Xcode 15.3.

[ 87%] Building CXX object CMakeFiles/N64Recomp.dir/src/analysis.cpp.o
In file included from /Users/me/code/recomp/N64Recomp/src/analysis.cpp:1:
In file included from /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/set:483:
In file included from /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__memory/allocator.h:14:
In file included from /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__memory/allocate_at_least.h:13:
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__memory/allocator_traits.h:304:9: error: no matching function for call to 'construct_at'
        _VSTD::construct_at(__p, _VSTD::forward<_Args>(__args)...);
        ^~~~~~~~~~~~~~~~~~~
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__config:715:17: note: expanded from macro '_VSTD'
#  define _VSTD std
                ^
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/vector:811:21: note: in instantiation of function template specialization 'std::allocator_traits<std::allocator<RecompPort::JumpTable>>::construct<RecompPort::JumpTable, unsigned int &, unsigned char &, int, unsigned int &, unsigned int &, unsigned int, std::vector<unsigned int>, void, void>' requested here
    __alloc_traits::construct(this->__alloc(), std::__to_address(__tx.__pos_),
                    ^
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/vector:1600:9: note: in instantiation of function template specialization 'std::vector<RecompPort::JumpTable>::__construct_one_at_end<unsigned int &, unsigned char &, int, unsigned int &, unsigned int &, unsigned int, std::vector<unsigned int>>' requested here
        __construct_one_at_end(std::forward<_Args>(__args)...);
        ^
/Users/me/code/recomp/N64Recomp/src/analysis.cpp:187:22: note: in instantiation of function template specialization 'std::vector<RecompPort::JumpTable>::emplace_back<unsigned int &, unsigned char &, int, unsigned int &, unsigned int &, unsigned int, std::vector<unsigned int>>' requested here
                        stats.jump_tables.emplace_back(
                                          ^
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__memory/construct_at.h:36:38: note: candidate template ignored: substitution failure [with _Tp = RecompPort::JumpTable, _Args = <unsigned int &, unsigned char &, int, unsigned int &, unsigned int &, unsigned int, std::vector<unsigned int>>]: no matching constructor for initialization of 'RecompPort::JumpTable'
_LIBCPP_HIDE_FROM_ABI constexpr _Tp* construct_at(_Tp* __location, _Args&&... __args) {
                                     ^

Unhandled instruction: teq

ELF built from https://github.com/star-haven/papermario-dx

Single instruction replacement

Building on top of #1, the config system should provide a way to replace single instructions during recompilation. This can be done via a step that happens after each function's instructions are extracted but before recompilation begins.

Accepting textual instructions would be ideal but would require a basic assembler, so the initial version can just support their assembled numeric values.

[Question] What if a game doesn't have any decompiled source?

Yea, sorry for bothering any of you guys, but I need to ask a question. I'm interested to port a game with this tool, and the game itself doesn't have any progress or anything related to decompilation yet.

Do I need to decompile the game myself? If that is, it is a really dificult process or nah?

Ahm, and the game in question is AeroGauge, which I already reserved a repository and everything but I just need to know this to continue, as I said before, the game itself doesn't have any progress or anything related to decompilation yet.

(I am dumb and posted this issue on other repo, as this issue lmfaoo)

How to compile Smash 64 on Steam Deck?

It's my first time using Cmake on steam deck. A step-by-step guide would be nice! The instructions are too vague so IDK what to do

Recomp to PS1

Is this possible? Because I think that playing Mario 64 on the original Playstation would be awesome

Disambiguate symbols during recompilation by preferring those in the same section.

Currently, some ROMs can have multiple symbols with the same address, which can cause the recompiler to emit incorrect function calls. In this case it should prefer symbols that are in the same section as the function currently being recompiled. If there are duplicates even with that preference, it should exit with an error.

Unlinked (.o) file recompilation

In order to streamline the modding process, the recompiler should be able to accept input that makes it easy to write mods in C (or C++) or assembly. The most straightforward way to do this is to allow it to accept .o files.

The main hurdle to tackle in this problem is symbol resolution. External symbols in .o files need to be resolved in order for the code to be usable. The .o files do not need to be relocated, so full linking isn't required. Two potential solutions are as follows:

• Use a linker (probably ld) to resolve (and relocate) these symbols. However, because .o file support is intended to allow function replacement there will likely be functions with the same names as their vanilla counterparts. Simply using the .elf file as a symbol input for the linker will cause multiple definition errors with this, so external referenced symbols will need to be extracted and fed to the linker directly instead.
• The more complex but also more direct solution would be to implement the symbol resolution directly into the recompiler. It would need to resolve relocations itself (R_MIPS_26, R_MIPS_HI16 and R_MIPS_LO16 at the very least) which includes hi/lo pairing. That pairing is already implemented in the relocatable overlay support functionality so that code could be reused potentially. This solution also removes the dependency on a MIPS linker in order to implement this feature.

The second solution is more ideal, so it will be the one implemented. The output of this feature will be a Function object ready to be run through recompile_function.

Error While Building

I get this error when trying to build;

ld: open() failed, errno=21 for 'N64Recomp'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/N64Recomp.dir/build.make:147: N64Recomp] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:149: CMakeFiles/N64Recomp.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

Ubuntu build instructions?

The build instructions are a bit hard to understand, sorry.

missing .dll files?

when i buildt and ran n64recomp, it said libgcc_s_seh-1.dll and libstdc++-6.dll are missing? it said reinstalling the program may fix this but i tried and it gives me the same error

Runtime code replacement/interpreter

In order to support mods that conditionally edit code (e.g. some randomizers) or distribute simple mods without requiring a separate executable for each, a mechanism for replacing code at runtime must exist.

The first part is actually running the replacement code. The following are all options for implementing that:

• Implement a simple MIPS interpreter targeting the N64's instruction set. It would only need to support a fairly limited subset of the instruction set. This option requires no preprocessing or runtime recompilation, as the replacement code could be run as-is. This option is the simplest one, but performance could be lackluster if there are many functions being interpreted.
• Recompile into a dynamic language, i.e. Lua. Normal mods would recompile their code ahead of time and distribute the recompiler output code files, while mods with conditional code changes would use a version of the recompiler made to be used at runtime. There are several downsides to this approach however. This approach would improve runtime speed over the interpreter (LuaJit would be far faster than any interpreter I could write, even if ran in interpreter mode). However, it's also far more complex: it requires being able to retarget the recompiler to a new language and would require bundling the recompiler itself as part of the recomp runtime. There's also the risk that mods could easily include malicious code in their files, so a good sandbox implementation would be needed to prevent that. Finally, the runtime for this dynamic language would bloat the executable.
• Recompiling mod code via the normal route (i.e. to C) isn't practical for several reasons. The first is that normal mods would include their custom code as a compiled binary, meaning they'd be non-portable and would have a huge security risk as there would be no way to sandbox them. The second reason is that runtime recompilation to C would then also require bundling a compiler (probably clang) to then compile that output C into a binary and load it. That would be a huge added dependency and could also potentially trigger antivirus programs due to native binary generation.

Regardless of which option is chosen to handle running the code, there also needs to be a mechanism for loading the code. Assuming the function set can't be known ahead of time, there are two options:

• Modify the recompiler to perform all function calls through the function lookup table. This could really hurt performance, even when not using any mods (unless a second mod-support binary was included with each recomp). However, this would be the most portable option and allow using mods on platforms that don't allow runtime code patching (e.g. iOS).
• Patch replaced functions with a jump hook to the function that runs replacement code when code is loaded. This would not impact performance of the game at all when mods aren't being used, but would exclude mod support on platforms that don't support runtime code patching. Platform-specific assembly would need to be written for supported architectures (which would probably just be x64 at the start with ARM64 later on potentially). This also requires that all functions are large enough to fit the hook. MSVC already meets this requirement due to its 16-byte function alignment and GCC/Clang have options to enable similar functionality, so this is mostly a non-issue.

how to run on MAC? a simple downloadable .exe and .dmg file?

its hard to to do when u r not a hardcore dev

Anyone try sf64?

I was able to get https://github.com/sonicdcer/sf64 decompiled and working as per their instructions. Thought it would be a good place, since the build folder generates the .elf file and spits out the uncompressed file at the end.

I was able to compile this project pretty easy, but ran into a snag when it came to the elf file (go figure). Im new to all this so maybe im doing this wrong here.
End result of sf64:
starfox64.us.rev1.elf
starfox64.us.rev1.map
starfox64.us.rev1.uncompressed.z64
starfox64.us.rev1.z64

Created a simple toml:

[input]
entrypoint = 0x80000400
# Paths are relative to the location of this config file.
output_func_path = "RecompiledFuncs"
# relocatable_sections_path = "overlays.us.rev1.txt"
symbols_file_path = "starfox64.us.rev1.elf"
rom_file_path = "starfox64.us.rev1.uncompressed.z64"

Syntax error parsing toml: starfox64.us.rev1.elf (line 1, column 1):
Error while parsing root table: expected keys, tables, whitespace or comments, saw '\u007F'
Failed to load symbols file

Useage guide?

Can anyone help me with going from a rom file to things working? I am kinda confused on how to make an elf file

Android port compatibility?

While Windows and Linux are already ports you have focussed on (Majora's Mask for example being released) is this capable of Android porting also for handhelds that are very popular now from Retroid/Anbernic that SM64 Port and Harkinian can work on already?

Thanks :)

System requirments

Would be really nice if there was a Requirments list for running ports made with this..

Not as in cpu or gpu horsepower but what does it compile to require as far as libraries and API? instructions sets like MMX or SSE2?
Does it need win10/11? vbruntimes? is it a software renderer or opengl or dx...9?12? vulkan?

Would love to run some good old games on some Retro PC with win98se or XP on.

Makefile issue?

C:\Program Files\Microsoft Visual Studio\2022\Community>cmake --build C:\Users\charl\N64Recomp\build
make: Makefile: No such file or directory
make: *** No rule to make target 'Makefile'. Stop.

How to find entry point?

Is it possible to use a program to find the entry point in a n64 rom needed for the recomp to do its magic, or should i wait for the documentation? Cause i got the elf files ready, just finding the entry point isn’t really my cup of tea.

How to handle handwritten assembly functions

I'm trying to recompile Harvest Moon 64 and am running into an issue when the recompiler reaches the osInvalDCache libultra func (handwritten assembly).

Unhandled instruction: cache
Error in recompiling static_5_800FF3A0, clearing output file
Error recompiling static_5_800FF3A0

(0x800FF3A0 is where this function starts in VRAM)

I see there are manual_funcs and ignored values in the toml file, but I'm not sure if those are where to put handwritten assembly functions. When I tried to put the function in the manual_funcs list, I got this error:

Missing required value in manual_funcs array
Failed to load config file: us.toml

.

Shii. Sry guys I was very sleep deprived

Hook insertion

Building on top of #1, a mechanism is required to insert function call hooks into the recompilation output. This will be mainly used to add overlay load hooks into games.

Hooks should provide a function name and the VRAM address of the instruction to insert the hook before. Hooks meant to be placed at the end of a function can provide the VRAM value of the instruction following the last one in the function. Hooks that omit an instruction VRAM address will be inserted at the start of a function, which will likely be the most common use case for hooks.

How do i build this???

I need instructions on how to figure this out, as the one's given are super vague.

Issue with Clang and aggregate initialization

There are some versions of clang (probably Apple versions) that don't support aggregate initialization even if it supports C++20 (see issue). Let's create constructors so it'll work in those versions as well.

[Feature Request] Recomp to native GameCube ISO

A modded GameCube can barely manage to run most N64 games at 20 FPS, if at all. Imagine how cool it would be if we could recompile N64 games to native GameCube games for playing on a modded GameCube. I know this is a wild ask but, it would be sick! Maybe it could be done by including the GC homebrew dev stuff with this project and putting it in a Docker image? IDK, I can dream...

Subsystem (TransferPak, MemoryCard)

Hello there!

I have yet to read the source in full, but the first thing that immediately came to my mind was support for accessories such as the TransferPak for Pokémon Stadium. Sure, it'll be perfectly fine for single-player stuff, but the real fun starts with the addition of players' own cartridges and pokémon. So, I wanted to ask about compatibility with that.

This is a super neat project - digging into this source will be very interesting!

Kind regards,
Ingwie

Split Ocarina of Time recomp into a separate repo

Same deal as #5.

Missing jump label in decompile output

Hi, I'm trying out this tool by recompiling a demo from the official SDK (seemed like a better idea than a full blown commercial game)

[input]
entrypoint = 0x80200060
elf_path = "teapot.elf"
output_func_path = "RecompiledFuncs"

[patches]
ignored = ["gspF3DEX2_xbusTextStart"]  # ucode should be ignored

I got some ambiguity warnings, but the expected functions were found.

Sections
Num symbols: 903
Found entrypoint, original name: boot
Function count: 766
Working dir: /home/lizardman/projects/n64
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206570
  sqrtf_recomp
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206570
  sqrtf_recomp
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206570
  sqrtf_recomp
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206570
  sqrtf_recomp
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206570
  sqrtf_recomp
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206570
  sqrtf_recomp
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206570
  sqrtf_recomp
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206EA0
  gcc2_compiled._80206EA0
  gcc2_compiled._80206EA0
  guNormalize
[Warn] Potential jal resolution ambiguity
  gcc2_compiled._80206EA0
  gcc2_compiled._80206EA0
  gcc2_compiled._80206EA0
  guNormalize

The main problem is that the recomp_entrypoint is missing some things:

#include "recomp.h"
#include "disable_warnings.h"

void recomp_entrypoint(uint8_t* rdram, recomp_context* ctx) {
    uint64_t hi = 0, lo = 0, result = 0;
    unsigned int rounding_mode = DEFAULT_ROUNDING_MODE;
    int c1cs = 0; 
    // addiu       $sp, $sp, -0x20
    ctx->r29 = ADD32(ctx->r29, -0X20);
    // sw          $ra, 0x1C($sp)
    MEM_W(0X1C, ctx->r29) = ctx->r31;
    // jal         0x802051D8
    // sw          $s0, 0x18($sp)
    MEM_W(0X18, ctx->r29) = ctx->r16;
    __osInitialize_common_recomp(rdram, ctx);
    goto after_0;
    // sw          $s0, 0x18($sp)
    MEM_W(0X18, ctx->r29) = ctx->r16;
    after_0:
    // jal         0x802015B0
    // nop

    osCartRomInit_recomp(rdram, ctx);
    goto after_1;
    // nop

    after_1:
    // lui         $s0, 0x8021
    ctx->r16 = S32(0X8021 << 16);
    // addiu       $s0, $s0, -0x4570
    ctx->r16 = ADD32(ctx->r16, -0X4570);
    // addu        $a0, $s0, $zero
    ctx->r4 = ADD32(ctx->r16, 0);
    // lui         $at, 0x8022
    ctx->r1 = S32(0X8022 << 16);
    // sw          $v0, -0x14F4($at)
    MEM_W(-0X14F4, ctx->r1) = ctx->r2;
    // lui         $v0, 0x8021
    ctx->r2 = S32(0X8021 << 16);
    // addiu       $v0, $v0, -0x23C0
    ctx->r2 = ADD32(ctx->r2, -0X23C0);
    // sw          $v0, 0x10($sp)
    MEM_W(0X10, ctx->r29) = ctx->r2;
    // addiu       $v0, $zero, 0xA
    ctx->r2 = ADD32(0, 0XA);
    // addiu       $a1, $zero, 0x1
    ctx->r5 = ADD32(0, 0X1);
    // lui         $a2, 0x8020
    ctx->r6 = S32(0X8020 << 16);
    // addiu       $a2, $a2, 0xCC
    ctx->r6 = ADD32(ctx->r6, 0XCC);
    // addu        $a3, $zero, $zero
    ctx->r7 = ADD32(0, 0);
    // jal         0x802039E0
    osCreateThread_recomp(rdram, ctx);
    goto after_2;
;}

it's trying to jump to a undefined label after_2, but the the disassembly shows that it was supposed to call osStartThread with the idle thread address:

80200060 <boot> (File Offset: 0x50060):
80200060:	27bdffe0 	addiu	sp,sp,-32
80200064:	afbf001c 	sw	ra,28(sp)
80200068:	0c081476 	jal	802051d8 <__osInitialize_common> (File Offset: 0x551d8)
8020006c:	afb00018 	sw	s0,24(sp)
80200070:	0c08056c 	jal	802015b0 <osCartRomInit> (File Offset: 0x515b0)
80200074:	00000000 	nop
80200078:	3c108021 	lui	s0,0x8021
8020007c:	2610ba90 	addiu	s0,s0,-17776
80200080:	02002021 	move	a0,s0
80200084:	3c018022 	lui	at,0x8022
80200088:	ac22eb0c 	sw	v0,-5364(at)
8020008c:	3c028021 	lui	v0,0x8021
80200090:	2442dc40 	addiu	v0,v0,-9152
80200094:	afa20010 	sw	v0,16(sp)
80200098:	2402000a 	li	v0,10
8020009c:	24050001 	li	a1,1
802000a0:	3c068020 	lui	a2,0x8020
802000a4:	24c600cc 	addiu	a2,a2,204
802000a8:	00003821 	move	a3,zero
802000ac:	0c080e78 	jal	802039e0 <osCreateThread> (File Offset: 0x539e0)
802000b0:	afa20014 	sw	v0,20(sp)
802000b4:	0c080ee8 	jal	80203ba0 <osStartThread> (File Offset: 0x53ba0)
802000b8:	02002021 	move	a0,s0
802000bc:	8fbf001c 	lw	ra,28(sp)
802000c0:	8fb00018 	lw	s0,24(sp)
802000c4:	03e00008 	jr	ra
802000c8:	27bd0020 	addiu	sp,sp,32

I'm not really sure if this is really a bug (it's probably my fault). Should I set the entrypoint to something else? I wasn't sure if boot should be bypassed (it felt important because some functions are not stubbed in the mm runtime)