emustudio/emulib

Run-time library for emuStudio and plug-ins

Welcome to emuStudio

emuStudio is a desktop application used for computer emulation and writing programs for emulated computers. It extensible; it encourages developers to write their own computer emulators.

The main goal of emuStudio is to support the "compile-load-emulate" workflow, aiming at students or anyone to help to learn about older but important computers or even abstract machines.

emuStudio is very appropriate for use at schools, e.g. when students are doing first steps in assembler, or when they are taught about computer history. For example, emuStudio is used at the Technical University of Košice since 2007.

Available emulators

• MITS Altair8800

• Manchester SSEM

• Random Access Machine (RAM)

• Random Access Stored Program (RASP)

• BrainDuck (brainfuck interpreter)

BIG THANKS

emuStudio was written based on existing emulators, sites and existing documentation of real hardware. For example:

Projects:

• simh project, which was the main inspiration for Altair8800 computer
• MAME project, which helped with resolving a lot of bugs in a correct implementation of some 8080 and Z80 CPU instructions

Sites:

• David Sharp's SSEM site, main inspiration for SSEM implementation
• Esolang's BrainFuck site, main inspiration for Brainfuck implementation
• DeRamp Altair, more inspiration for Altair8800
• Altair Clone, more inspiration for Altair8800
• Study of techniques for emulation programming, emulation techniques classic
• 8080 instruction table

Discord:

• Discord Emulation Development

Getting started

At first, either compile or download emuStudio. The prerequisite is to have installed Java, at least version 11 (download here).

Then, unzip the tar/zip file (emuStudio-xxx.zip) and run it using command:

• On Linux / Mac

> ./emuStudio

• On Windows:

> emuStudio.bat

NOTE: Currently supported are Linux and Windows. Mac is NOT supported, but it might work to some extent.

For more information, please read user documentation.

Contributing

Anyone can contribute. Before start, please read developer documentation, which includes information like:

• Which tools to use and how to set up the environment
• How to compile emuStudio and prepare local releases
• Which git branch to use
• Code architecture, naming conventions, best practices

Related projects

There exist some additional projects, which are used by emuStudio, useful for contributors:

• emuLib - a shared runtime library
• Edigen - instruction decoder and disassembler generator
• Edigen Gradle plugin - Edigen Gradle plugin
• CPU testing suite - a JUnit-based test suite for comfortable testing of CPU plugins
• emuStudio website - emuStudio website

emulib's Issues

Add support for raw interrupt signals inside CPU context

Currently, the CPU context interface supports only "high-level" interrupt handling, by calling one of the functions:

signalInterrupt(DeviceContext context, int mask);
clearInterrupt(DeviceContext context, int mask);
isInterrputSupported();

However, some interrupts can pass "raw" data to the CPU, which can be represented for example as an instruction (as it is in 8080 case). Currently, the raw interrupts are implemented only in 8080 CPU extended context.

Add exceptions to all IO methods in DeviceContext

API should allow the user of the API to detect whether the I/O operations failed or not. Currently, read returns the read content, write is void...

Deploy emuLib to Maven Central

It will be better accessible

Add methods for getting the context to ContextPool without interface parameter

Many times, plugins just obtain standard context. It is annoying to always specify the standard interface, such as:

MemoryContext memory = getMemoryContext(..., MemoryContext.class);

when it is clear that MemoryContext is standard interface.

During plug-in loading, load also all plug-in dependencies

The dependencies are specified in the manifest file, in the Class-Path item. This is needed for example when a plug-in implements context interface defined in another plug-in.

Resolve two versions of disassembler interfaces

After pull request #8 there exist two ways of disassembler creation. The ways are related to disassembler/decoder main interface and CPU instruction class.

The first pair is IDisassembler main interface, and CPUInstrucion class representing disassembled instruction. The second pair is IDecoder main interface and DecodedInstruction class for decoded CPU instruction.

There is a need to resolve the problem and use only single pair.

ContextTest randomly fails

Unit test emulib.runtime.ContextTest tests a singleton class Context. The success or failure of the method testContextRegistration() depends on the order of test method execution, but JUnit does not guarantee any specific execution order.

If testPasswordAssign() is called before testContextRegistration(), the result is success. Otherwise, the result is failure. This is because the method testContextRegistration() depends on the fact that Context.assignPassword(password) was already called.

Implement support for edigen optimizations

• Move here formatter
• Implement Bits (emustudio/edigen#49)
• Implement more number utils, bits reading (emustudio/edigen#49)
• ...

Get rid of emuLib logger

rather use slf4j api.

When user runs emuStudio in automatic mode, StaticDialogs are still displaying GUI dialogs.

Info/error messages should be logged in that situation and input dialogs should be ignored for now. In automation process we shouldn't ask for anything.

Add generics type for 'words' in memory contexts

Currently, only byte value type is specified as generics, but not word type. The final interface should look like this:

interface MemoryContext<ByteType, WordType> {
    ...
}

Ensure backward usability

Recognize and enable to execute virtual machines designed with older emuLib or emuStudio. This issue is related to emuStudio issue emustudio/emuStudio#8.

The best solution to preserve correct functionality is to use OSGi framework.

Move computer managing and emulation to emuLib

emuStudio should become just a front-end to the emuLib. Then there can coexist many front-ends, either command line or gui based.

Allow logging events to plug-ins and emuStudio

The preferred system for logging is log4j. It supports 4 levels of logging: INFO, WARN, ERROR, and DEBUG. Each plug-in should have opportunity to log own events.

Add NumberUtils class

• reversing bits
• reading / writing integers on various strategies (little/big endian, and with bits reversal)

Refactor exceptions

Allos PluginInitializationException to have not set plugin

Emulation automation

Enhance API so the plugins report whether they support automation or not. I imagine this as

interface Plugin {
  ...

  boolean isAutomationSupported();
}

Add support for TZX and TAP file format

https://documentation.help/BASin/format_tape.html

it should be similar as intel hex

NumberUtils should be extended

NumberUtils contain interesting set of functions to reduce Java boiler-plate. It should be extended with the following:

• Byte[] numbersToBytes(Number[] numbers)
• byte[] numbersToNativeBytes(Number[] numbers)
• Byte[] shortsToBytes(Short[] numbers)
• byte[] shortsToNativeBytes(Short[] numbers)
• short[] shortsToNativeShorts(Short[] numbers)
• byte[] nativeShortsToNativeBytes(short[] numbers)
• Byte[] nativeShortsToBytes(short[] numbers)
• Short[] nativeShortsToShorts(short[] numbers)
• Byte[] nativeBytesToBytes(byte[] array)

Move internal classes to "internal" package and start using semantic versioning

emuLib is already using some form of semantic versioning, but this issue should make it even more concise. See http://semver.org

Add support for String radix parsing

1. Hexadecimal, of the format 0[xX]([0-9a-fA-F])+, or ([0-9a-fA-F])+[hH]
2. Decimal, of the format [0-9]+, or [0-9]+[dD]
3. Octal, of the format 0[0-9]+, or [0-7]+[oO]

There should exist a class for parsing or converting these numbers.

Remove emails of developers from all source codes

Current comment style is:

/**
 * SomeFile.java
 * 
 * (c) Copyright 2012, N. Surname <[email protected]>
 * 
 * KISS, YAGNI, DRY
 *
 * ...license...
 */

The emails should be removed from all source codes and added to AUTHORS file, located in the root directory, in format:
nsurname at somemail.com. Advantages are:

1. They won't be found by spammers
2. If some mail changes in time, there would be single place to do it.

This would still preserve informations about author per source code, while reducing risk of abuse.

Change IOException to RuntimeException in device context

IOException forces a user to catch it all the time, and this is especially annoying when in lambdas. RuntimeException will solve the issue

Lexical analyzer should be refactored

What we really need from lexical analyzer is:

• get next token
• is at EOF ?
• iterator capability to support Java forEach construct

Memory should be able to read/write various amount of cells at once in one API call

Currently memory cells can be written one-by-one or as a "word" (= 2 cells). Why? There should exist just one API for r/w as many cells as we want. It will be useful also for compilers, when transferring compiled code to memory.

Add build CI service

After any push, automatical build should be executed and report the status to GitHub. GitHub supports labeling of commits with statuses and Travis CI (http://travis-ci.org/) can use it for free.

Implement support for accurate sleeping

LockSupport.parkNanos() is more precise than Thread.sleep(), but works accurate enough only on Linux, not on Windows. Unfortunately we need some reliable and accurate sleeping algorithm which should be used anywhere where accuracy is required. See emustudio/emuStudio#181 for more details

Change own implementation of ClassLoader with URLClassLoader

The URLClassLoader has much more freedom of loading classes and it's native implementation in Java.

Rewrite pluginLoader to correctly load plugins

As it seems, it is not possible to modify URLClassLoader in runtime (add new URLs), even if a new class loader extends it and calls protected addUrl() method.

I figured out this from a try to write a unit test for it. In order to work it out, all plugins for the virtual computer must be loaded at once, when creating new instance of the URLClassLoader (passing all plugin URLs as parameter).

This is good, because it allows to separate class loaders for different computers and will allow to implement changing virtual computers at runtime.

Enhance API for CPUs to have custom debug columns

For example, some CPUs, like RAM machine, uses labels in the debug table. And it looks horrible. It would be therefore very good if any CPU could customize its own debug columns.

Also, add default debug columns to emuLib.

This might violate emustudio/emuStudio#90, but for now it makes perfect sense.

Fasten DecodedInstruction

Instead of HashMaps use TreeMaps

Move away from plugin annotations to configuration

Java annotations are easy for use by plugin programmer, but it has also few disadvantages, like emuStudio must "search for" annotated classes at runtime. Having everything in plugin configuration file decreases plugin development comfort a bit, but adds many positive features, like:

• Ability to describe the plugin offline, including context requirements or availability.
• plugin meta-data will be available in computer editor, before emulator starts.
• emuStudio won't need to search for annotations anymore, every class can be specified directly.

Suggested format is either yaml, toml, or hocon. Currently emuStudio utilizes toml.

Better interaction with plugins in UI

Currently, regarding UI - plugins can have their own UI windows. In addition, a compiler can compile directly into a memory and CPU controls debug panel. I think it might be interesting to let plugins influence other UI controls, like:

• A plugin can be used to open/save files from "virtual" filesystem (e.g. MITS-88-DCDD could open files from CPM image into editor and save it into the image)
• CPU plugin could "decompile" a memory range into source code editor
• etc.

Add BCD code conversion to NumberUtils

Packed BCD code would suffice for now (2x4 digits per byte)

Add resetInternal method to AbstractCPU

Currently, all CPU plugins must override reset() method in AbstractCPU, even if this method is not abstract and after resetting internal state, call super.reset().

This is a big flaw in behavior since if the CPU was running, resetting its internal state will not "stop" it, it will be stopped after calling super.reset().

This can be prevented if the plug-ins don't have to override reset directly, but they will rather implement e.g. resetInternal() from AbstractCPU.

Start using gradle-nexus publish plugin

https://github.com/gradle-nexus/publish-plugin/

Add independent logging support for plug-ins

Plugins that want to log an information must have their own logging mechanism. EmuLib should provide one, independent and unique mechanism which allow plug-ins to log information easily.

Create more tests in emuLib

The emuLib JUnit tests do not cover the most important scenarios. They need to be tested. The scenarios will be written as comments (and then implemented in commits) of this issue until the author would think it is enough.

Get rid of context HASH

They were introduced because of preserving security - to preserve that classes with the same name in plug-ins can be misused. However, they introduced very unmanageable complexity when creating plug-in contexts. The new mechanism should somehow preserve the previous requirement, but in some other way.

Change way of loading plug-ins - allow to load all plug-ins at once

This is needed because of emuStudio issue no.26

NumberUtils.readBits behaves wrong

Calling it with:

• LITTLE_ENDIAN strategy, it should read bits from LSB to MSB
• BIG_ENDIAN strategy, it should read bits from MSB to LSB

Edigen decoder is reading bits from left to right according to the grammar file, thus using big endian strategy. So, expected is:

input = 0b1111101010101111 (big_endian=0xFAAF, little_endian=0xAFFA)
start = 5
lenght = 8

byte[] word = new byte[]{(byte) 0xFA, (byte)0xAF, 0, 0};

• little endian:

//  expected = 01111111 (=0x7F)
int bits = NumberUtils.readBits(word, 5, 8, Strategy.LITTLE_ENDIAN);
assertEquals(0x7F, bits);

• big endian:

//  expected = 01010101 (=0x55)
int bits = NumberUtils.readBits(word, 5, 8, Strategy.BIG_ENDIAN);
assertEquals(0x55, bits);

Add description of contracts in the Javadoc

emuStudio behavioral contracts became defacto standard way how a plugin developer knows how the plugin should be implemented. The most intuitive way how to get to those contracts is through javadoc, when it is used.

Document all technical details for developers

There should exist 2 types of documents:

• manual (html, pdf)
• JavaDOC

Get rid of nulls

Some API methods return nulls, e.g.: If plug-in doesn't have the permission to access it, return null in ContextPool.

Instead of returning nulls in such cases, execption should be thrown.

Move from Maven to Gradle

Enable of loading dependencies for plug-ins or defining general class-path.

There are lots of problems when plug-ins use own third party libraries. They are not loaded and plug-ins are dependent on those libraries loaded by emuStudio or emuLib. There should exist a mechanism, probably a directory where class-path is pointing to, which would allow to load those libraries by current PluginLoader automatically.

Note: The PluginLoader is derived from application class loader, so we can probably already load several computers at once.

Plugin incompatibility caused by new exceptions

Probably all plugins are now incompatible with emuLib 9.0 because the method Context.register() throws new types of exceptions (AlreadyRegisteredException, InvalidImplementationException, InvalidHashException) which are not caught in the plugins.

Are we going to wait for an OSGi-enabled emuLib / emuStudio (Issue #3), or fix the incompatibility in all plugins?

Move emulation control from emuStudio to emuLib

In future, emuStudio should be only a GUI front-end. Direct emulation control should be moved to emuLib. There are more reasons for this, but the main reason is to preserve Single Responsibility Principle (SRP).

Main advantages of this approach are:

• GUI front-end can be programmed and managed independently,
• There will be better support of automatic and "shell" emulation

Support other computer architectures than von Neumann

There exist various architecture types that are used. Besides uniprocessor von Neumann model there exist also uniprocessor Hardvard architecture, and multiprocessor architectures. emuStudio/emuLib should allow developing and using such architectures. For now, it would be enough to support only uniprocessor ones.

Convert project files to maven3

Maven is great tool for managing projects and supports automated building, dependences resolution and much more.