tornupnegatives/tms-express

LPC speech encoder for the TMS5220

tms-express's Issues

Address Qt-Related Memory Leaks

Summary

TMS Express is a learning project for me. One new skill I have begun developing through it is GUI development, specifically using Qt. I am still unfamiliar with the odds and ends of managing memory when using Qt, so v2.0.0 is leaking quite a bit of memory. These leaks must be identified and fixed.

Note

According to some seemingly knowledgable people on the Qt forums, Qt may allocate globally acquired resources which are freed automatically on program termination but will still make Valgrind upset. As long as there are no user-created leaks, those can (and will) remain.

Source: https://www.qtcentre.org/threads/14880-Memory-leaks-in-QT-apps?p=76242#post76242

Waveform Beautification

Summary

• The waveforms should be updated to be more accurate, perhaps with fixed scales for amplitude and frequency units for pitch
• The waveforms should be beautified slightly, with improved line quality and colors

Decoding?

Hi, are there any plans for a decoder?

Code Style Refactor

Ensure conformity with the most important aspects of Google's C++ style guide.

This project started as a personal toy to learn C++. However, as the codebase gets larger, I would like for it to reflect the practices and style I've picked up from both hobby exploration as well as professional experience. It is a long and largely useless endeavor from a functional standpoint, but worthwhile nonetheless.

Provide Pre-Built Binaries

Originally, the intent was to provide GitHub Actions workflows which would automatically generate artifacts. Unfortunately, the process is less straightforward than expected. It it possible that this solution will be revisited, but for now build scripts should be provided instead, assuming a clean install for each operating system.

Generate C headers for embedded development

Description

The eventual goal of TMS Express is to serve a one component in an embedded system for realtime LPC processing with a TMS5220. As such, it will be useful to include functionality to export bitstreams in a C-friendly format. This will also allow for compatibility with the Talkie library for Arduino, which is quite popular.

Solution Space

The ability to prefix hex bytes with a 0x is already built-in. An additional function should be added to the FrameEncoder which wraps the bitstream in a C-style struct or header.

Batch Encoding

Summary

• Create a new “Batch Job” window, which can be populated with a table of filenames and processing parameters
• Selecting a row of the table will display the file and the parameters in the main editor
• Allow saving to individual LPC files, individual BIN files, or a C header
• Add to CLI

Removed Deprecated Functions

Semi-related to #40, the AudioBuffer::reset() and FramePostprocessor::reset() functions encourage poor coding practices, and should be done away with.

TMS6100 Batch Encoding

Summary

• Support the generation of TMS6100 Voice Synthesis Memory images for use with original hardware. This functionality may be baked into the Batch Job window
• Add to CLI

Revise CLI Functionality

Summary

Update the CLI to reflect the latest changes in structure and functionality. This may involve re-writing or removing the BitstreamGenerator class. If it is maintained, it is best to utilize it in both the CLI and GUI to avoid redundancy

Catch Exceptions in GUI

Certain classes, namely the AudioBuffer, may throw runtime exceptions. The GUI should catch these, display a dialog to the user, and then halt execution.

Command line should support flags

Description

There are currently two command-line options which store boolean values: include-prefix and include-stop-frame. However, they require the user to supply them with an argument (true/false). Ideally, the user can simply include the flags with no argument to set them as true.

Current Behavior

$ TMS-Express --input input.wav --output output.lpc --include-prefix true

Desired Behavior

$ TMS-Express --input input.wav --output output.lpc --include-prefix

Solution Space

The boost library certainly supports the desired behavior. It must simply be discovered and implemented.

Maximum gain destabilizes bitstream

Description

When a Frame's gain is the maximum value of the Tms5220CodingTable, the signal may become unstable or silent

Workaround

An easy workaround is to prevent the gain from ever becoming too high. This would involve ceilings in the Frame::setGain(), Frame::setQuantizedGain(), and FramePostprocessor::shiftGain() functions.

Solution Space

It would be interesting to figure out why this happens rather than restrict the gain values.

Input Validation

Summary

• Determine reasonable ranges for all inputs, and enforce them by changing the color of invalid inputs and/or disallowing further action until they are rectified
• Update post-processing sliders such that their limits and increments dynamically updated based on the frame table, such that the bitstream may not be de-stabilized
• Add to CLI

Save Formats

Summary

• Update the save dialog with specific format options
• The LPC file format should allow the user to select whether or not a hex prefix (“0x”) is included before each byte. The separator should also be customizable within limits, so that TMS Express can still easily open them back up
• The C header file format should allow the user to select whether or not to include the Arduino.h library and PROGMEM macros, for use with the Talkie library
• Add to CLI

Feature Request - Export subcommand

Hi there! I've been playing around with your project for a few days now. It's very neat! I've been poking around seeing what I could do with it.

I noticed you had feature in the GUI called "Export" that allows a user to save a WAV file made by TMS-Express' built-in emulator.

I was wondering if it would be possible to add that same functionality when invoking TMS-Express from the command line? Having a subcommand to output a WAV instead of a bitstream seems like it would be really handy! In fact, after taking a few minutes to look around, I think that would make this not only an encoder, but possibly one of the only standalone LPC emulators around that doesn't require an Arduino.

De-Couple Classes

At present, there are several instances of classes which hold instances of other classes-namely holding AudioBuffer objects. This is confusing, and should be amended.

LPC Buffer Resolution Settings

Summary

• Allow populating the LPC buffer with either 16-bit integer or 32-bit float samples, which may impact audio quality
• Add to CLI

Improve Documentation

Ensure all functions have a detailed doc comment, and that these comments are consistent and uniform.

Generate ROMS for a TMS6100 (emulator)

Description

Currently, TMS Express can generate a single bitstream from a file, which can drive a TMS5220 Voice Synthesis Processor in "Speak External" mode. However, there is another way to drive the device: via a TMS6100 Voice Synthesis Memory IC (datasheet). It would be useful for TMS Express to encode several files in general: into a C header for Arduino talkie, per #5; and into a binary file that could be flashed to a TMS6100 or equivalent device.

More realistically, these binary files would end up on an FPGA/MCU/CPLD clone of the TMS6100, which could be used to restore failing arcade machines which use the TMS5220, breathe new life into a dusty BBC Acorn, or add new words to a Speak and Spell—for example.

Solution Space

The datasheet linked above as well as this webpage by @furrtek have enough information to go off of. It seems that the issue can be broken down into the following milestones:

1. Encapsulate all of TMS Express' current functionally into an encode command, which handles a single file
2. Add a batch command which allows users to encode several files at once
3. Add a rom command which builds upon batch to configure and store a binary file for use in a TMS6100 equivalent. This will likely require Verilog clone of the TMS6100 for testing

If anyone out there is interested in collaborating on this with me, don't hesitate to reach out. Otherwise, I'll be tinkering away, one step at a time ;)

Missing Include for FrameEncoder

note: ‘std::bitset’ is defined in header ‘<bitset>’; did you forget to ‘#include <bitset>’?

Any chance of a prebuilt windows binary or possibly a windows build process?

I started to strip out the external libraries and then thought that it would be better if others could have a windows build and any updates.

Repeat frame detection

Description

Because the human vocal tract changes rather slowly, it is likely that adjacent Frames are similar enough that they do not each require their own set of parameters. Instead, the first one maintains a full parameter set, and subsequent Frames are marked as repeats and contain only energy and pitch.

Solution Space

The FramePostprocessor should compare the reflector coefficients of all Frames and determine whether or not to classify them as repeats. Criteria must be established to determine what constitutes Frame similarity.

Address Compiler Warnings

Increase the compiler warning level (-Wall -Wpedantic -Wextra...) and ensure code is clean

MAME TMS5220 Emulator?

Hi, the MAME project has a much more accurate TMS5220 emulator.

Do you think it would be possible to integrate MAMe's TMS5220 emulator into your project?

Bitstream Statistics Window

Summary

• Add “Bitstream Statistics” window to display interesting metadata such as frame count, average gain/pitch, voicing ratio, number of repeat frames, etc.
• Add to CLI

Updated Estimation Algorithms

Summary

• Experiment with alternative methods for solving the Toplitz equations
• Implement YIN pitch estimator with voicing score
• Add to CLI

Windows Builds?

Hi, are there any plans to include Windows builds in the releases?

fatal error: 'experimental/filesystem' file not found

When I am installing using Homebrew on MacOS 13.4, the build fails with the following error:

==> Installing tornupnegatives/tmsexpress/tms-express
==> cmake -S . -B build
==> make -C build
Last 15 lines from /Users/michael/Library/Logs/Homebrew/tms-express/02.make:
make[2]: *** [CMakeFiles/TMS-Express.dir/src/Interfaces/BitstreamGenerator.cpp.o] Error 1
/tmp/tms-express-20230611-22591-1tgnyn4/TMS-Express-1.1.2/src/main.cpp:6:10: fatal error: 'experimental/filesystem' file not found
#include <experimental/filesystem>
         ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/TMS-Express.dir/src/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/TMS-Express.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 48%] Linking CXX static library ../../../lib/libgtest.a
cd /tmp/tms-express-20230611-22591-1tgnyn4/TMS-Express-1.1.2/build/_deps/googletest-build/googletest && /usr/local/Cellar/cmake/3.26.4/bin/cmake -P CMakeFiles/gtest.dir/cmake_clean_target.cmake
cd /tmp/tms-express-20230611-22591-1tgnyn4/TMS-Express-1.1.2/build/_deps/googletest-build/googletest && /usr/local/Cellar/cmake/3.26.4/bin/cmake -E cmake_link_script CMakeFiles/gtest.dir/link.txt --verbose=1
/usr/bin/ar qc ../../../lib/libgtest.a "CMakeFiles/gtest.dir/src/gtest-all.cc.o"
/usr/bin/ranlib ../../../lib/libgtest.a
[ 48%] Built target gtest
make: *** [all] Error 2

If reporting this issue please do so at (not Homebrew/brew or Homebrew/homebrew-core):
  https://github.com/tornupnegatives/homebrew-tmsexpress/issues

Eliminate Buffer Pointers

Summary

As of v2.0.0, the AudioBuffer objects and Frame vector (frame table) owned by the MainWindow are pointers. While this makes initialization of the MainWindow easy, ideally pointers (and references) should be used sparingly.

Actionable Items

• Remove the use of pointers entirely. Although this might result in more memory usage, it spares the headache of debugging use-after-free, null dereference, etc.
• Reconsider the use of references. Having to store the original copies of buffers inside objects is odd, and it’s not always clear what copy of a buffer an object holds

TMS5220 Energy Quantization

The process of encoding bitstreams for the TMS5220 involves LPC analysis and then frame packing. During frame packing, parameters found during LPC analysis are quantized to their closest value in the TMS5220 coding table.

For pitch period and the LPC reflector coefficients, this is pretty straightforward. However, I am having a difficult time figuring out how to handle the energy parameter. Computing energy as the sum of squares of the audio signal results in a value between zero and one, however the TMS5220 coding table contains only large integer values.

Does anyone know the algorithm used to compute/translate energy for the TMS5220?

Bitstream Table Window

Summary

• Create a new “table view” window, which displays the properties of each frame in an editable table

Pitch Interpolation

Summary

• Implement post-processing pitch interpolation, to smoothen the pitch curve of synthesized speech
• Add to CLI