[Question] Interested in LCDd support?

[munt]

[Question] Interested in LCDd support?

[dwhinham]

Or might be mt32-pi font could be double-licensed somehow to make it possible to add to libmt32emu?

To be honest I hadn't even thought of the license; of course GPL is no good for the library. I have no issues whatsoever with you using the content of font6x8.h under whatever license suits libmt32emu best.

I hereby explicitly grant a public license to use the contents of font6x8.h from the mt32-pi project, either in part or in full, under the terms of the GNU Lesser Public License.

If this isn't good enough I'll update the header of this file when I get back to my computer.

[vintagepc]

Tell you what, I'll push my changes to a repo so we can do a compare and if you like the core of what you see I'm happy to spend some time polishing the rough edges :) That should be mostly just more graceful error handling as for my case I could safely assume LCDProc would always be present and take some shortcuts for the purposes of getting it working. :)

[sergm]

Right, we might include LCD support via lcdproc as an optional dependency, but just one of many, so that other backends can be easily used on other platforms to communicate the data to hardware or simulated LCDs. No reinventions I have in mind (like rewriting lcdproc, heh), it's all about interfaces and basic implementation of stuff specific to MT-32 to some degree.

[sergm]

@dwhinham That's superb! Although, when this file gets settled in libmt32emu, you might want to eliminate the code duplication 😉

[sergm]

It depends on the way you've implemented it.

Several LCD-related issues come up in different forms lately, that probably implies interfacing the synth with different LCD supporting libraries, to show the virtual MT-32 LCD screen either on a some hardware LCD or a LCD software emulation. If it comes to connecting an application to an external LCD, I realise that the current rather low-level support for the LCD emulation in the mt32emu engine isn't great. It'd be advantageous to put some core functions to the mt32emu library and introduce a new simpler abstraction, so that the application author may implement communication with a hardware / software LCD easier. At least, I'm not sure that lcdproc is the library of choice for everyone. This way we keep the mt32emu core library free of non-essential dependencies, yet enable applications to show the MT-32 LCD content with many inherent features.

[vintagepc]

Oof, this code isn't very elegant at all and embarrassingly bad 🤣 I have become a LOT more familiar with Qt since I did this several years ago so suffice it to say that I will definitely be spending some time refactoring pieces to be cleaner and less hack-y.

Here's what I have so far:

https://github.com/munt/munt/compare/master...vintagepc:munt_lcdproc?expand=1

I'm happy to file a draft PR if it's easier for you to comment inline but at this point I'm mostly just looking to see if there are high level/architecture/layout changes you'd like to suggest since it's not my project and I should be following whatever established preferences there are. Again, I'm happy to do the actual work, but I just want to make sure what I plan to do will align with the long term goals for the project so it's easier to maintain, as opposed to changing it up only to find out it needs more refactoring. 😄

I'll pre-emptively say if we want to target more than just lcdproc I already think the LCD handling should become its own interface class as opposed to my butchery of the current code. Not sure if we'd want one class per size and backend, or just handle the layout configuration all in the same file, with one derived class per backend. For the most part the changes depending on LCD size are not that complex and just formatting of the same information. For example:

• 20x4: fine as is, maybe change the top title though.
• 20x2: Lose the title and ROM name lines.
• 16x4: Lose the title blocks/whitespace and "vol:" bit (or shorten last line to MIDI Msg - [vol] or so
• 16x2: Combo of the previous two.
• 16x1 or 20x1: Just the channel indicators like the single line display in the application itself.

(Volume is kinda less useful on PC than on a headless system - I had wired up a rotary encoder to be able to adjust the volume on it when this was for a Pi)

[sergm]

Well, I'm afraid this is not it. There is a similar hacked mt32emu-qt version to support a Logitech LCD made by @HunterZ, and I personally think we should invent something better, because the demand for this task rather grows in time.

Additionally, I plan to move some functions to the library to simplify implementation of the MT-32 LCD emulation in other applications, without necessity to run mt32emu-qt. For example, it might be a DOSBox plugin, a MIDI player and even a bare metal kernel, see e.g. mt32-pi, etc. And yes, ideally we should have an abstraction, something like SampleRateConverter class that communicates with different SRC libraries, so we might implement several adaptors and/or interface with a LCD directly.

[vintagepc]

OK - sounds like there is some preliminary work and reorg you'd like to do before we commit to a new design for this feature.

Obviously still happy to contribute since I have an end use for the changes, but if I'm reading this right it sounds like the LCD support will remain on the back burner for a bit then :)

At least with respect to LCDProc there is a curses output driver which means it's not strictly necessary to have real LCD hardware to do the implementation and maintain it going forward.

[HunterZ]

My hack is definitely a messy proof-of-concept, as I had to sprinkle code all over the place to get the data out.

It would definitely be nice to see the code refactored into some kind of proper API for cleanly interfacing with display solutions.

Personally I don't think Munt should need to concern itself with presentation. It would almost be enough to just provide the display data as a string, maybe along with an ability to register for callbacks that notify when it changes.

I say "almost" because there are two extra bits to deal with: The MT-32 sometimes displays filled-rectangle characters, and there's also the "MIDI Message" LED. The LED state can probably just be exposed as a boolean value (again, maybe with a change callback, or just use one callback for everything). The filled rectangles could be represented in the display string as an ASCII character that isn't in the MT-32's character set, or separately as a bitmask integer or array of booleans.

[sergm]

Personally I don't think Munt should need to concern itself with presentation. It would almost be enough to just provide the display data as a string, maybe along with an ability to register for callbacks that notify when it changes.

I say "almost" because there are two extra bits to deal with: The MT-32 sometimes displays filled-rectangle characters, and there's also the "MIDI Message" LED. The LED state can probably just be exposed as a boolean value (again, maybe with a change callback, or just use one callback for everything). The filled rectangles could be represented in the display string as an ASCII character that isn't in the MT-32's character set, or separately as a bitmask integer or array of booleans.

Yes, albeit we might also consider inclusion of a simple renderer that utilises an external font, not sure. I gather, those "filled-rectangle characters" aren't quite common in existing LCD representations, one likely has to hack the font for that. But overall, mt32emu does provide all the info you describe, just at a lower level, requiring some further processing by each and every applications, which is not good 😒

Anyway, I'll look into it and try to come up with a sane API shortly...

[dwhinham]

The filled-rectangle character is part of the LCD character ROM.

The MT-32 uses the Sanyo DM2011, and like all HD44780-compatible LCDs using the "A00" character set, it is represented by character code 0xFF. The following table is from the DM2021 datasheet (which is the 2-row version of the DM2011; I couldn't find a copy of the DM2011 datasheet); you can see the filled rectangle in the bottom-right:

The style of the characters in HD44780 displays varies slightly between manufacturers - I did some work to create a pixel-perfect version of the Sanyo DM2011's font so that when rendering out the MT-32 status line to a graphical OLED display in mt32-pi, you get an authentic representation:
https://twitter.com/_d0pefish_/status/1316443589287776256

The font can be found here, feel free to use it for whatever purposes:
https://github.com/dwhinham/mt32-pi/blob/master/include/lcd/font6x8.h

Note that I didn't bother with the upper half of the character set (with the Japanese kana characters) because they're not really needed, it's not possible to send these characters to an MT-32 via SysEx (they'd be interpreted as SysEx bytes), so I just placed the full block character at the end of the array. The blank CGRAM area used for custom characters at the beginning of the character set is also omitted in this font.

[HunterZ]

Good point regarding the character set.

I suppose it would be friendly to provide some means for library users to be able to get a pixel-perfect representation of the emulated LCD state. If I were going to implement this, I might do it by providing an array of character code bytes for the current LCD state, combined with separate APIs for performing static lookups of pixel and/or ASCII representations of a given character code.

[vintagepc]

FWIW I have an HD44780 simulation I've written for a different project of which we might be able to leverage for the draw routines or character data, if we want.

https://github.com/vintagepc/MK404/blob/master/parts/components/HD44780GL.cpp

[sergm]

The filled-rectangle character is part of the LCD character ROM.

Can it be easily rendered on every LCD out there? 😃 I'd still expect some compatibility issues should one route raw text out from the libmt32emu straight to lcdproc or the likes. Perhaps, only graphical output from the libmt32emu (using the proper font for rendering characters) will enable us to show the emulated display correctly in a common case. Or some character mapping might be needed, dunno. Any idea?

The font can be found here, feel free to use it for whatever purposes:
https://github.com/dwhinham/mt32-pi/blob/master/include/lcd/font6x8.h

Yes, that's nice, but the license is a bit incompatible....

The blank CGRAM area used for custom characters at the beginning of the character set is also omitted in this font.

Hmm, how does it render the "pipe" character that separates the master volume in the main display mode? It looks like that's a special character 0x02, or have I missed something?

[sergm]

More to it, the MT-32 control ROM is likely not using character 0xFF at all. AFAICT, it's yet another special character 0x01 what we really see as the filled-rectangle.

[sergm]

Anyway, my current plan is to move some rendering stuff to libmt32emu, so that mt32emu-qt will merely receive a callback "LCD updated", retrieve the actual text content and call a libmt32emu rendering function to get a bitmap (which is useful for any hardware graphical LCD screen). The font we use for rendering atm is to be improved to match DM2011 better, yet to include all relevant special characters along the way. Yet the bitmap rendering should also emulate the pixel grid (as mentioned in #50) for added accuracy 😃

Not too sure about enabling support for external LCDs in mt32emu-qt though. Should we have a proper plugin interface? Or do you guys feel more convenient with recompiling it to add support for some (there will be a convenient interface for ad-hoc LCD drivers anyway)?

[vintagepc]

The pipe character is present in the standard HD44780 rom at 0x7C. Just about every char LCD driver out there uses the standard HD44780 protocol and has the same character set, including the more modern KS0076 that most blob-on-board displays these days use.

For my mod I ended up just checking the masked char array and chucking an 0xFF over the connection to LCDProc, it worked fine. One could also utilize the ICON_FILLED_BLOCK feature instead to draw blocks, if desired.

To be honest I'm not sure if it's over engineering to make an extensive plugin system for a per-lcd interface. LCDd uses a local socket to communicate so you can easily build in support all the time and just disable it if the service is not found, and it supports a large number of displays and connections out of the box. Have not dived into its graphic LCD handling.

But we should definitely consider how to handle varying layouts and display sizes. As noted we could probably support the full range of known char display sizes in a single class by just using subsets of all the info; but this could get very messy if we later need to do something that is completely orthogonal in behaviour. I might be inclined to make a primary backend interface which receives the data and then inheriting classes whose sole task is just producing a correct layout for a given display size.

These could be extended for graphical displays to also be responsible for the pixel management.

On the Windows side the main utility there is LCDSmartie. Not sure how it communicates, it's been ages since I used it.

[sergm]

The pipe character is present in the standard HD44780 rom at 0x7C. Just about every char LCD driver out there uses the standard HD44780 protocol and has the same character set, including the more modern KS0076 that most blob-on-board displays these days use.

Right, there is a "pipe" character 0x7C, but it looks a bit differently to 0x02 of MT-32 (the line isn't contiguous). mt32emu-qt uses '|' exactly for that, and the character that is contained in the built-in font (coming from LCD4linux, btw) is "hacked" to look closely to the one MT-32 displays.

To be honest I'm not sure if it's over engineering to make an extensive plugin system for a per-lcd interface. LCDd uses a local socket to communicate so you can easily build in support all the time and just disable it if the service is not found, and it supports a large number of displays and connections out of the box. Have not dived into its graphic LCD handling.

Okay, locking the cross-platform qt app (not mentioning the library) down to use lcdproc doesn't seem like a good idea. Let's just have a simple interface to permit adding support for any LCD imaginable without the plugin system for starters, it may be introduced some time later if appears really needed, I guess.

Speaking of character-based vs. graphics-based LCDs, I definitely want to support both. As character-based LCDs seem to be more a common hardware, we surely need that. Perhaps, we leave fonts and mapping out of question, so the LCD driver implementation will handle that (likely with ease). But I also want to use the same interface that supports graphical LCDs in mt32emu-qt for the emulated LCD widget. So IMO, both are needed.

[vintagepc]

Maybe it's a terminology thing but I don't see using a loosely coupled socket interface as "locking" us in to a hard dependency; the headers to interface seem fairly lightweight and for platforms where LCDd is not supported it shouldn't be too hard to determine this and just not attempt to use the interface.

I'm also not saying we shouldn't have an abstraction to interface different backends for actually talking to the LCD hardware - I think I read into it too much as thinking the intent was to reinvent the entire stack of driver layers and handle talking directly to modules "ourselves"

[sergm]

Will start with #66, then update mt32emu-qt.
Sadly, I can't find an LCD font to add as a fallback to the library, which is supposed to be compatible with LGPL 2. @dwhinham could you please suggest any? Or might be mt32-pi font could be double-licensed somehow to make it possible to add to libmt32emu?

[vintagepc]

Not an expert on license compatibility but if that falls through we can talk about something similar for https://github.com/vintagepc/MK404/blob/master/parts/components/hd44780_charROM.h or even make one ourselves if we have a bitmap of the charset, from e.g. the display datasheet

[dwhinham]

RE: the "broken" pipe, I always thought that's just how "|" looked in the DM2011 charset. The datasheet snippet I linked above isn't quite accurate - it also gets characters like "p" and "q" wrong.

When I wrote that font I basically just SysEx'd a load of ASCII at my MT-32, took closeup photos and plotted the pixels by hand. '|" resulted in the broken pipe, so unless the MT-32 ROM is doing something crazy like remapping "|" to a custom character - even when receiving a "|" into display memory via SysEx - then I would have thought this is just how it looks; a quirk of this model of LCD.

One day I should take the LCD module out of my MT-32 and wire it up to my Pi, then I can get the rest of the characters and verify what they look like.

For what it's worth, I managed to dig up the photos I took when creating the font:

[sergm]

When I wrote that font I basically just SysEx'd a load of ASCII at my MT-32, took closeup photos and plotted the pixels by hand. '|" resulted in the broken pipe, so unless the MT-32 ROM is doing something crazy like remapping "|" to a custom character - even when receiving a "|" into display memory via SysEx - then I would have thought this is just how it looks; a quirk of this model of LCD.

That's very interesting, I'll double-check what's going on if a pipe comes in SysEx!

[vintagepc]

According to the DM2021 (2 line version) datasheet... 0x7C is solid. So it's either a custom char or Roland had Sanyo make a special-order run with a different ROM. I'd not be surprised if it was available as both a masked ROM and a PROM version where they would burn you a custom character set to-order.

[sergm]

When I wrote that font I basically just SysEx'd a load of ASCII at my MT-32, took closeup photos and plotted the pixels by hand. '|" resulted in the broken pipe, so unless the MT-32 ROM is doing something crazy like remapping "|" to a custom character - even when receiving a "|" into display memory via SysEx - then I would have thought this is just how it looks; a quirk of this model of LCD.

That's very interesting, I'll double-check what's going on if a pipe comes in SysEx!

Well, that was fun. Apparently, the character that separates the master volume is a special one, but the delimiter shown on program change is the "normal" pipe... So, these two seem to be aliased. I've changed the LCD font in emulator so that 0x7C is without breaks, and it looks amazing 😃