kmkfw/kmk_firmware

Clackety Keyboards Powered by Python

kmk_firmware's Issues

Allow users to add their own functions callable by keypress

This is absolutely a missing feature, and one that would have me go back to QMK personally if not added. This is not an easy task with current structure, though is a critical feature IMHO or the point of not writing C is mostly moot.

Fix gesc yet again.

Seems like GESC is now partially working. If you don't let go of super, it will not allow you to type it again. It discards all keys pressed as intended, but does not re add it to the state.

Re-support rotary encoders

kmk/rotary_encoder.py has not been updated to support the new event loop/internals. Fix this, and possibly as part of this work also use rotaryio before falling back to our internal implementation.

Burn down the README. With fire.

General structure should probably change a fair bit (though some sections can stay):

• Title

• Badges, Matrix links, Discord mentions, whatever

• Synopsis/summary of project

• Supported Hardware

  • Feather M4 Express
  • ItsyBitsy M4 Express
  • Feather nRF52840

• Installation instructions
  This section includes all the "meat and potatoes" of how to flash CircuitPython, copy KMK's core, and copying a main.py (and why that's important, and how it relates to all the other moving parts we're describing)
  This section should include breakouts where necessary for:

  • Linux, generic BSD
  • MacOS
  • Windows

  This section basically replaces flashing.md in the docs/ tree (or at least heavily augments it)

• How to report issues

• Thanks, community shout-outs, author information, etc.

• Legal, copyright, CLA, etc.

Support "standalone keys"

This could be the push button on a rotary encoder (such as Adafruit's p/n 377), a kick pedal, whatever (though kick pedal support is a totally different type of beast than the button on the Adafruit 377). Support both of these cases as "secondary" matrices, probably scanned before (but in addition to, rather than instead of, as @kdb424 does for split keyboards) the primary matrices.

This came up in the CircuitPython Weekly for 22 Oct 2018 as something someone is interested in (though the comment was not in a KMK context, it's still a good idea)

Add backspace/Delete key combo

Through MOD4, similar to GESC.

Add MEH and HYPER keys

KC.HYPR - Hold Left Control, Shift, Alt and GUI
KC.MEH - Hold Left Control, Shift and Alt

Support various core QMK functionalities (Round One)

• Core 104-key ANSI keycodes
• Extra keycodes
  • KC_GESC
  • RESET
  • Media Keys: VolUp, VolDown, VolMute, MediaPrev, MediaNext, MediaPlayPause
• Held-modifier keys
  • 1 Mod + Letter
  • N Mods + Letter
• Shifted Keys
  • This is a convenience keysym which allows for sending, for example, @ directly. It basically emulates Shift+2 in a single logical action
• Some Layer Switching
  • MO(x)
  • TO(x)
• Macros
  • Raw keycodes by const (ex. macro([KC_A, KC_B, KC_C]))
  • SEND_STRING equivalent
  • This task does NOT include anything aside from sending series of keystrokes
• Leader Mode
  • Timer-based
  • Interrupt-based (sequence of keys ended by an explicit key, ex. <Leader>ddg<Enter>)
• Send Unicode sequences by sequence of hex (req. ibus, MacOS, or Windows)

Consider adding support for "weak" controllers as initiators

I have gotten the Itys Bitsy M0 to work as a proper uart slave (with a single pin remapped to converter). The changes are... somewhat messy. This can be seen in the topic-slave-uart for an example of a very quick and dirty example change. It involved slimming down the firmware massively, along with the internal state for ram savings or something this weak wouldn't even boot. Keyboard is just stripped of everything that isn't needed, and target set to the new circuitpython_samd21 mcu, which then knows it's a slave, and calls the slave versions instead. Thoughts?

Implement at least basic tests of the InternalState

It should be reasonable to implement a shim/patch firmware object which manually injects various matrix scans into the "reducer" (that will be going away with #70 but the ideas will generally stand), and validate the state afterwards. This will let us ensure there are no regressions going forward (and actually give us something useful to run on Circle now that we're not stuck validating patched CircuitPython builds)

Remove micropython-lib dependency

This will allow us to remove all git submodules (and the various hackarounds in our repository to make them suck less) from the repo and simplifies deployments even further (to now just needing a main.py and the KMK folder, rather than also needing string.py to be copied)

This depends on CircuitPython providing string as a standard library module OR us bundling string.py from micropython-lib in the KMK source directly. micropython-lib is licensed under MIT, so this is is feasible with some updated documentation, and is arguably easier than upstreaming string

Allow users to add code to the main loop in a sane way.

We had a request to add "parallel" code such as a joystick also running. This may be useful for any number of things such as rotary encoders that aren't keyboard related and the like. There are several other "toy" projects that I could also see people using this for as well. As people who forked QMK to add functionality, I can definitely see the use for this, even if that function did get upstreamed, but I digress.

(Investigate?) Move CI to Github Actions

This perhaps invalidates or changes #144

Github Actions CI is available for our repo finally. It's probably sane to keep everything in one place, so investigate moving to Github Actions. We should generally be able to do all the same things sanely I hope.

I literally can't type

I wired up my Planck Rev 6 with a Feather M4 Express today and drop keystrokes like crazy. It turns out this is because of the garbage collector having to clean up after a million and one inefficiencies in the codebase.

Things that are now getting refactored (after an evening of prototyping and discussing with @kdb424):

• Flattening Store and InternalState
• Moving from the Redux-style event/action dispatcher system to raw class method calls
• Rewriting the matrix scanner to speak a more minimal 3-byte protocol (saves list/tuple allocations)
• Probably others I'm not remembering.

We're also officially killing off MicroPython/PyBoard support as part of this - not worth regression testing an MCU we don't care about.

I'll update this ticket with more sane details in the morning when I'm more awake.

Convert keymap from list of lists, to flat list.

This is needed for "non square" pcb's or ones that don't align the colums due to pin limitation. For example, the luddite is a "square" in software, each row is 8 columns, but that makes the keymap a lot harder to read. We should consider making this change during boot and forming the rows and columns for internal used based on pins, and not a user defined list of lists.

Example of why this is not good.

kmk_firmware/user_keymaps/kdb424/luddite.py

Lines 49 to 59 in d5272ce

	keyboard.keymap = [
	[
	[KC.GESC, KC.N1, KC.N2, KC.N3, KC.N4, KC.N5, KC.N6, KC.N7],
	[KC.N8, KC.N9, KC.N0, KC.LBRC, KC.RBRC, KC.BSPC, KC.LEAD, KC.QUOT],
	[KC.COMM, KC.DOT, KC.P, KC.Y, KC.F, KC.G, KC.C, KC.R],
	[KC.L, KC.SLSH, KC.EQL, KC.BSLS, KC.TAB, KC.A, KC.O, KC.E],
	[KC.U, KC.I, KC.D, KC.H, KC.T, KC.N, KC.S, KC.MINS],
	[KC.ENT, KC.LSFT, KC.SCLN, KC.Q, KC.J, KC.K, KC.X, KC.B],
	[KC.M, KC.W, KC.V, KC.Z, KC.RSFT, KC.LCTL, KC.LGUI, KC.MO(FN1)],
	[KC.SPC, KC.LEFT, KC.DOWN, KC.UP, KC.RIGHT],
	],

Macros missing documentation.

Unicode macros seem to be fine, but simple macros could use a documentation page.

Fix up kmktime

Put all time related functions related to time and put them in kmktime. Also ensure that it will work on circuitpython.

GUI keyboard layout editor

As simple as it sounds, a gui tool to make things easier. Still trying to decide if it would be a native app or web app.

Epic: Split Keyboards

• Over Bluetooth LE - the only inspiration I have here is from Wez
• Over I2C
• Over Serial/UART

Feather NFR52840 broken?

Getting hang on booting any keymap on a ghirkin running on this. Default board no longer boots with basic keymap. Debugging reveals this, and nothing happens further.

main.py output:

KMKInit(release=copied-from-git)
KMKKeyboard(debug_enabled=True keymap=truncated coord_mapping=truncated row_pins=truncated col_pins=truncated diode_orientation=0 matrix_scanner=<class 'MatrixScanner'> unicode_mode=1 tap_time=350 leader_mode=0 leader_dictionary=truncated leader_timeout=2000 hid_helper=USBHID extra_data_pin=None split_offsets=() split_flip=False split_side=None split_type=None split_master_left=True is_master=True uart=None uart_flip=True uart_pin=None)
InternalState(keys_pressed=set() coord_keys_pressed={} leader_pending=None leader_last_len=0 hid_pending=False leader_mode_history=[] active_layers=[0] start_time={'tg': None, 'leader': None, 'lt': None, 'tt': None, 'lm': None} timeouts={} tapping=False tap_dance_counts={} tap_side_effects={})
GCStats(pre_alloc=144912 pre_free=77744 alloc=146672 free=75984)

as Running latest stable Circuitpython. Worked before updating KMK. Some change since July 27th 2019 has broken this.

Add Simple QMK->KMK keyboard layout converter

Simply take in a QMK file, spit out a KMK file, and throw away things that we don't directly convert such as user defined keys, ect. The user will also be asked during the conversion process on board, ect to make a simple conversion as long as the user does not use advanced functionality or things that we do not support. Currently in planning stage.

RFC: Change jurisdiction of contributor CLA from BC to WA

This seems like an entirely logical change now that I look that document over again and realize I used my current resident jurisdiction rather than my "legally a citizen of" jurisdiction in the docs. Whoops.

This change would solely impact section 9.1, which can be found here

9.1 This Agreement and all disputes, claims, actions, suits or other proceedings arising out of this agreement or relating in any way to it shall be governed by the laws of British Columbia, Canada excluding its private international law provisions.

This would change to

9.1 This Agreement and all disputes, claims, actions, suits or other proceedings arising out of this agreement or relating in any way to it shall be governed by the laws of the State of Washington, United States of America, excluding its private international law provisions.

General rationale:

• I have no idea how BC software laws work anyway
• I have no idea how long I'll ultimately reside in BC given my immigration status up here literally has "temporary" in the name
• 2/3rds of the members of this Github organization live in Washington, and the last one (myself) is still somewhat of a legal subject of Washington (I'm still a WA resident for voting purposes)
• Changing the CLA after more contributors come on board is going to be way more painful.

Need signoff/comments from all current signers of said CLA, so @kdb424 and @tannewt. If this seems sane and we make the change, we'll also all have to re-click the sign button.

Thanks all!

Remove InternalState.update, it's inconsistently used and of marginal if any value at this point

Add Debug Key

Needs to be able to cycle between different debug levels. Most likely
Off, Debug, Warn, and a higher level if possible to only trigger on user defined code.

Remove any mention of Proton C or Planck Rev 6

There is only 40k of ram on these devices, which is not enough to even load the basic firmware file. I have tested on itsy bitsy M0 express and with it's 32k of RAM just absolutely died with no chance that GC could even help.

Profile and speed up code

Areas that appear to need turning is the delay in pyb_hid.py, though there may be other problem areas.

Target upstream CircuitPython

Per conversation in adafruit/circuitpython#1249, it'd be great to completely eliminate the KMKfw/circuitpython repository and try to track upstream commits/tags. This involves adapting and/or mainlining a few parts of KMK, particularly as it relates to the building/flashing process:

• Devise a system to replace the single frozen entrypoint _main.py on CircuitPython targets (this is what the upstream PR was submitting in the first place). So far the best alternative found is reading keymap off of MSC flash with a fallback on exception to a "default" or "safe mode" mapping, which feels like a solid "meet in the middle" between the projects' intentions.

• Restore MSC HID report and CircuitPython background task, which were removed in https://github.com/KMKfw/circuitpython/commit/5c606deecc93c573516c46787486cee846a6d599. This is a dependency of the above.

• Mainline MakerDiary NRF52840 MDK support? While our support of that board is very infantile and a WIP right now, mainlining this support rather than maintaining it in a fork is probably overall better for the community/ecosystem as a whole.

Thanks to @tannewt for the feedback and ideas, and feel free to use this as a scratchpad for other ways we can help to be better FOSS citizens (or at least in CircuitPython-land specifically).

CircleCI envvars are not passed through to forked PRs for security reasons, so builds fail

Solution is open-ended and I can think of several options, but it does need fixed - our builds only pass for branches on this repo, not on forks thereof, due to the DigitalOcean API tokens used as env vars.

Can't macro layer switch?

simple_key_sequence([KC.DF(1)]) I would expect to work. Bug maybe?

Rapid key presses may in some scenarios lead to dropped key presses

In some gaming situations I have experienced dropped key presses when having to press multiple keys rapidly.

RFC: Removal of PyBoard/MicroPython support

Right now our supported boards list consists of the PyBoard v1.1 (STM32/MPY), and two Adafruit boards (the Feather and ItsyBitsy M4 Expresses, both SAMD51/CPY). Our pipeline for supported devices right now includes the NRF52840 (NRF/CPY), Planck Rev6 (no existing port, most feasible is likely MicroPython to STM32F303), and possibly the Proton, which if I had to guess is likely to be similar to the Planck r6.

The common pattern here is that our best-supported, most economical, and most feature-complete boards are CircuitPython-based. We effectively support MicroPython for one expensive, RAM-constrained STM32 chip (and hold out hope for a future extremely-flash-constrained chip). It's worth deciding if the technical weight of this port is actually worthwhile: a single Python target would be much easier to reason about.

It's also entirely possible to have our cake and eat it too, preserving device support while moving to target CircuitPython exclusively, by reviving the stm32 port in CircuitPython's source tree, though it's deviated very heavily from upstream MicroPython at this point and that refactor would not be small.

Some factors to consider in this: with extremely economical options such as the Feather and ItsyBitsy available, and more feature-filled devices such as the NRF Feathers available for similar or lower pricepoints to the PyBoard, will that port actually be used enough to justify the support? Is porting to STM32F3 a worthwhile endeavor to add support for the Planck? Will the Proton perhaps be a bit less contsrained than the Planck, and thus a more worthwhile porting target (and thus another reason to keep the MPY tree around)?

Epic: TapDance and its many flavors

Add support for tap dance features allowing users to define custom actions based on different counts of taps or tap and hold.

This will be achieved by setting a timer in the state (user configurable, possibly per key), and when the timer expires, it will act as a macro, that fires other macros.

• TD(KC.SPACE, KC.LEAD)
• TD(KC.U, send_string('test'))
• TD(MO(2), MO(3))

Those should all be valid. Depending on what is received, the tap dance will handle differently. For example, things like MO(2) would be fired on keydown as there is no output to suppress and offers the best experience having instant results where possible. If something like KC.SPACE or any other keycode that would trigger HID, or a HID changing action, such as KC.LEAD, would be fired ONLY after the timer expires.

Internal keycodes should have a tag on them that would offer unblocking of tap dance, or it will wait until final resolution of the timer to facilitate easy additions of keycodes, and ensure that tap dance will function properly with all new additions by default.

With this addition, keycodes such as TT could simply be written by using
TD(MO(2), TG(2))

ModTap shift fires inconsistently

@kdb424 can fill this with more details since he ran into it, but he wasn't able to use Shift-Click in Minecraft, yet was able to type fine in normal conversations

Testing matrix bot integration

Investigate why keys on layers still sometimes get stuck

My example case that happens at least a few times a week on the iris_r2:

• VolUp is triggered with Fn2+{Key1}
• VolDn is triggered with Fn2+{Key2}

On the default layer, Key1 is itself a Fn key (KC.MO), and Key2 is my spacebar. VolUp will get stuck pressed if it is not released before the Fn2 key. VolDn will never exhibit such behavior.

Repro case: Press MO(2), press VolDn, release MO(2), release VolDn. The end result is the expected behavior, the volume down key gets released. Do this same sequence with VolUp and the volume up key never gets released, almost always resulting in dangerous volume levels on the target PC.

Not sure if this has anything to do with the keys underneath the volume functions but it's the only discerning factor I can think of so far in months of dealing with this bug.

Implement BLE HID

adafruit/circuitpython#1050 is moving courtesy of @dhalbert. We should do whatever is necessary (both within KMK and, if necessary, upstream) to support connection between a single-piece keyboard or master component of a split keyboard and a BLE host (PC, phone, whatever) as soon as is reasonable. This scope does not include communication between two halves of a keyboard over Bluetooth.

Our HID system is fairly abstracted from the underlying implementation at this point, so a simple implemention of kmk.hid.BLE.send likely gets us most/all of the way there as far as targeting upstream's BLE libs. It is likely we'll want to split this HID implementation out of kmk.hid into an Extension assuming https://github.com/KMKfw/kmk_firmware/tree/topic-merge-keyboard-and-state (misnomer at this point; that branch implements the abstract kmk.extensions.Extension API and begins moving most non-core KMK components to use it) merges before this is worked on. We'll also need to provide workflows (how? keys? main.py interfaces?) to pair and bond the keyboard with the host; ideally we could support multiple host pairings (think similar to a Logitech K810 for easy swapping.

Tech Debt hitlist

This is a probably-ever-expanding catch-all ticket for tech debt that should be cleaned out for v1

• Remove kmk/pins.py entirely - this was a leftover from days of supporting MicroPython targets under a unified KMK-side API. It's gnarly, does too much, and will probably not do what the user expects if they try to use it as a generic GPIO pin in a macro.

• Clean up ModifierKey implementation? I dunno. It feels smelly.

• Is kmktime even worth keeping? Investigate what, if any, of this file is necessary in its current form vs. just importing time directly. Again, this file is mostly vestigial from MicroPython days

Layers get stuck

This can be reproduced by hitting 2 layer keys in rapid succession (1 processing cycle) allowing for the layer to be switched twice. Example activating MO(2) and MO(3) in the same cycle will activate the correct layer 3, then get stuck on layer 2 on release of both.

Change entrypoints to keyboards

To add keyboard definition support and clear up confusion, make the current "handwire" folder into mcu as that should point to a microcontroller. This would be a lot clearer terminology, and allow us to have pre-built keyboard definitions, and even common hand wire setups be defined, allowing users to build keyboards much easier, not having to understand "what goes on", just "that it is this keyboard", along with making defining a PCB much easier.

Leader key timeout still affects Enter Mode?

I am timing out in enter mode seemingly, which seems to defeat the purpose. Maybe this is a feature? If so, there should be a higher default timeout when setting it.

Set up KMKfw Matrix server with Discord bridge

Most likely hosted on AWS or DigitalOcean, we need a community chat area that isn't Github issues or Reddit. A Matrix server is ideal for this: federated, fully open-source, and has great clients available for all platforms.

However, since much of the keyboard enthusiast community is on Discord (perhaps for QMK, or because they're gamers anyway), in addition to much of the MicroPython/CircuitPython community being on there as well (since Adafruit has a Discord community for support issues and a million other things), we should be accessible to these folks as well. Thankfully, we can do a semi-seamless bridge to have the best of both worlds, with https://github.com/Half-Shot/matrix-appservice-discord

Flatten kmk.common.* into just kmk.*

This is a needless bit of nested stacking dolls that absolutely serves no purpose anymore.

Create a kmkfw Docker image

This will not only allow for significantly easier user flashing (hopefully...), but will also cut down CircleCI runtimes, which are currently bonkers due to all the bootstrapping and system dependencies we have.

Clean up some docs.

I'd like to see the main readme changed to be a little more.... Clean I guess. Along with that, the new general docs structure should likely also be applied to pages like Tap Dance. That being

• General information about what it is, and a short high level description of how
• Config settings (if applicable)
• Small examples
• Troubleshooting tips
• Dev docs or links to helping understand the inner workings (if applicable)

This allows the users to get what they need right at the top to get them up and running, and then later explains more if they want to take their ideas further, while not complicating it if they don't need that information.

Actually add "Mod Tap" keys

These are missing, and still in our docs as a reminder to make them.

No input sent to OS (presumably main.py frozen) until REPL is opened in CircuitPython 4.0-alpha.3+

@kdb424 ran into something similar and rolled WAY back (to like 3.x or something) when building out split keyboard support, but it's now come to a head when trying to put my Iris together tonight.

On CircuitPython 4.0.0-alpha.2, the keyboard works as expected, fully unattended. On all CircuitPython releases from 4.0.0-alpha.3 onwards (including the latest, alpha.5), the keyboard will send no data at all to the OS (presumably main.py is hung somewhere?) until the CircuitPython REPL is connected to in userspace (picocom or whatever).

The knock-on effect of this is that split keyboards are completely nonfunctional on these CircuitPython releases, as data is ALSO not sent across to the primary half of the keyboard.

I have only personally tested this on a split configuration with ItsyBitsy M4 Expresses converted to Pro Micro pinouts using the kitsym4 board. I have not upgraded my Planck/Klaranck past alpha2 as of yet, so I can't speak to whether single-piece keyboards and/or the Feather M4 Express are affected, but it's logical to assume that if single-piece keyboards are, the bug will affect that keyboard as well (I doubt the tiny differences between the IBM4 and FM4E are the smoking gun here).

This is effectively "critical priority", or whatever our equivalent of such is.

It is important, also, to note that breaking UART changes came in alpha5, so some reworking may be required while targeting these new releases.

Add leader key

• Leader key
• Leader Mode - Timeout
• Leader Mode - Enter

Sort why not working in UEFI/Booting

I am unable to use my keyboard until I get to an operating system. This is very likely an upstream issue, though is definitely our problem as you would need another keyboard on hand. It's not an issue of the firmware not having time to boot as it affects booted boards upon system reboot which AFIK does not drop power from any of my keyboards therefor does not reboot the keyboard.

Allow users to add their own custom LED animations

#111 May be related as I would have to run user functions in order to allow this. With the current implementation, that is not possible unless they do static animations.

KC_GESC never sends KC_GRV/KC_TILDE

It correctly works as KC_ESCAPE, however.

This is some sort of regression in the past couple of months - this key used to work. Likely during the InternalState cleanup in ~November this got broken?