iOS functionality - key repeat issue

Working on expanding the iOS functionality from my initial PR, my next item is to address an issue w/ hardware keyboards (specifically I use the iPad magic keyboard). A few keys will send repeat actions correctly (comma and period keys) but most others won’t (alphanumeric). I’ve been doing debug traces and it looks like it comes down to the event SDL_TEXTINPUT not re-firing for the latter. I’m not sure if this is related to the way events are picked up, or is related to how keysym mode detects events. I’ve wondered if the mapping plays in to it somehow, given that a partial set works and doesn’t (another is if I do opt+keys that hit the unknown range it’s 50/50 if they replay correctly).

Once I get this issue worked out I plan to turn my attention to working on on-screen controls, and aim to expand the existing functionality built for android to work on iOS.

Well, having put a significant time in to researching this issue, I have a pretty good amount of clarity about the both specific issue as well as an understanding of the input / action system as a whole. As this particular side quest wraps up I wanted to share my findings:

1/ For the immediate issue it looks like the keys not repeating problem stems from the fact that keys that continue to be held down have their events recycled as SDL_TEXTINPUT events that is the result of some fascinating but mildly disturbing interplay between SDL and getch (recognizing there are quirks on both sides of that line). I haven’t gotten quite to the full root cause, but I suspect that the codes being used by the ipad magic keyboard are of a non standard variety (most likely a result of being presented piggybacked as screen keyboard events) that is causing them to fall out a range that is considered in bounds by getch. In addition to mapping these out in specifics, I’m going to do some testing on windows and mac as well just to get a sense for the different ranges. Regardless, it points to a looming problem of scalability to support future localizations, input devices, and platforms.

2/ Zooming out a bit, the bigger issue is the sprawl of the input system itself. The sheer complexity resulting from the nested nature of what it has evolved into has been staggering to comprehend, especially without someone to explain it to me. I recognize that this sprawl is the result of both the passage of time as well as feature growth into places likely not originally anticipated (mobile, IMEs, etc). And it’s success that brings those kind of problems, and why someone like me shows up years later and puts this much time in to thinking about it unprompted.

To give you an idea of the problem, here is the process tree from the main game loop down to the level where hardware events are being processed:

main(int, char **)
do_turn()
game::handle_action()
game::get_player_input( … ) ← Note this is where the initial mention of input occurs
game::handle_mouseview( … ) ← This took me forever to believe I wasn’t ending up in the wrong place
input_context::handle_input()
input_context::handle_input(int)
input_manager::get_input_event(keyboard_mode) ← It’s never good when the manager is last to know
CheckMessages() ← This is where the input is actually processed (and vaguely named)

And while some of these layered calls are simple, others influence behavior and logic on the way down and back up. Adding to this the number of features, platforms, locales, and forms of accessibility this aims to support, even beginning to think of reimagining a better system feels weighty. But none the less, as someone who is actively trying to expand the capabilities of the system I can say that the level of tech debt this carries has made forward progress incredibly painful. And that by now, industry patterns for solving this kind of problem offer well thought out alternatives / options to consider.

So far I’ve been working in a vacuum as it’s been hard to connect with anyone who is familiar with this stuff (and wants to talk about it). While it’s made for some interesting learning experiences, I don’t fee comfortable investing much more time in the issue without some kind of feedback as to whether this is a known issue that no one is currently working on and the help would be welcomed or that efforts are in progress that I could plug in to (or people are working on it but don’t want help and to just wait).

I am still really excited about the idea of building out the iOS version of this game, as well and combining the features already built for android and ultimately being able to expand both simultaneously. But improving the mobile experience is going to lean heavily on the input system, which is why I’m taking the time to lay out this narrative as I don’t see a way forward without addressing some of these foundational issues. At the very least I will fix the issue that bothered me enough initially to cause me to spend hours debugging the game instead of playing it, and I will likely continue to switch back and forth between the two activities as I try to make the iOS version support the basic features I want. But that alone is likely to leave most people disappointed relative to what is available on Android or the existing 3rd party version (at which point iOS players have to choose features vs. playing on experimental).

FWIW, I am so grateful for all the work this community does and has done, and am just hoping to give back in a similar manner so that even more people can come and enjoy the game in the future.

Cheers!