STEM 1

• Home
• About Me
• STEM 1
• STEM 2
• Humanities
• Intermediate Spanish
• Math Modeling
• Physics
• Computer Science

Using a Mathematical Model to Personalize Keyboards for Users Missing Digits

Overview

This project was a six-month long research project that was conducted between August 2022 to February 2023. The majority of this project was conducted independently, with the input and advice from Mr. Nicholas Medeiros and Dr. Kevin Crowthers.

Though computers are ubiquitous, and the keyboard is crucial for computer use, there haven’t been many attempts to create more accessible keyboard layouts with specific users in mind. While there are shortcuts such as sticky keys, these focus on modifier keys, rather than the letters that certain users, such as typists who are missing digits, might have a hard time reaching. Currently, there is no literature containing information about keyboard layouts for users who are missing digits. However, countless forum posts online suggest that there are people looking for these types of keyboards, and that this is an issue that requires a solution. Thus, the main goal of this project was to help create custom keyboard layouts for users missing digits. The end result was four different keyboard layouts for users missing index and middle fingers that showed to increase maximum typing speed by around 2 to 3 words per minute compared to typing on the Qwerty layout. In addition, by rearranging, the more commonly typed keys and key pairs became more easily accessible and were placed better ergonomically. Finally, the keyboard layouts were visually similar to Qwerty, keeping 77% or more of the letters in their original locations.

Abstract

⠀⠀⠀⠀The Qwerty keyboard layout is ubiquitous and used by virtually everyone in America. However, research has shown that this layout is ergonomically insufficient compared to existing alternative layouts. While switching keyboard layouts usually has many downsides, for some typists, such as finger amputees, there could be some benefits. This is important since each year, thousands of people face finger amputations, which can hinder their typing performance. In addition, the more dexterous index and middle fingers are the most commonly injured. Although these users can relearn how to type, some commonly typed keys might become less accessible, causing typing speed to decrease. However, by rearranging only the important letters, typing speed and efficiency can be improved, while keeping the rest of the keyboard the same. Thus, this project used a mathematical model to create personalized keyboard layouts for users missing index and middle fingers. By referencing ergonomic principles and typing data, the model analyzed the efficiency of a given keyboard layout by calculating its cost, based on estimated digraph times (time to type two keys) and digraph frequencies in English. This cost was essentially a weighted average digraph time for the layout. Then, the keyboard was rearranged, and the cost was calculated again. The rearranged keyboards showed to be, theoretically, faster to type on, while still maintaining visual similarity to Qwerty. Although these layouts still need to be tested with typists, and more missing digits need to be assessed, these layouts can help finger amputees who are dissatisfied with their typing.

Research Proposal

Click here to view the research proposal and project notes.

Background

History of the Keyboard
⠀⠀⠀⠀Before typing on smartphones and computers, typing was done on typewriters. The Qwerty layout, denoted by the “q”, “w”, “e”, “r”, “t”, and “y” keys in the top row was created for the typewriter in the 1870’s and has since become the de facto standard of keyboard layout in America and many other regions of the world (Harford et al., 2019). Despite being the most common layout today, it was invented to make typing slower. Christopher Sholes patented the Qwerty keyboard in the 19th century, and it quickly became the standard for all typewriters. This layout made typing slower, but also solved a major issue. Typewriters would frequently jam or print letters on top of each other if the keys were hit too fast, so to combat this, Sholes had placed frequently typed pairs of keys on the opposite sides of the keyboard to slow typists down (Harford et al., 2019). Though the layout worked well for typewriters, modern keyboards do not face the issue of jamming, nullifying the original purpose of the Qwerty layout. Despite this, Qwerty has become ingrained in the daily lives of millions of people regardless of its inefficiency and outdatedness.

Keyboard Layout Optimization
⠀⠀⠀⠀Researchers have realized the issues with Qwerty, and many attempts have been made to optimize keyboard layouts using a variety of different methods. Onsorodi & Korhan (2020) used a genetic algorithm to rearrange keys, aiming to reduce travel time when typing. Similarly, Eggers et al. (2003) looked at multi objective optimization and used an ant colony algorithm to tackle the keyboard optimization issue. Others, such as Li et al. (2006) analyzed key distance for typing with one finger on a virtual keyboard. Even during Sholes’ time, Dvorak et al. (1936) created his Dvorak layout using ergonomic principles that reduced finger movement to improve typing speed.
⠀⠀⠀⠀These layouts all claim to be superior to Qwerty, citing either measured or calculated typing speed increases. Despite these benefits, only a small number of enthusiasts commit to learning a new layout. The time and difficulty of learning a new layout does not offset the slight benefits of typing faster, an issue that is exacerbated because most optimized keyboard layouts look completely different from the Qwerty layout. Even the most popular alternate layout, the Dvorak layout, looks nothing like Qwerty (Figure 1), which is an issue because if one were to learn an alternate layout, not only would they have to learn the new locations of the keys, but they would also have to undo years of muscle memory from typing on the Qwerty layout. However, if the majority of the keyboard were to be kept the same — by only moving a few keys — typing could theoretically still be improved. Specifically, focusing on those who are missing digits, moving keys that are difficult to reach while keeping the rest of the keyboard the same could increase the quality of typing for some users, without the additional retraining time.

Background - Continued

The Target Audience
⠀⠀⠀⠀An estimated 45,000 finger amputations occur per year in the United States alone (Reid et al., 2019). This means that the number of Americans who are missing one or multiple digits is even higher. Currently, these people are forced to relearn how to type: a time-consuming task. Not only would some keys be located inconveniently, but also overriding years of muscle memory would be difficult as well. Time that is already spent learning to type with fewer digits would be better used learning a custom keyboard that accounts for those missing digits. While there have been advancements in assistive technology for other parts of the computer, the keyboard has been largely neglected.
There currently are no studies showing impacts of finger loss on typing speed and comfort. However, numerous personal experiences have been shared on the internet, attempting to bring light to this issue. Looking through various forum websites such as Reddit, as well as personal blog pages, it is clear that many people struggle to reach their maximum typing speed after the loss of a finger and are looking for solutions. Furthermore, while there is technology that allows for accessible text input, no comprehensive solutions for the physical keyboard currently exist. There has, however, been past research that provides the fundamentals to reassign keyboard keys according to specific criteria.

Digraphs and Touch Typing for Keyboard Optimization
⠀⠀⠀⠀Touch typing is a style of typing that utilizes all ten fingers in the typing process in order to increase typing speed. For this style of typing, each finger is assigned its own column on the keyboard (or two columns in the case of index fingers) and the thumbs remain on the space bar. By designating each finger to only a few keys, the keys on the keyboard can be memorized easier and thus most touch typists can type quickly without having to look at the keyboard. This style of typing is also the easiest to analyze because the typing patterns are more predictable than if the typist were to use only two fingers to hunt for the keys. Thus, this project this project will use data obtained from touch typing trials, and so the rearranged keyboard will also be optimized for touch typists.
⠀⠀⠀⠀One way to quantify the efficacy of a keyboard is to look at words per minute (WPM) typed on that keyboard. This is usually calculated by taking the number of keys typed in a minute and dividing them by 5 (the average length of a word). Another way to look at typing speed is the time it takes to type a pair of keys (digraphs). İşeri & Ekşioğlu (2015) measured and estimated the time it takes to type pairs of letters on the keyboard, also known as digraph times or interkey tapping times. The rationale behind this was that theoretically, the interkey tapping times between two keys is independent of what letters lie on those keys. This means that typing key 1 and then key 2 would always yield the same digraph time, regardless of what letters are bound to those keys (Figure 2). This provides a way to analyze different keyboard layouts since typing can essentially be modelled by multiplying these interkey stroke times with their relative frequencies in English corpus and summing these values up, you get a quantified “cost” of the keyboard that is similar to typing speed. Put simply, by multiplying the time it take to type a pair of letters by the frequency that the letter pair is used in English and summing up the results for each letter pair, you can quantify how optimal the keyboard is. If it took less time to type a more commonly occurring pair of letters, the cost would be less, showing the layout is better. This is what Light & Anderson (1993) did when they optimized the keyboard via a simulated annealing algorithm.

Potential Pitfalls and Solutions
⠀⠀⠀⠀Though many strategies have been employed to optimize the keyboard layout, it is almost impossible to tell how optimized the layout is due to the sheer size of the solution space: 26! = 403,291,461,126,605,635,584,000,000 possible different keyboard layouts (Light & Anderson, 1993). Though a pitfall, this is only the case if all twenty-six letters are rearranged. If we instead limit the rearranging to 3-6 letter pairs, the solution space decreases, and it is easier to find the optimal or near-optimal solution. Though this project does not use machine learning to search a solution space, the mathematical model will be able to iterate many times to find an improved layout. In addition, only rearranging a few letters also ensures that the ending keyboard is incredibly visually similar to the Qwerty keyboard. This solves a major issue with most new keyboard layouts. By placing each key in a completely different location, the user must scan the entire keyboard to find the key each time. However, by only moving a few keys, and furthermore, keeping those keys on the same side of the keyboard they were originally on, the user will be able to find them quickly. In addition, muscle memory will be maintained because the key is still being typed with the same hand as on the Qwerty keyboard.
⠀⠀⠀⠀The lack of studies on missing digits and typing is another potential pitfall for this project. Though there is evidence online to show the need for this type of study, the anatomy of hands and fingers makes it difficult. For example, a user missing an index finger would require a different keyboard layout than a user missing a middle finger, etc. In addition, missing half of one finger is different than missing the entire finger. To overcome this pitfall, this project only focuses on index and middle fingers, which are the most commonly lost digits (Saraf & Tiwari, 2007). However, extensions may be considered in the future, as it would be relatively straight-forward to analyze other missing digits.

Rationale and Objectives
⠀⠀⠀⠀In summary, there are very few solutions for users missing digits who wish to improve their typing. Though alternate layouts exist, there are none developed specifically with these users in mind. In addition, most alternate keyboard layouts often change the keyboard drastically, making them hard to learn.
⠀⠀⠀⠀This project aimed to create custom keyboard layouts for users missing index or middle fingers by rearranging only a few keys. This not only keeps can improve typing speed by moving more frequently typed keys into easily accessible locations, but it also is not difficult to learn, since the majority of the keyboard is kept the same.
⠀⠀⠀⠀The objectives for this project were as follows:
Obj. 1: The first objective was to rearrange keyboards for users missing one digit: either the index or middle finger. This was because index and middle fingers are the most commonly lost digits.
Obj. 2a: The second objective was to re-estimate digraph times for key pairs that would be typed differently due to a missing digit.
Obj. 2b: The second object was also to use these estimated digraph times to show that the newly arranged keyboard layouts would improve typing speed for users missing that specific digit.

Procedure

Equipment and Materials
⠀⠀⠀⠀The mathematical model to rearrange the keyboard was programmed using Java 8 in Eclipse IDE. The standard Java 8 Library was used, with the exception of Apache POI (Poor Obfuscation Implementation), which was used to read an excel spreadsheet. Within the mathematical model, the digraphs used in this project were from İşeri & Ekşioğlu's 2015 study, and the frequencies for letter pairs were taken from Norvig’s study which analyzed English Letter Frequency Counts in Google corpus data. The mathematical concepts behind the model itself were borrowed from Light & Anderson's (1993) study that used simulated annealing to optimize the keyboard layout.

Mathematical Model
⠀⠀⠀⠀The goal was to create a program in Java which could use digraph information to rearrange a keyboard for a specific user. By giving the program a keyboard layout and a file with the digraph times and frequencies, it would be able to organize all the frequencies and times into two parallel-constructed arrays that corresponded with each other. That means that the time it took to type “QU” would be located at the same index as the frequency of “QU”. Then, the corresponding indexes of the two arrays were multiplied together and summed to find the cost of the keyboard. Finally, each pair of keys on one side of the keyboard were swapped, and the keyboard with the lowest cost was kept. The reason only keys on one side of the keyboard were swapped, was because the criteria for optimizing the keyboard layouts were:
A) Only three pairs of keys will be rearranged.
B) Keys that are swapped will remain on the same side of the keyboard as they were originally. This is to allow users to find the new key positions easier, thus decreasing learning time.
⠀⠀⠀⠀After receiving the lowest cost, this keyboard was accepted and then put back into the model and optimized again. In total, each keyboard was optimized times, with the lowest cost keyboard being accepted as the default keyboard after each optimization. This three-time optimization allowed for the maximum of three pairs of keys to be swapped (6 keys in total).

Extrapolating Digraph Times
⠀⠀⠀⠀Since there was no data for digraphs typed by users missing digits, these had to be extrapolated. After analyzing the digraph data from İşeri & Ekşioğlu (2015), it seemed that times between columns appeared to be proportional to each other. By comparing digraph times for specific key movements to “baseline” times for specific fingers, digraph times could be extrapolated. The digraph times for the home row keys that involved hitting the same key twice were used as the baselines because there was no travelling involved, and so the digraph time most likely reflected the dexterity of the finger. From here, if one were to compare a diagonal reach using the index finger, they would be able to extrapolate a diagonal reach for any other finger on the same hand using ratios and baselines. Though this worked well with known data, getting within 20 msec of the actual averages, it is unclear if this was actually the case. Thus, this was a major assumption of the model. However, through this method, new digraph times were calculated and passed into the mathematical model.

Methodology - Continued

Extrapolating Digraph Times - Continued
⠀⠀⠀⠀In addition, in an attempt to improve upon the digraph times, hit direction was taken into account. Once again, the differences in baseline times were used, and then the same hand digraphs that stroked from the pinky to the index had half of the difference subtracted, while the digraphs that stroked from index to pinky had half the time added. This was so when averaged, the average would still align with İşeri & Ekşioğlu’s (2015) study, but there would be a slight difference in terms of hit direction.

The Missing Digits
⠀⠀⠀⠀Four cases were analyzed using this mathematical model. Using the aforementioned techniques, digraphs were extrapolated for users who are missing their right middle finger, left middle finger, right index finger, and left index finger. For the middle fingers, the assumption was made that the ring finger would take over and type the keys in the column of the missing finger. This was to keep typing distribution between fingers more even so the index finger would not be responsible for typing in three different columns. For the missing index fingers, it was very difficult to come up with a way to type using the touch-typing method while still being able to access all the columns. Because of this, it was assumed that the hand missing the index finger would shift their home row position one key towards the middle, except for the pinky. This meant that for the right hand, the middle finger would rest on “j” and the ring finger on “k”. However, the pinky was left in its original position. With this configuration, the ring and middle fingers would be typing in two columns each, and the less dexterous pinky would remain in its own area. Though there is no information on if users who are missing index fingers type like this, the assumption was made for the sake of calculations. In the future when more data is collected, this will be subject to change.

Layout for Missing Left Middle Finger

Layout for Missing Right Middle Finger

Figure 3: The resulting optimized keyboard for a user missing their left middle finger. This layout rearranged the "R", "F", "V", "T", and "E" keys. The cost of the Qwerty keyboard was 168.7177 msec, while this keyboard has a cost of 162.6482 msec, meaning the cost decreased by 6.0695 msec.

Figure 4: The resulting optimized keyboard for a user missing their right middle finger. This layout rearranged the "K", "P", "H", "I", and "J" keys. The cost of the Qwerty keyboard was 166.3186 msec, while this keyboard has a cost of 161.3272 msec, meaning the cost decreased by 4.9914 msec.

Layout for Missing Left Index Finger

Layout for Missing Right Index Finger

Figure 5: The resulting optimized keyboard for a user missing their left middle finger. This layout rearranged the "R", "F", "W", "A", "T", and "E" keys. The cost of the Qwerty keyboard was 171.8442 msec, while this keyboard has a cost of 164.8221 msec, meaning the cost decreased by 7.0221 msec.

Figure 6: The resulting optimized keyboard for a user missing their right middle finger. This layout rearranged the "K", "O", "N", "H", "I", and "J" keys. The cost of the Qwerty keyboard was 169.3917 msec, while this keyboard has a cost of 161.7522 msec, meaning the cost decreased by 7.6395 msec.

Analysis

Another way to look at cost, is to turn it into words per minute (WPM). WPM is a way typists measure their typing speeds, and so turning the cost into a WPM score can more easily show the improvements the keyboard layouts can create. When typing a digraph, two keys are hit, but the second key is used as the first key for the next digraph. Because of this, digraphs were treated as typing one key. Then, by the way WPM is calculated, the inverse of cost (letters per millisecond) was divided by 5 (1 word per 5 letters) and then multiplied that by 60,000 (milliseconds in a minute) to find words per minute.
By analyzing the costs for the four keyboards in this way, a user with the extrapolated digraph times for a missing left middle finger would have a WPM of 71.12 on the Qwerty keyboard and a WPM of 73.78 on the optimized keyboard (Figure 3). This is a 2.66 wpm improvement. While this isn’t a large increase, it should be noted that after an amputation, the user wouldn’t have a typing speed of 71.12 wpm on the Qwerty keyboard. While further research needs to be conducted, it is hypothesized that the rearranged keyboard would allow the user to reach a higher typing speed faster. This is because in addition to increasing typing speed, the new layouts also satisfies some ergonomic criteria, which improve typing comfort as well as speed.
Dvorak et al. (1936) created a list of ergonomic criteria for rearranging keyboard layouts. The ones that apply in this situation be summarized as follows:
1) Typing in the home row should be maximized, and movement to other rows should be minimized.
2) Typing should be distributed evenly between the hands and fingers.
3) Reaching to other keys over the distance of 1 or more key lengths should be minimized.
4) Typing with less dexterous fingers should be minimized.
Looking at the first keyboard layout for the left middle finger, firstly the “E” key was moved to the “F” key. This makes sense both ergonomically and mathematically as “E” is the most commonly typed letter in the English alphabet. Thus, it makes the most sense for it to be located in the home row position of the most dexterous finger on the left hand. This was a big optimization for the first keyboard, and other arrangements made small adjustments to improve cost.
Looking at the second keyboard (Figure 4) ergonomically, since “TH” is the most commonly typed digraph, it makes sense that “H” is relocated to the home row under the index. In addition, since “I” is more commonly typed, it was also moved onto the home row, despite not being shifted out of the column of the missing digit. In addition, since “P” is typed more frequently than “J” is, it also makes sense that the two were swapped.
The typing speed for the optimized keyboard was 74.38 wpm while the original Qwerty keyboard has a typing speed of 72.15 wpm. Again, the typing speed was improved by 2.23 wpm.
For keyboard 3 (Figure 5), once again “E” was moved to the home row under the index finger. However, in this case, it was actually being typed by the middle finger due to the change in home base position. Then, “A” was also moved to be typed by the middle finger instead of the pinky finger, which makes sense because more commonly typed keys shouldn’t be typed by remote fingers that are less dexterous. This keyboard was also able to improve upon typing speed by 3 wpm.
Then for keyboard 4 ( Figure 6 ), the more commonly typed keys were moved to be closer to the middle and ring fingers (the most dexterous fingers) in the home row. This layout was able to improve typing speed by a 3.3 wpm.

Discussion and Conclusions

Limitations
Though the numbers show to not be extremely promising, it is important to note the other aspects of the keyboard layouts that cannot be quantified without human testing. The first is that the keyboards theoretically increase typing comfort by putting more commonly typed keys in easier to reach areas. In addition, the optimized keyboard was kept extremely similar to the original Qwerty layout in an attempt to decrease learning time. Finally, most of the digraph times were estimated and extrapolated, so the data might not be accurate. In fact, the digraphs most likely underestimate the typing times, and thus if the model was to be run with actual measured times, the results most likely would be a lot better.
The main limitation of this project was the lack of data. This study was conducted with data that was taken from a study that did not have users who were missing fingers. In fact, there was no literature regarding affects of missing fingers on typing speed and comfort. In addition, these layouts were not able to be tested with real clients, and thus this project remains theoretical.
While this was a project looking at keyboard layout optimization, it was different than all past studies because this project specifically looked at keyboard layouts as a way of improving typing quality of life for finger amputees. In addition, while most other studies focused on making the most optimal keyboard, this project focused on only rearranging a few keys and seeing if there would be any noticeable improvement.

Future Work
If this study were to continue, the goal would be to find users who are missing digits and measure their digraph times. Then with this data, the model could be improved, and new layouts could be arranged. Then, the users would be able to test the layouts and learning speed as well as comfort and feedback could be taken. This is important because typing is a very important task with the ubiquity of computers and some people who are missing digits may find it more difficult to type on a normal Qwerty keyboard. By giving them a custom layout, their quality of life can be improved.

Conclusions
In summary, the goal was to create custom keyboard layouts for users missing index or middle fingers. This was done by coding a mathematical model to find the weighted average typing time for any pair of keys. This average was the cost and was the metric used to determine the efficiency of the keyboard. In the end, the keyboards were arranged and show to theoretically improve typing speed and comfort while maintaining visual similarity to Qwerty. However, the improvement was not that much. This study requires further analysis, as well as field trials with actual users who are missing digits to get more accurate data, as well as feedback.

References


Eggers, J., Feillet, D., Kehl, S., Oliver Wagner, M., & Yannou, B. (2003). Optimization of the keyboard arrangement problem using an Ant Colony
⠀⠀⠀⠀algorithm. European Journal of Operational Research, 148(3), 672–686. https://doi.org/10.1016/S0377-2217(02)00489-7
Dvorak, A., Merrick, N. L., Dealey, W. L., & Ford, G. C. (1936). Typewriting Behavior Psychology Applied To Teaching And Learning
⠀⠀⠀⠀Typewriting . http://archive.org/details/in.ernet.dli.2015.74878
Harford, T., Garcia, C., & Smith, S. V. (2019, April 16). Why Do We Still Use QWERTY Keyboards? NPR.
⠀⠀⠀⠀https://www.npr.org/sections/money/2019/04/16/714082890/why-do-we-still-use-qwerty-keyboards
İşeri, A., & Ekşioğlu, M. (2015). Estimation of digraph costs for keyboard layout optimization. International Journal of Industrial Ergonomics,
⠀⠀⠀⠀48, 127–138. https://doi.org/10.1016/j.ergon.2015.04.006
Li, Y., Chen, L., & Goonetilleke, R. S. (2006). A heuristic-based approach to optimize keyboard design for single-finger keying applications.
⠀⠀⠀⠀International Journal of Industrial Ergonomics, 36(8), 695–704. https://doi.org/10.1016/j.ergon.2006.04.009
Light, L., & Anderson, P. (1993). Typewriter Keyboards via Simulated Annealing. AI Expert. https://scholarworks.rit.edu/article/727
Norvig, P. (n.d.). English Letter Frequency Counts: Mayzner Revisited or ETAOIN SRHLDCU. Retrieved February 2, 2023, from
⠀⠀⠀⠀https://norvig.com/mayzner.html
Onsorodi, A. H. H., & Korhan, O. (2020). Application of a genetic algorithm to the keyboard layout problem. PLOS ONE, 15(1), e0226611.
⠀⠀⠀⠀https://doi.org/10.1371/journal.pone.0226611
Reid, D. B. C., Shah, K. N., Eltorai, A. E. M., Got, C. C., & Daniels, A. H. (2019). Epidemiology of Finger Amputations in the United States From
⠀⠀⠀⠀1997 to 2016. Journal of Hand Surgery Global Online, 1(2), 45–51. https://doi.org/10.1016/j.jhsg.2019.02.001
Saraf, S., & Tiwari, V. (2007). Fingertip injuries. Indian Journal of Orthopaedics, 41(2), 163–168. https://doi.org/10.4103/0019-5413.32051

February Fair Poster