The Team

Publications

Improving Text-entry Experience for Older Adults On Tablets. In Proceedings HCII2014.

Developing a Multimodal Interface for The Elderly. In Proceedings DSAI 2013.

Improving Text Entry Performance On Tablet Devices. In Proceedings of the 5th Portuguese Conference on Human-Machine Interaction.

PAELife Élvio Rodrigues. MSc Dissertation.

Related Research

PAELife - Older Adult Performance Using Body Gesture Interaction

Funding

AAL PAELife Project AAL/0014/2009

Accessible Virtual Keyboard for Seniors

Touchscreen interfaces are increasingly more popular. Since they allow directly interacting and manipulating the information displayed on screen, it is considered one of the easiest technologies to use. However, they lack haptic feedback, making it harder to perform certain tasks. This is the case of text-entry, where users have to constantly select one of many small targets. This problem particularly affects older users, whose deteriorating physical and cognitive conditions, combined with the unfamiliarity with technology, can discourage them from using touch devices. We analyzed the performance and behavior of 20 elderly when inputting text on a tablet, testing a baseline QWERTY keyboard, as well as 2 variants that use text prediction. We also performed a simulation study with 6 other variants, taking into account the typing behavior learned in the user study.

The problem

Since touch devices are easier to use than the traditional computer interfaces, they present an opportunity for user groups that, until now, have shown some resistance in adopting technology. The fact that this technology interface relies more on software than hardware makes it highly flexible, and thus easy to adapt to users' needs. However, it also has the disadvantage of lacking the haptic feedback of physical buttons, making it harder to accurately select targets. This characteristic particularly hampers certain tasks, such as text-entry, where the user has to constantly select one of many small targets. Moreover, since text-entry is a task transversal to many applications, it particularly affects users that have difficulties in aiming and performing movements that require precision.

That is the case of the elderly, whose deteriorating physical and cognitive conditions combined with the unfamiliarity with technology, deprives them from the innumerous opportunities created by touch devices. Furthermore, the lack of experience with the QWERTY layout can discourage them from using this technology. Although there is a large body of work that tries to understand the touch behavior and improve the typing experience on touchscreens, studies that target the elderly are few. Since the requirements for elderly users are different due to the declining of their motor and cognitive abilities, the solutions found for young adults may not be suited for seniors.

Therefore, we performed a study to better understand how we can improve the typing speed and/or reduce the error rate of older adults on tablets. Also, we take into account that the elderly may have little or no experience with the QWERTY layout, and thus developed 2 QWERTY keyboard variants that aim to aid the elderly typing. Particularly, we aim to aid users that are not completely familiarized with the QWERTY layout.

Developed QWERTY Variants

Due to the lack of haptic feedback, text-entry remains slower and more error-prone on touch devices than on traditional computer keyboards. Since one of our goals is to aid new users to input text, without hindering older users who are already experienced with QWERTY keyboards, we developed alternative keyboards based on the QWERTY keyboard layout. After developing the regular QWERTY keyboard to serve as a baseline, we developed 2 variants, which use text prediction to anticipate what the user is going to write.

Color Variant

The Color keyboard variant uses the developed letter prediction algorithm to highlight the next most likely letters for the current word. We expect this variant to perform better than the regular QWERTY keyboard, especially if the user is not acquainted with the QWERTY layout, which might be the case of older adults. By highlighting the most likely letters, senior users may find the desired letter quicker. We also expect that users make fewer errors by noticing they are about to press a key that is not highlighted, or by acknowledging they missed or omitted a key press. We decided to highlight four keys, since a previous study concluded that it was the optimum number between 1, 2 and 4 keys. The highlighted key changes its color from black to gray to avoid the cultural connotations that are associated with particular colors (e.g.: the green and red colors may have positive and negative connotations, respectively). We also increase the size of the key's label. The highlight is continuous: the more probable the letter, the brighter the color and bigger the label on the key.

Predict Words Variant

This variant is an alternative already used in some touch devices. While the user is typing, a list of the most likely words is shown in a horizontal ribbon above the keyboard. If the word the user wants to write is on the suggested list, he can save some key touches by tapping it so the full word, along with a space character, will be inserted. While this solution is fairly popular with younger users, we are not sure if it will be adequate for older users. Since there is a cognitive effort required to process the list of suggested words, it might be harder for this group of users to divide their attention between the actual typing and the scanning of the suggestions' list. We opted to suggest 4 words to achieve a balance between the success rate of the prediction and the cognitive effort required to process the suggestions list (which increases with the number of words).

Results of the user study

Twenty participants, 15 females and 5 males, took part in the user study. We asked participants to copy several sentences that were sequentially displayed at the top of the screen. After typing the current sentence, the user could proceed to the next sentence by pressing a button. Our application would log data regarding touch and timing information, so we could analize it in detail.

Speed and errors

A repeated measures ANOVA revealed significant differences between keyboard variants on text-entry speed (F(2,30)=3.84, p< 0.033). Bonferroni post-hoc tests showed significant differences between QWERTY and Color variant, meaning that users type significantly slower with the latter. This result contradicted our hypothesis that inexperienced users, who are not acquainted with the QWERTY layout, would benefit from the Color variant. We believe that the main reason for the lower input rate in the Color variant is because the highlighting of the keys was distracting. However, no user reported this. Regarding the Predict Words variant there was no significant difference when compared with the QWERTY keyboard. Still, we must take into account that only 7 of the 20 participants accepted at least one suggested word from the list during evaluation; the remaining 13 participants used the Predict Words variant as a normal QWERTY keyboard.

We expected both Color and Predict Words variants to outperform the QWERTY keyboard regarding MSD. However, no statistical significant differences were found between variants. Although we are not sure why the Color variant did not outperform the QWERTY keyboard, several situations occurred that are important to report. For instance, one participant ended up typing a word similar to the expected one because the Color variant suggested it, and he tapped the suggested letters without thinking too much. The performance of the Color variant was also affected by the fact that older users committed many errors. This means that the Color variant cannot make good suggestions, because once there is an error in the current word, the system is not able to correctly predict the sequence of letters intended by the user. The Predict Words variant also had a MSD similar to QWERTY, mainly because most participants (13) did not accept any suggestion.

Simulation study

By using the touch data gathered from the user study, we can simulate touches as if users were typing on other keyboard variants. For the simulation, we used the touch data gathered on the QWERTY keyboard, since all simulated variants do not have visual changes: they only differ from QWERTY in the attribution of a key when a touch is recognized. Although a simulation is not as good as running real user tests, it can give us some insights on the impact of additional keyboard variants. We simulated the following variants:

Shifted variant. The approach of shifting the real touch area of keys (or shifting the point of touch, which yields the same result) from its visual representation have been reported to improve the error rate when typing in smartphones. However, no systematic studies have been performed for tablet devices. These devices vary from the former not only in screen size, but also in the typing posture users assume when using them: in smartphones, users usually type with the two thumbs while grabbing the device; in tablets, users usually type with the tablet resting on a surface, and then can use multiple hands and/or fingers to type. A Wilcoxon test revealed that the Shifted variant significantly improved the MSD in comparison to QWERTY (Z=-2.746, p=.006).
Size increase variant. Another approach to correct neighbor substitution errors is to internally increase the size of the most likely keys, while the keyboard remains visually static. This approach has already been proved successful for adult users. In our solution, the size of the key is increased in a rectangular shape with rounded corners. If two adjacent keys are highlighted and a touch occurs in an ambiguous area, the original boundaries of keys are preserved. With this variant we expect users to commit less substitution errors by hitting the desired key instead of the neighbor keys, since the most likely keys are internally bigger. A Wilcoxon test revealed significant improvements in the MSD between QWERTY and Size Increase variant (Z=-2.073, p=.038).
Single Touch variant. We decided to simulate a Single Touch keyboard because we noticed that some older users ended up unintentionally touching the surface of the tablet with more than one finger or other part of the hand, which lead to many accidental insertions. Other users sometimes touched the space bar with two fingers, but did not realize that it meant that both touches were recognized, leading to an insertion. We found that the MSD of this variant was significantly better when compared to QWERTY (Z=-1.223, p=.221), although at a lower significance level (77.9%). We hypothesize that this is because few participants highly benefited from this variant, while most of participants were not affected at all, or they were very slightly affected.
Inter-key Timed variant. This variant aims to correct accidental insertion errors when users quickly insert two characters instead of one. These insertions are characterized by a reduced time interval between the release of the first key and the press of the second key. In this variant, if the minimum threshold between keys is not respected, that touch is discarded and no output is produced. We found that with a generic threshold set, the MSD improves 5.7%. However we also found that this time interval is highly user-dependent. When using an adaptive solution, i.e., the best threshold for each user, we improve the MSD in 11.4%. Indeed, we found significant differences in MSD between the Inter-key Timed adaptive solution and QWERTY (Z=-2.373, p=.018) and also with Single Touch (Z=-3.006, p=.003).
Intra-key Timed variant. The Intra-key Timed variant is aimed to correct accidental insertion errors within the same touch interaction, that are characterized by a reduced time interval between the press and release of a key. We found that intra-key time interval is highly user-dependent, and even within a specific user, it can vary a lot. This makes it very difficult distinguishing accidental touches from regular touches with this approach. Even considering a solution adapted to each user, a Wilcoxon test revealed no significant differences in the MSD of this variant and QWERTY, nor this variant and Single Touch.

Design implications for virtual keyboards

From our results, we derive the following design implications for text-entry solutions in tablet devices for the elderly.

Keep visual changes to a minimum. As verified in the user study, visual changes that aim to focus the user attention on the most probable keys have a negative impact in text-input speed. Also, the Color variant had twice the cognitive substitution errors, when compared with the traditional QWERTY. When further analyzing this type of error, we found out that 44.8\% of the times, the substitution character inserted by the user was highlighted, while the expected one was not. Although we cannot tell for sure what led users to press the wrong character, we think they were influenced by the highlighting of the keys. Therefore, visual changes should only occur to give feedback about the pressed and released key.
Shift the touch points to the top and to the opposite side of the hand the user is using to type. Different touch patterns emerge from different interaction techniques. For instance, users who used only their right hand to interact with the virtual keyboard had a tendency to touch on the bottom-right of targets. This means that users will benefit from a top-left shift of their touch points to compensate the tendency. Conversely, users who only used their left hand benefit with a top-right shift of their touch points. Users who interact with both hands will benefit with a top-left shift of the touch points performed on the right side of the keyboard, and a top-right shift of the touch points performed on the left side of the keyboard. If it is not possible to detect the user's hand posture, an upward vertical shift of touch points will also benefit users. We found that the best result was obtained with a horizontal shift of 5% of keys' width and a vertical shift of 10% of keys' height.
When a vertical slide occurs between two keys of subsequent rows, produce the character in the row above. We verified in our user study that, when users perform a vertical slide from one row to a subsequent row (up or down), 96.4% of the times the user intends to select the key from the row above. In the remaining 3.6% times, we do not know exactly what were the intentions of the user were, since the slide was an accidental touch in the touchscreen.
Choose single touch over multi-touch. Older users are all different with different necessities and capabilities. Regarding a generic keyboard that should fit all types of older users, single touch is the right choice. The quality of the sentences of the 2 most problematic participants in our user study increased drastically, while it only slightly prejudiced some other participants and had no effect at all on most participants.
Omit touch interactions that are below a certain threshold (inter-key interval). During the simulation study, we verified that the Inter-key Timed variant was able to increase the quality of the transcribed sentences, when compared with the Single Touch variant. This means that, having a keyboard that omits interactions based on the time elapsed between the release of a first key and the press of a second key can enhance older adults' error rate. The threshold value must be defined carefully since this value is highly user-dependent, and a generic solution can discard too many false positives. However, we found improvements from the Single Touch variant to the Inter-key Timed variant, with a generic inter-key interval set to 125ms.
Allow personalization. We found patterns that emerge from the different interaction methods (only right hand, only left hand or both) adopted by the participants. Still, when looking at each participant of the same group, we can verify that some of them have different typing behaviors, particularly hit point locations and inter-key interval. Therefore an adaptive model that constantly updates the center of each individual key seems the best solution.