On a desktop computer, a virtual keyboard might provide an alternative input mechanism for users with disabilities who cannot use a conventional keyboard, for multi-lingual users who switch frequently between different character sets or alphabets, which may be confusing over time, or for users who are lacking a traditional keyboard. Although hardware keyboards are available with dual keyboard layouts (e.g. Cyrillic/Latin letters in various national layouts), the virtual keyboards provides a substitute while working at different stations or on laptops.
Virtual keyboards can be categorized by the following aspects:
Virtual keyboards with touchscreen keyboard layouts or sensing areas[2]
Character variants, punctuation, and other special characters accessible through a menu and through holding buttons
Keyboard software may include a number pad feature to facilitate typing numbers.[3]
Virtual keyboards are commonly used as an on-screen input method in devices with no physical keyboard, where there is no room for one, such as a pocket computer, personal digital assistant (PDA), tablet computer, or touchscreen-equipped mobile phone. Text is commonly inputted either by tapping a virtual keyboard or finger-tracing.[10] Virtual keyboards are also used as features of emulation software for systems that have fewer buttons than a computer keyboard would have.
The four main approaches to enter text into a PDA were: virtual keyboards operated by a stylus, external USB keyboards, handwritten keyboards, and stroke recognition. Many early PDAs were not primarily focused on virtual keyboards. Microsoft's mobile operating system approach was to simulate a completely functional keyboard, which resulted in a slightly overloaded keyboard layout.[11] The main problem that early PDAs faced was support for multi-touch technology, and as a result, usability problems for the user.
First iPhone
When Apple presented the first iPhone in 2007, the decision not to include a physical keyboard was seen as a detriment.[12] However, Apple brought the multi-touch technology into their new device, which enabled them to overcome the usability problems of PDAs.
Implementation and use
Both the most common mobile operating systems, Android and iOS, give the developer community the possibility to individually develop custom virtual keyboards.
Android
Android Keyboard Settings
The Android SDK provides an "InputMethodService".[13] This service provides a standard implementation of an input method, which final implementations can derive from and customize, enabling the Android development community to implement their own keyboard layouts. The InputMethodService ships with it on Keyboard View.[14] While the InputMethodService can be used to customize key and gesture inputs, the Keyboard Class loads an XML description of a keyboard and stores the attributes of the keys.[15]
As a result, it is possible to install different keyboard versions on an Android device, and the keyboard is only an application, most frequently downloaded among them being Gboard and SwiftKey; a simple activation over the Android settings menu is possible.[16]
iOS
Apple iOS UIInputViewController
Apple also provides the possibility for the community to develop custom keyboards but does not give any access to the dictionary or general keyboard settings. Further, iOS automatically switches between system and custom keyboards, if the user enters text into the text input field.[17][18]
The UIInputViewController is the primary view controller for a custom keyboard app extension. This controller provides different methods for the implementation of a custom keyboard, such as a user interface for a custom keyboard, obtaining a supplementary lexicon or changing the primary language of a custom keyboard.[19]
Auto correction and spelling checker
The Android platform offers a spelling checker framework that offers the possibility to implement and access spell checking in the application itself. The framework is one of the Text Service APIs offered by the Android platform. Based on the provided text, the session object returns spelling suggestions generated by the spelling checker.[20][21]
iOS uses the class UITextChecker, an object used to check a string (usually the text of a document) for misspelled words, commonly known as Apple's autocorrection. UITextChecker spell-checks are using a lexicon for a given language. It can be told to ignore specific words when spell-checking a particular document and it can learn new words, which adds those words to the lexicon.[22]
Users may be able to add a custom dictionary of whitelisted terms that are treated by auto correction as usual words, and specify "aliases" or "text shortcuts", where entering a specified text string causes it to get replaced with a target text string, or the target text string appears as suggestion. The former means they are not replaced with other terms but may be corrected to from other terms. It may be possible to exclude unwanted existing suggestions.[23][24]
Word suggestions
Diverse scientific papers at the beginning of the 2000s showed even before the invention of smartphones, that predicting words, based on what the user is typing, assisted in increasing the typing speed.[25][26] At the beginning of development of this keyboard feature, prediction was mainly based on static dictionaries. Google implemented the predicting method in 2013 in Android 4.4. This development was mainly driven by third party keyboard providers, such as SwiftKey and Swype.[27] In 2014 Apple presented iOS 8[28] which includes a new predictive typing feature called Quick Type, which displays word predictions above the keyboard as the user types.
Gesture typing
iOS and Android allow developers to replace its keyboard with their own keyboard apps. This has led to new features such as the gesture-typing feature that's made its way into Android's official keyboard after being used in third-party keyboards. Research by Google itself confirmed that gesture-typing is increasing the typing rate by 22% and is decreasing the error rate near to 0%.[29] Google further showed that the gesture-typing method is also useful on smartwatches. Their scientific research is primarily based on research made by I. Scott MacKenzie and papers about modeling finger touch with Fitts' law.
Haptic feedback
Haptic feedback provides for tactile confirmation that a key has been successfully triggered i.e. the user hears and feels a "click" as a key is pressed. Utilizing hysteresis, the feel of a physical key can be emulated to an even greater degree. In this case, there is an initial "click" that is heard and felt as the virtual key is pressed down, but then as finger pressure is reduced once the key is triggered, there is a further "unclick" sound and sensation as if a physical key is respringing back to its original unclicked state. This behaviour is explained in Aleks Oniszczak & Scott Mackenzie's 2004 paper "A Comparison of Two Input Methods for Keypads on Mobile Devices" which first introduced haptic feedback with hysteresis on a virtual keyboard.[30]
Special keyboard types
Keyboards are needed in different digital areas. Not only smartphones need virtual keyboards, also devices that create virtual worlds, for example, virtual reality or augmented reality glasses, need to provide text input possibilities.
Optical virtual keyboard
An optical virtual keyboard was invented and patented by IBM engineers in 1992.[31] It optically detects and analyses human hand and finger motions and interprets them as operations on a physically non-existent input device like a surface having painted keys. In that way, it allows to emulate unlimited types of manually operated input devices such as a mouse or keyboard. All mechanical input units can be replaced by such virtual devices, optimized for the current application and for the user's physiology maintaining the speed, simplicity, and unambiguity of manual data input.
One example of this technology is the "Selfie Type" - a keyboard technology for mobile phones made by Samsung Electronics. It was intended to use the front-facing camera (the selfie camera) to track the user's fingers, enabling the user to type on an "invisible keyboard" on a table or another surface in front of the phone.[32][33] It was introduced at the Consumer Electronics Show 2020[34][35][36] and was expected to be launched in the same year but never did.
Augmented reality keyboards
The basic idea of a virtual keyboard in an augmented reality environment is to give the user a text input possibility. A common approach is to render a flat keyboard into augmented reality, e.g. using the Unity TouchScreenKeyboard. The MicrosoftHoloLens enables the user to point at letters on the keyboard by moving his head.[37]
Another approach was researched by the Korean KJIST U-VR Lab in 2003. Their suggestion was to use wearables to track the finger motion to replace a physical keyboard with virtual ones. They also tried to give audiovisual feedback to the user, when a key got hit. The basic idea was to give the user a more natural way to enter text, based on what he is used to.[38]
The Magic Leap 1 from Magic Leap implements a virtual keyboard with augmented reality.[39]
Virtual reality keyboards
The challenge, as in augmented reality, is to give the user the possibility to enter text in a completely virtual environment. One big issue is that most augmented reality systems on the market are not tracking the hands of the user. So many available systems provide the possibility to point at letters.[40]
In September 2016, Google released a virtual keyboard app for their Daydream[41] virtual reality headset. To enter text, the user can point at specific letters with the Daydream controller.[42]
In February 2017, Logitech presented an experimental approach to bring their keyboards into the virtual environment. With the Vive Tracker and the Logitech G gaming keyboard, it is possible to exactly track every finger movement, without wearing any type of glove. 50 of such packages were sent to exclusive developers, enabling them, in combination with Logitech's BRIDGE developers kit, to test and experiment with the new technology.[43][44]
Security considerations
Virtual keyboards may be used in some cases to reduce the risk of keystroke logging.[45] For example, Westpac's online banking service uses a virtual keyboard for password entry, as does TreasuryDirect (see picture). It is more difficult for malware to monitor the display and mouse to obtain the data entered via the virtual keyboard than it is to monitor real keystrokes. However, it is possible, for example by recording screenshots at regular intervals or upon each mouse click.[46][47]
TreasuryDirect login screen, showing the virtual keyboard
The use of an on-screen keyboard on which the user "types" with mouse clicks can increase the risk of password disclosure by shoulder surfing, because:
An observer can typically watch the screen more easily (and less suspiciously) than the keyboard, and see which characters the mouse moves to.
Some implementations of the on-screen keyboard may give visual feedback of the "key" clicked, e.g. by changing its color briefly. This makes it much easier for an observer to read the data from the screen. In the worst case, the implementation may leave the focus on the most recently clicked "key" until the next virtual key is clicked, thus allowing the observer time to read each character even after the mouse starts moving to the next character.
A user may not be able to "point and click" as fast as they could type on a keyboard, thus making it easier for the observer.
↑ Tom Stocky, Alexander Faaborg, Henry Lieberman (2004). "A Commonsense Approach to Predictive Text Entry".{{cite journal}}: Cite journal requires |journal= (help)CS1 maint: multiple names: authors list (link)
↑ Dunlop, Mark D. and Crossan, Andrew (2000). "Predictive text entry methods for mobile phones".{{cite journal}}: Cite journal requires |journal= (help)CS1 maint: multiple names: authors list (link)
A graphical user interface, or GUI, is a form of user interface that allows users to interact with electronic devices through graphical icons and visual indicators such as secondary notation. In many applications, GUIs are used instead of text-based UIs, which are based on typed command labels or text navigation. GUIs were introduced in reaction to the perceived steep learning curve of command-line interfaces (CLIs), which require commands to be typed on a computer keyboard.
A wearable computer, also known as a body-borne computer, is a computing device worn on the body. The definition of 'wearable computer' may be narrow or broad, extending to smartphones or even ordinary wristwatches.
A smartphone is a mobile device that combines the functionality of a traditional mobile phone with advanced computing capabilities. It typically has a touchscreen interface, allowing users to access a wide range of applications and services, such as web browsing, email, and social media, as well as multimedia playback and streaming. Smartphones have built-in cameras, GPS navigation, and support for various communication methods, including voice calls, text messaging, and internet-based messaging apps.
A voice-user interface (VUI) enables spoken human interaction with computers, using speech recognition to understand spoken commands and answer questions, and typically text to speech to play a reply. A voice command device is a device controlled with a voice user interface.
A tablet computer, commonly shortened to tablet, is a mobile device, typically with a mobile operating system and touchscreen display processing circuitry, and a rechargeable battery in a single, thin and flat package. Tablets, being computers, have similar capabilities, but lack some input/output (I/O) abilities that others have. Modern tablets largely resemble modern smartphones, the only differences being that tablets are relatively larger than smartphones, with screens 7 inches (18 cm) or larger, measured diagonally, and may not support access to a cellular network. Unlike laptops, tablets usually run mobile operating systems, alongside smartphones.
A projection keyboard is a form of computer input device whereby the image of a virtual keyboard is projected onto a surface: when a user touches the surface covered by an image of a key, the device records the corresponding keystroke. Some connect to Bluetooth devices, including many of the latest smartphone, tablet, and mini-PC devices with Android, iOS or Windows operating system.
TouchPal is an alternative input method for mobile devices, designed and developed by Shanghai-based CooTek. It is a software application running on multiple platforms, including Android, iOS, Windows Mobile, and Windows 8. It simulates a keyboard on the screen of the device, which is used to enter text by tapping on the screen or sliding a finger between letters constructing the word. It is an optional text input method to the traditional physical keyboards and default keyboards provided by the device manufacturer.
The HTC Dream is a smartphone developed by HTC. First released in October 2008 for $179 with a 2-year contract to T-Mobile, the Dream was the first commercially released device to use the Linux-based Android operating system, which was purchased and further developed by Google and the Open Handset Alliance to create an open competitor to other major smartphone platforms of the time, such as Symbian, BlackBerry OS, and iPhone OS. The operating system offers a customizable graphical user interface, integration with Google services such as Gmail, a notification system that shows a list of recent messages pushed from apps, and Android Market for downloading additional apps.
Swype was a virtual keyboard for touchscreen smartphones and tablets originally developed by Swype Inc., founded in 2002, where the user enters words by sliding a finger or stylus from the first letter of a word to its last letter, lifting only between words. It uses error-correction algorithms and a language model to guess the intended word. It also includes a predictive text system, handwriting and speech recognition support. Swype was first commercially available on the Samsung Omnia II running Windows Mobile, and was originally pre-loaded on specific devices.
Mobile technology is the technology used for cellular communication. Mobile technology has evolved rapidly over the past few years. Since the start of this millennium, a standard mobile device has gone from being no more than a simple two-way pager to being a mobile phone, GPS navigation device, an embedded web browser and instant messaging client, and a handheld gaming console. Many experts believe that the future of computer technology rests in mobile computing with wireless networking. Mobile computing by way of tablet computers is becoming more popular. Tablets are available on the 3G and 4G networks.
Microsoft SwiftKey is a virtual keyboard app originally developed by TouchType for Android and iOS devices. It was first released for Android in July 2010, followed by an iOS release in September 2014 following Apple's implementation of third-party keyboard support.
This is a comparison of mobile operating systems. Only the latest versions are shown in the table below, even though older versions may still be marketed.
Fleksy is a third-party, proprietary virtual keyboard app for Android and iOS devices. It attempts to improve traditional typing speed and accuracy through enhanced auto-correction and gesture controls. Fleksy uses error-correcting algorithms that analyze the region where the user touches the keyboard and feeds this through a language model, which calculates and identifies the intended word. Swiping gestures are used to control common functions, such as space, delete, and word correction.
Samsung Knox is a proprietary security and management framework pre-installed on most Samsung mobile devices. Its primary purpose is to provide organizations with a toolset for managing work devices, such as employee mobile phones or interactive kiosks. Samsung Galaxy hardware, as well as software such as Secure Folder and Samsung Wallet, make use of the Knox framework.
MessagEase is an input method and virtual keyboard for touchscreen devices. It relies on a new entry system designed by Saied B. Nesbat, formatted as a 3x3 matrix keypad where users may press or swipe up, down, left, right, or diagonally to access all keys and symbols. It is a keyboard that was designed for devices like cell phones, mimicking the early cell phones' limited number of 12 keys.
WebXR Device API is a Web application programming interface (API) that describes support for accessing augmented reality and virtual reality devices, such as the HTC Vive, Oculus Rift, Oculus Quest, Google Cardboard, HoloLens, Apple Vision Pro, Magic Leap or Open Source Virtual Reality (OSVR), in a web browser. The WebXR Device API and related APIs are standards defined by W3C groups, the Immersive Web Community Group and Immersive Web Working Group. While the Community Group works on the proposals in the incubation period, the Working Group defines the final web specifications to be implemented by the browsers.
Kubity is a cloud-based 3D communication tool that works on desktop computers, the web, smartphones, tablets, augmented reality gear, and virtual reality glasses. Kubity is powered by several proprietary 3D processing engines including "Paragone" and "Etna" that prepare the 3D file for transfer over mobile devices.
Samsung Experience is a discontinued software overlay for the Android "launcher" by Samsung for its Galaxy devices running Android 7.x “Nougat” and Android 8.x “Oreo”. It was introduced in late 2016 on a beta build based on Android 7.0 “Nougat” for the Galaxy S7, succeeding TouchWiz. It has been succeeded in 2018 by One UI based on Android 9 “Pie” and later versions.
Bootloader unlocking is the process of disabling the bootloader security that makes secure boot possible. It can make advanced customizations possible, such as installing a custom firmware. On smartphones this can be a custom Android distribution or another mobile operating system. Some bootloaders are not locked at all, others can be unlocked using a standard command, others need assistance from the manufacturer. Some do not include an unlocking method and can only be unlocked through a software exploit.
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
