Improving the Windows input experience

Three students look at a lap top together.

Input is at the core of what you do when interacting with your PC, and today we’d like to share some of the updates our Input and Language team have been making to the input features in Windows.

What exactly is input? Input is what we use to talk about features used to enter data, whether you’re typing on a hardware or touch keyboard, writing with a pen, or dictating with your voice. Do any of these things on a Windows 10 Insider Preview Build, and chances are, you’ve made an impact on improving Windows input features for users in multiple languages around the world.

So how has the Windows input experience gotten to where it’s at now, and how are Windows Insiders helping us determine what’s next? Let’s take a look!

Insiders weigh in on keyboards, emoji & handwriting

Before 19H1, one the most highly requested improvements was for the integration of shapewriting with the new touch keyboard, a change we made with the Windows 10 Fall Creators Update. We also introduced another very popular Insider request, the ability for hardware keyboard users to easily input emoji in Windows. Press WIN + (period), or WIN + (semicolon) to bring them up.

As we moved into the next release, the April 2018 update, Insiders continued to help shape input developments. We started by adding an acrylic background to the keyboard as part of our efforts to incorporate Fluent Design across the system. Later, based on your requests, we also expanded our shapewriting support to the wide touch keyboard layout, along with other improvements to the touch keyboard. We also extended language support for emoji, bringing it to 190 locales, with emoji type-to-search support available in 150 of them.

 

Handwriting panel with keyboard.
Insiders inspired several handwriting panel improvements, including a better button layout and more intuitive gestures.

Next, we turned our attention to an entirely different input challenge: the handwriting panel, which lets you use a pen (or your finger) to enter words in Windows. Based on your feedback, we started by changing the button layout to give you a more streamlined experience when inserting text. Insiders also made some good suggestions about gesture recognition, which led to simpler, more intuitive gestures, like the insertion gesture to quickly add space between words or letters and commit gesture.

How Insiders helped shape our latest innovations

We evaluate Insider features in a variety of ways. This includes telemetry from devices running Insider Preview builds and feedback submitted in the Feedback Hub. We also sometimes do controlled feature rollouts and A/B studies, which help us see the impact of certain changes and roll them out with confidence and quality.

Here are a few highlights that we rolled out during 19H1, with Insiders’ help:

  • Increased typing accuracy with hit target resizing – Improving the experience when typing with the touch keyboard has been an ongoing topic with our Insiders, and the latest change we’ve made in this space was for dynamic hit target resizing. What does that mean? Basically, before now, you had to be really precise when it came to touching the right key. Easy enough when typing slowly, but as soon as you start to type quickly, your fingers tend to slip a little, and the key you end up pressing might not be the one you meant to. This 19H1 feature dynamically adjusts the hit target of each key, which is the area that will count as pressing a key, based on predictions of what letter most likely will be typed next. Sounds cool—but would it actually pay off for users?To find out, we started by rolling out the feature to 50 percent of our Insiders. Sure enough, for Insiders who got the change, we saw a 15 percent drop in use of the backspace key compared to our control group. The change was a success! We’d proved that it helped reduce the need to go back and correct text, so we shipped it to all Insiders as part of the 19H1 release.

 

Close up of touch keyboard.
A representation of what the touch keyboard looks like with hit target resizing.
  • Easier access to kaomoji and symbols – It’s hard to remember all the Alt key combos! One of the things we heard from Insiders was that they wanted an easier way to type special characters in Windows. In RS5, we introduced kaomoji and symbols into the same panel we have for emoji for Chinese (Simplified), based on data from the Pinyin IME.(You might ask, what’s kaomoji? Basically, it’s using characters to create characters! Like ¯\_(ツ)_/¯ (shrug) and ༼ つ ◕_◕ ༽つ (gimmie).)From there, we wanted to see if adding kaomoji and symbols views would appeal to a broader audience. We updated the symbols and kaomoji to work for all languages, and sent the update to a group of Insiders. We saw a 10 percent increase in usage when we included  them in the panel. Haven’t tried it yet? Press WIN+(period) on your Insider device to bring up the panel. You can then switch between emoji, kaomoji, and symbols using the categories at the top.

 

Examples of Expressive Input Panels.
A request from Insiders for kaomoji and symbols in the emoji picker led to a jump in usage.
  • Expanded Clipboard history – Based on your feedback, we introduced a new clipboard experience (WIN+V) in RS5, where you could save multiple copied text clips and images to access later. One of the things we heard most was that the UI wasn’t compact enough, that each item in Clipboard History was “taking up too much space” or was “visually way too big.”We heard you, and based on our analysis of clipboard history usage, we shrunk the height of each entry, giving you access to more entries without needing to scroll. We then introduced the experience as a Quest in the Feedback Hub. Feedback about the new design was great, with Insider comments like this, “You’ve already made it better by letting it show more copied items,” or “UI change makes Clipboard more in line with existing Windows Design language.”
  • East Asian Microsoft Input Method Editor (IME) Improvements – During 19H1, we began work on a new Japanese IME. Insiders loved the updates, but they also saw room for improvement, so we decided to take the IME offline for a while to do more work based on your suggestions. We did the same for the Simplified and Traditional Chinese IMEs. Recently with 20H1, we re-introduced these IMEs and are happy to announce that they’re more secure, more stable, and easier to use than ever.

What’s next

Our article on input wouldn’t be complete without a call-out to the Vietnamese Telex and Number key-based keyboards. This update applied to just one language, but here’s something that might surprise you. It was one of the top-10 most upvoted feature requests by Insiders in 19H1! This proves what we’ve known all along. The typing experience in all languages is a critical part of the overall Windows experience—just like the Insiders who help drive it.

This is only a selection of input-related highlights we’ve been working on, and we hope you’ve found it interesting! As we continue down the latest release path, we’ve got even more updates ready for you to try, including expanded language support for dictation, expanded SwiftKey typing intelligence, and more. We love your feedback, so please keep it coming—it helps us make Windows better for everyone.