So there we were, hanging on the edge of our seats, waiting for the test results of a test that could dramatically improve Windows Search efficiency. But if wasn’t for our Windows Insiders, this test might never have happened. Here’s the scoop on how Windows Insiders helped test the Windows Search Indexer.
First, a bit of background: The Windows Indexer organizes your files so that you can locate a document, photo or other file almost instantly through Windows Search. When it comes to search, speed is crucial — usability studies show that anything slower than 50 milliseconds results in a bad search experience.
Which bring us to index merges. As the Indexer moves from folder to folder, query speeds drop. When the Indexer detects that query speeds could be dropping, it merges indexes to increase performance. A fully merged index returns results up to 10x faster than an unmerged index.
But there’s a catch: merges require a lot of CPU — which, in turn, drains the battery. The more files you have, the greater the power drain. In fact, the biggest complaint that we heard from Insiders was that the Indexer uses too much battery. Here’s just one example from the Feedback Hub: Search indexer taking up too much CPU.
Fortunately, we’d been noodling a solution. Looking closely at those index merges, we discovered that the Indexer might actually be performing too many merges. Lower the threshold at which merges occur and our thinking was that we could improve battery life — without impacting search speeds.
From the lab to the real world
While the concept made a lot of sense in theory, the whole team was nervous about pushing out the change needed to make it happen. The Indexer has been around since the early ‘90s so there was a lot of code with the potential to break. Even worse, this change could break the Microsoft Edge address bar, Cortana search, and File explorer search at the same time. If something did go wrong, there was a strong possibility that we couldn’t push a fix until the next update.
Extensive testing was crucial. The threshold change we devised seemed to work well in the lab. But how would it work in the real world? Better question — where would we find thousands of users needed to validate the change?
That’s where the amazingness of the Windows Insider Program comes in. In fact, the only reason that I felt comfortable moving ahead with our indexer experiment was because I had seen the enthusiasm of Insiders to try new code and give fast and effective feedback.
Working closely with our friends on the data science team here in Windows, we started testing in the middle of June. To eliminate hardware variations as a possible cause of battery changes, we pushed out the change specifically to Insiders running Surface Pro 3 and Pro 4 devices.
And the results are….
Everyone was on pins and needles waiting for the feedback. I was asked “Are you sure about this?” more times in the weeks leading up to the experiment than ever before in my life.
Slowly the data started to trickle in. What we saw on our screens was amazing — despite the threshold drop, the query speeds were unchanged. Even more amazing — our heaviest users were gaining up to 10 minutes of additional battery life with the threshold change.
The test ran for 6 weeks last summer and, over that time, we gathered data from over 10,000 Insiders. The data gave us all the support we needed to push the change out to our production builds. We also used the testing to validate a bunch of other indexer improvements: a 40% reduction in the amount of processing required to index text heavy files, the addition of code to smooth out some spikey CPU usage, and dozens of fixes to prevent cases where the index is corrupted and needs to be repaired.
The finalized updates went out to all Windows users in the Windows 10 Fall Creators Update.
This is the kind of innovative work that we can only make with the help of Insiders. Having a group of excited folks who are willing to try all our new code makes it so much easier to test — and most importantly makes Windows better for everyone. All users of Windows around the globe are now going to have better battery life because of the work of Insiders.
I responded directly to all those Insiders who provided such valuable feedback. You can read our response here in the Feedback Hub. Check upcoming Windows 10 Insider Preview builds for even more exciting innovations to the Windows Search experience.
Thanks, as always, for being Windows Insiders.