< Back to podcasts archive

Business or Personal? A Little Bit of Both.

December 11, 2019

In this episode, we’re looking to the future with an important feature announced at Microsoft Ignite, the Update Staging Lab and a product that’s always searching for what’s next—Bing.

First, we’re joined by Raji Rajagopalan and Rama Shastri to talk about how their team’s new service, Update Staging Lab, which will help independent software vendors validate their app’s compatibility with Windows faster. Then, Michael Kroll joins us to talk about Bing’s content strategy, the Bing Insider Program, Microsoft Rewards, and the future of search.

Windows Insider Podcast Ep 26: Business or Personal? A Little Bit of Both.


JASON HOWARD: Welcome to the Windows Insider Podcast, where leaders from Microsoft and Windows Insiders discuss tech trends, careers, and innovation. I'm your host, Jason Howard.

This is Episode 26, Business or Personal? A Little Bit of Both.

But first, if you're not yet a Windows Insider, head over to our website, insider.windows.com, and register for free. Insiders get access to upcoming Windows features before they're released to the public, plus exclusive opportunities to experience all Microsoft has to offer. All right, on to the show.

As we wind down 2019, we'll be looking to the future with an important feature announced at last month's Microsoft Ignite Conference, as well as a Microsoft product that's always searching for what's next—Bing.

Up first, Raji Rajagopalan is back with Rama Shastri to talk about the new service their team announced at Ignite for independent software vendors, or ISVs, the Update Staging Lab. This is especially exciting for our business Insiders.

Then, we'll be joined by Michael Kroll to take a look at Bing, their Insider program, some of what they announced at Ignite, and a whole bunch more.

Without further ado, I'm excited to welcome our first guests, Raji Rajagopalan and Rama Shastri to share details about their new service, the Update Staging Lab.

Welcome to the podcast, Raji and Rama. Can you please introduce yourselves and tell us a little bit about your roles here at Microsoft?


RAMA SHASTRI: Thanks, great to be here, Jason.

JASON HOWARD: So real quick for the listeners out there, can you please introduce yourselves and tell us about your roles here at Microsoft?

RAJI RAJAGOPALAN: Yeah, my name is Raji Rajagopalan. I actually was featured on the previous podcast as well about inclusive technology, where I talked about the Africa Development team. I am an Engineering Director at Microsoft. I have been at Microsoft for about 14, 15 years now. I don't remember all the details of what exactly happened those 14, 15 years. (Laughter.)

But I've had a fun time at Microsoft. I run a team in Azure now. It's a team called Update Staging Lab. It's a brand new service that we announced at Ignite last month, and it's a service for our ISVs, which Rama will talk about more-more in details. But I've been running this team for about a year now.

Before that, I ran the Engineering team for Desktop Analytics, another tool that was loved by—that has continued to be loved by our customers. We—Rama and I actually, ran that team as well. And then, I run a team in Africa Development Center in Lagos, an engineering team there as well. Quite excited about all these exciting things that are happening in my life right now.

JASON HOWARD: Got a lot going on.

RAJI RAJAGOPALAN: Yeah, for sure.

RAMA SHASTRI: I'm Rama Shastri. I'm the Program Management leader and Raji's counterpart on the Update Staging Lab team. Super excited to be here. As Raji explained, you know we—I started my role at Microsoft in the Microsoft Partner Network organization, did some work with partners. You know, it's just come full circle where we're now working again with partners.

Did a stint in the Windows App Compat team, and that's where we got a lot of ideas around the Analytics project we ran. I have a great interest in running startup teams, and that's what Update Staging Lab is about. It takes a different kind of a-a culture to run a startup team. That's what excites me to come to work every day.

So, super excited to be here. And then you layer on the stuff around Africa Development Center and the far-reaching consequences of doing business on another continent, that's just super exciting. And so Update Staging Lab is a—is a dream come true.

JASON HOWARD: Before we dig further into this new feature that your team's working on, can you give me a little bit of background about why app validation is important, app compat—these are terms that we've talked about to the Insiders kind of broadly, like in past podcasts and on the webcast. But like, generally, as a concept, like why is this so important to Windows?

RAJI RAJAGOPALAN: Right. So, before USL, Update Staging Lab, and before Desktop Analytics, I was actually running engineering for Windows App Compat, so I know a lot of things about app compat, as well. So, I kind of feel like, oh yeah, I mean, I'm the right person to be running this team and also to be talking about app compat, as well.

I-I used to talk about app compat quite a bit when I did conferences in the past, and pretty much a lot of my opening lines were about how Windows 10 is the most compatible Windows ever. And it's the most secure operating system ever.

And I know that because my team, Windows App Compat team, we ran tests, monthly app compat tests, on thousands of applications, thousands of applications that are popular in the ecosystem. And what we found was that the, you know—we actually did a lot of work to make sure the compatibility bar was pretty high, and we maintained the compatibility for every upcoming release of Windows.

But every time we released Windows, we kept the promise of our app compat pretty high as well. So, at that time, when I was running App Compat team, it was 99 percent of applications were consistently kept compatible. And now, it's actually 99.8 percent.

JASON HOWARD: So, I mean, folks out there listening can't see my face, but there is a bit of an exasperated look on my face.

RAJI RAJAGOPALAN: Yeah. I like the—I like that look. I like that face, but I like that look, as well. (Laughter.)

JASON HOWARD: That's a pretty high bar you’ve set for yourselves.

RAJI RAJAGOPALAN: For sure, and I know this because it's not just from the-the data that is coming from our internal testing, Microsoft's internal testing. Are you familiar with this program called Desktop App Assure?


RAJI RAJAGOPALAN: So Desktop App Assure, for folks that are listening that don't know what it is, it's a program that we announced last year at Ignite, and it was because we had this such high app compat promise to our customers that over 99 percent of our applications would be compatible, that if anything broke in your applications on the upcoming release of Windows, we promised our customers that we'll fix the application for them for free, right? We'll take on the cost, and that's because we were so confident in our app-app compat promise.

And we are constantly monitoring the data that is coming back from this program, and the data is telling us that we were actually right. You know, 99.8 percent of our applications are compatible. And so, we are able to run this program in a sustainable manner, as well.

But given all of that, what I said, that we were pretty rigorous about how we are testing Windows. We are pretty rigorous about how we are maintaining our app compat, setting the bar for app compat pretty high, monitoring data, listening to customer feedback—all of that—we still realized that our customers, our enterprise-enterprise customers, app compat is a very important thing for them.

And that's because, you know, for our Windows customers, it’s-it's our responsibility, from Microsoft point of view, it's our responsibility to not just keep them protected on Windows to make our operating system the most secure operating system ever, but also to keep them productive, right? So, their applications need to continue working.

And our enterprise IT professional customers, they are testing their line of business applications, their-their business-critical applications to make sure that their users will-will continue to be productive, and their experience will not get disrupted. And we want to provide them the tools and technologies, the data, and insights to make sure that, as they go about testing their business critical applications, as we—as they roll out Windows to broader and broader populations within their enterprise—it's easy, it's cost-effective, and it doesn't take a lot of work from their side.

So, that's why it's important for us to build this kind of technology, the Update Staging Lab kind of technology which gives them an intelligent and cost-effective way to-to do that testing.

JASON HOWARD: That's awesome. Like (laughter), it's just like—I'm-I'm still stuck on the stat itself of such a high bar having been set. And just out of curiosity, is this include enterprises, like their in-house built like line of business apps, or is this only commercially available software that kind of the whole globe could potentially install? Like, where does that line get drawn between stuff they're doing versus stuff everybody could potentially do?

RAJI RAJAGOPALAN: We announced our private preview at Ignite, and the MVP that we have released is for commercially available applications, and it's a joint testing environment for ISVs, the application developers, and Microsoft to exercise these commercially available applications that are still used heavily by enterprises.

So, we already have been working with a number of ISVs out there to make sure that we are co-developing these applications, the Update Staging Lab with our customers. But we have our-our ask for at Ignite was that enterprises nominate the third-party ISVs that they have been working with so that, you know, their applications are also on USL.

JASON HOWARD: And real quick, just-just because acronyms being what they are, ISV is Independent Software Vendor?


JASON HOWARD: Okay, perfect. Since we're talking about ISVs, right, so these independent software developers, vendors—you know, whatever we choose to call them—historically, through their validation up until this point and time, what were some of the issues they were facing, and like what has—what has their process been up until now, really, is what I'm trying to ask?

RAMA SHASTRI: Let me maybe touch upon this, and maybe this is a little bit of a history lesson. I think there was a turning point when we started with Windows 10. With Windows 10, we introduced this concept of Windows as a Service, where the pace of innovation with Windows became fast. And that was done primarily based on feedback from our customers as well as partners saying, "Don't go ship Windows every five to seven years. You know, give me the security and the productivity innovations that you have faster."

When we moved to that Windows as a Service model, it actually started putting pressure on our partner ecosystem, as well as our customers, because now, we're cranking out Windows, what we call the semi-annual channel, twice a year. And then, on top of it, you layer on the burden of monthly security updates, which are coming fast, as well.

In recent times, the security updates have become more and more invasive, where the security vulnerabilities have cost us to go deep into the OS code to make changes.

Now, if you take the ISVs, they have been validating, where we release an update, and then they pick up the released update, and they go off and go do testing for weeks or months. And then, they will come back with some sort of a confidence signal that says, "You know what? My app is ready for this build of Windows." Right?

There are a couple of things that are—that are problematic from a customer's point of view. Microsoft releases Windows, and then these business critical apps that customer are dependent on don't have any signal from their ISV owners, so customers are-are sitting on a—on a build of Windows and waiting for the ISVs to signal to them that they need—they are good to go. Right?

If you pivoted from the ISVs perspective, there are three fundamental things that they are struggling with. One is with us releasing faster, and then if you add on their application update, and they are working on new versions of applications, their test matrix is growing. They have to support different forms of their applications—you know, browser applications, mobile applications, desktop applications. They have to do different kinds of testing. They have to do regression testing, user acceptance testing, performance testing. Their test matrix is growing, and we keep on releasing more and more Windows on top of it, right? So, that's a challenge that ISVs have to deal with.

JASON HOWARD: It's that combination of the number of touchable surfaces that users will interact with their product in.

RAMA SHASTRI: That's right.

JASON HOWARD: On top of the number of Windows releases, you know, the next one that's being developed, plus X number of revisions to the past where users are still actually, like, using that product.

RAMA SHASTRI: Absolutely, and-and think of it this way: from an engineering and a test capacity perspective, their capacity is not increasing. It's the same number of people. It's the same number of money. It's the same amount of money, and they're probably under-under pressure to go cut costs, just like any other organization. Then how do they prioritize that testing? That's a big challenge for them.

The second aspect of this is, they spend a significant amount of time actually deploying, configuring test environments that they maintain, and these test environments are constantly changing based on whatever change vector comes down. It could be an app update. It could be a platform update that Microsoft releases. It could be policy update or configurations updates. It could be hardware updates that OEMs are making. So how do they maintain all of this test infrastructure in a meaningful way, so that they are able to test all of these change vectors, and then be able to still generate a confidence signal that enterprise customers can use?

And then, you know, we have—we've kind of tried to make it easy. Microsoft obviously has the responsibility to bring this app ecosystem along. We've introduced this big change. I think our customers have embraced it. We want to make sure that we—our partners are coming along on this journey, so we've made it easy with programs like Windows Insider where we are—we are being more transparent. We release all of our in-development builds.

But then, I think the other option in the ISV ecosystem hasn't been great, and how do we encourage our ISVs to pick up more and more of these in-development builds and be able to test their apps against them, so that they can—they can find issues early, they have more time to fix the issues, and then they impact the end user ecosystem even less. I think those are some of the challenges our ISVs are grappling with.

JASON HOWARD: Totally sensible, right? So pivoting a little bit from the ISV side of things and looking back at the enterprise, which I know you touched on some—a little bit already, but looking at the way they're doing their current analytics, what was it that you saw that-that kind of highlighted the need for a new solution, right, to help both ISVs as well as these enterprise customers kind of resolve their compat issues a little faster? Maybe there was a better way of doing it. Like, what was kind of that lightbulb moment where you were like we really have some new territory to go explore?

RAJI RAJAGOPALAN: Right. So, I think there's-there's one thing that I want your-your listeners to take away from this podcast. It's actually for Update Staging Lab. There are three differentiators that are three things that they are getting out of this service, right?

Number one is the ability to get access to prerelease updates is unprecedented in a—in a way where, you know, it can be given to a scaled-out audience. We work with a very small number of very close partners right now to get—give access to prerelease updates. But this is the first time, you know, a number of people are able to get access to prerelease updates, including ISVs, but in future, enterprise IT professionals, as well. So, access to prerelease updates is number one.

And number two is intelligence on how their tests and their applications are doing against the prerelease updates, right? So, that's number two.

And number three is having the-the ability to work with Microsoft and get Microsoft's expertise in debugging any app failures and-and problems that they're facing in the test cases that they're running on prerelease updates.

So, those are the-the three things that I want your-your listeners to be able to understand that Update Staging Lab brings to the table, right?

Now, with all of those three things, how does that help them, enterprise IT professionals, but also ISVs, is that it gives them access to the-the prerelease updates much faster than zero day, which is when most customers today have access to our-our Windows updates, right, which means that it pulls in their lead time for rolling out updates.

So if you were testing the Windows updates starting on zero-zero days, and you took typically three weeks to roll out, or three months to roll out your updates currently, now you're able to fast—you know, rewind your-your testing, pull it in a little bit, so you're actually as code complete happens for our Windows update release, as Microsoft hits code complete, you already can start testing, because it's happening in the cloud environment as soon as we detect that a package has been made available, right?

So, it really pulls in the lead time for-for updates, and that way, you can resolve your app compat issues. If there are any issues that crop up, you can resolve them faster because you have access to the—to these build faster, right?

And how can they resolve them better by having all this world-class intelligence on how exactly the application that you own is being impacted by the upcoming release?

So, it's sort of the-the three things that we have with the availability of Microsoft expertise, access to prerelease updates, and the intelligence that you're getting, we feel like it's going to make the current process a whole lot more effective.

JASON HOWARD: So, comparing the changes in this environment with the Update Staging Lab and kind of looking in a comparative fashion to the Insider program, especially if you want to drill down into like the WIP for business efforts that tend to be for enterprise, commercial, IT professionals, are-are the ISVs and these enterprise customers, are they seeing the same kind of preview builds that are available for Insiders? Like, how-how do those two things compare?

RAMA SHASTRI: I think there's—the Insider program right now is very focused on feature updates. The product that we announced a couple weeks ago at Ignite, we're starting off with monthly security updates. Monthly security updates, actually we do not share prerelease updates very broadly, like in the Insider program.

We do have—we do have the ability to share these security updates, but we want to keep them protected, because they-they have changes related to a recent security vulnerability, and we don't want to widely distribute it, so that somebody could just pick it up and reverse engineer it, and weaponize it.

JASON HOWARD: Yeah, you don't want to zero day yourself.

RAMA SHASTRI: Exactly. (Laughter.) So, how do we basically scale this while keeping our intellectual property secure, is what Update Staging Lab will enable ISVs to do and customers to do, is come into this secure environment and still be able to test their application workloads against our prerelease security updates, and then be able to hopefully scale this program—scale the whole validation process.

JASON HOWARD: So, it sounds like you’ve already got something of a roadmap you're kind of looking forward to, right?

RAJI RAJAGOPALAN: Sure, sure. Let me add a little bit to what Rama said on the previous question, as well, because there is—it's a very important question.

RAJI RAJAGOPALAN: So, Insider community is-is helping us provide feedback on our-our preview builds, right, by using this mechanism called flighting. They're flighting the builds, and they're actually using the operating system that is going to come out very soon, and they're giving us feedback and, you know, monitoring data, monitoring feedback that's coming from the community, right?

And then, there is the aspect of testing which, you know, typically what happens is that when we—when we are rolling out our-our monthly updates, our feature updates, we actually have Windows App Compat team that's doing testing, Window's Servicing team that's doing testing, right?

But we'll—we also find out that our enterprise customers, in addition to having their own ring-based validation strategy where, in their—in their own enterprises, they have various rings where they first flight to this set of customers, and then flight to this set of customers, and they're constantly, just like we do in our consumer population with our Insider community, as well, they are monitoring what's happening at each successive ring before broadly rolling out to their enterprises, right?

But that doesn't mean that they are completely getting rid of testing. Like I said, there are a very small percentage of applications that are business critical for them that they continue to test, and it's absolutely important for them to have that peace of mind that these business-critical apps are going to work, right?

So, we—I mentioned before, and this is what Insider community is all about—we're constantly listening to customer feedback. And over the last few years, we have been looking at what the enterprise IT professionals are doing, learning from what they're doing, listening to them. And our guidance, as wel, and our technologies and tools that we are building is constantly iterating on what we are learning from how we are listening from our—how we are listening to our enterprise customers, right?

We are also listening to customers like Gartner, right, where Gartner is giving our enterprise customers advice on what is the best practice here. And Gartner is also advising our customers to make sure that they're testing a very small number of applications.

So while flighting and monitoring, and using data to understand how your-your rings are doing is a great strategy—we should definitely encourage and make sure that our customers are doing that—our enterprise customers, there's also going to be a small number of testing that happens, and we need to absolutely, as Microsoft, we need to provide tools and technology to empower them to do that testing faster and better. And that's where Update Staging Lab comes in conjunction as a complement to programs like the Insider program and the Windows Insider for business.

JASON HOWARD: Yeah, the more support that we can offer them, the more prepared they'll be for this rollout, the next rollout, and subsequently thereafter.



JASON HOWARD: So, something interesting that you mentioned was about giving these ISVs the ability to preview some of the security updates. And it's kind of a tricky thing with the Insider program, where we keep Insiders secure. We definitely release the security updates to them, but we don't deliver them early to where they get them before the rest of the world does.

RAMA SHASTRI: That's right.

JASON HOWARD: But it sounds like there's a little bit of unique footprint here, where potentially that's happening with this particular set of-of ISVs.

RAMA SHASTRI: That's right. I mean, this is a unique opportunity where Microsoft is making prerelease security updates, Windows monthly security updates available to ISVs as early as 30 days before we release it into the—into the wild.

JASON HOWARD: That's a significant change.

RAMA SHASTRI: That is a significant change, but think about what are the ramifications to our enterprise customers? Our enterprise customers actually are trying to roll out these security updates as quickly as possible. Some of them, at least the-the aggressive ones, have policies in place that require them to roll these updates out to their entire population within seven days to 30 days, right?

But a lot of them are also struggling to meet these deadlines and policies, right? And how can we make their lives easier? How can we make the CISO in these organizations happier?

And that's where a service like USL can help. A lot of times, our security updates can break their business critical apps, and if they were able to find these issues up to 30 days before we release them, they actually now have the runway to go in and remediate these issues, and be able to deploy these security updates with much more confidence, and be able to still meet their policy deadlines. And that's where I think the-the beauty and the benefit of this service comes into play for enterprises.

JASON HOWARD: And then, of course, there's no delay of what happens with the broader retail and Insider populations.


JASON HOWARD: They still get the full security updates in patches directly on the continued timescale of which we've been providing them, you know, in a sense, historically, without delay.


RAJI RAJAGOPALAN: The key thing that I want to stress in what Rama just said is that, you know, as a—as an engineering leader for an organization whose mission it is to keep our customers protected, I worry when I hear from customer that they're taking as long as three months to four months to roll out security updates broadly.

JASON HOWARD: Yeah, that's significant.

RAJI RAJAGOPALAN: Significant. So, the number of—the amount of time that malicious actors out there in the wild have to compromise these machines that are left unprotected, it just—you know, that worries me a lot, right? So, as an engineering leader in an organization that wants to keep our world protected, I want—I think it's my responsibility to provide these tools and technologies to make sure that, you know, testing as quickly as possible so the-the lead time for rollouts is reduced, I think that's-that's critical for us.

JASON HOWARD: Absolutely. So obviously, with this-this new environment that's been created, this is an—it's an important kind of a two-way street, right? So, we're creating this environment where ISVs and enterprises can participate, right. We're enabling them. There's going to be some awesome things to come forth from it.

So, during this process, what kind of engagement and what kind of information will Microsoft provide in return to help them resolve any issues that they encounter as they go through this effort?

RAMA SHASTRI: Yeah, ISVs have a phenomenal opportunity to come in and-and share their applications and onboard to this new service. As part of that testing process, they get access to preview month—Windows monthly security builds, against which they can go in and test their applications in an environment which Microsoft maintains, deploys, configures, and maintains for-for the ISV. So that takes the burden of maintaining a test environment out of the ISVs’ worry list.

And then in addition, their applications are tested in a very predictable manner, because every time we publish a package, we automatically spin up these Azure VMs, on which their ISV apps are installed, and the add—the tests are running automatically. And then, we publish these test results on a-a dashboard, which we host in Azure portal, right.

So, the ISVs can go in and check their test results. If there is a break or if there is a failure, then we actually do the bulk of the work to investigate the failure, root cause it, and if it turns out to be an OS issue, we'll take it, and we will actually go off and try to fix it. And then, if it's an app issue, we're working closely with the ISVs. We're-we’re hoping to work with the Desktop App Assure program, and then bring them in so that they can help the ISVs actually go in and remediate the app.

All of this in order to limit the impact to end users, which are our mutual customers.

JASON HOWARD: So I've got to ask, right—again for the listeners out there, you don't get to see the look that's on my face, but it's a bit of an exasperated look, because the question that's sitting here on the tip of my tongue is, why wouldn't every single ISV out there want to jump on board and participate in this effort?

RAMA SHASTRI: That's exactly right. Like, for the ISVs that we've talked to, everybody's been super excited about this service, and they're like, "How can I get in?" And that's one of the-the challenges of—we are in a development stage, where we can't just scale and bring everybody in, but we want to have some meaningful engagements with ISVs who would come work with us, take our service for a spin, kick the tires, and give us some good feedback, so that we can build a-an awesome service that everybody will benefit from.

JASON HOWARD: And then kind of scale it up longer term once you get kind of what you believe to be the right and solid foundation that everybody can build upon.

RAMA SHASTRI: That's right. And I want—I want to touch on, you know, the benefit. Obviously, there's awesome benefit for the ISVs, but then there are also aspects that benefit enterprises, right. Enterprises care about the apps that these ISVs publish, and then those apps are causing deployment blockers in the enterprise environment. And by giving—by testing these apps against preview builds, we're able to generate a confidence signal much faster, which ultimately benefits the enterprise ecosystem in adopting these Windows builds faster and rolling these security updates, so that their end users are protected and productive.

JASON HOWARD: So, one of the interesting things that, for the Insiders out there who either don't know about what we call WIP for biz, or haven't really participated in that effort, like, I want to share a little bit of insight briefly because, as a-an Insider who's just installing the standard what we call, like, retail preview builds, right—you go through the normal development cycle, you get all the preview—you get to test out the features and the apps, and you know, the driver updates, and things that come forth. One of the things that makes what we call WIP for business, which it's-it's the same builds, really, but what makes it different is, for businesses, there tends to be a longer runway for them to do all this early compatibility testing, get prepped, because like for me, as one user, I'm—you know, I wait for the retail build to come out. It's general availability. I install it on my device at home, and I just keep using it, right?

But when you are in control of an enterprise where you're running 100 machines, 1,000 machines, 100,000 machines, for some companies, and there's some that are even bigger than that, you have to make sure that you don't roll out broadly and potentially crater a ton of machines all at the same time, right? You-you impact 100,000 machines and you can kind of imagine what the business impact would be.

So rather than them starting to do their compatibility testing and everything on day one once general availability hits, by these businesses participating in this WIP for business effort, much like you do as an Insider, it gives them this extended runway ahead of time, so they can see the builds, start using it on their internal rings that they have the ability to create and deploy on whatever number of machines they choose, and they can actually create their own internal rings—Fast, Slow, Release Preview—the same type of cadence where they roll it out progressively, and it gives them that runway.

It's really about them having that additional time to go through and find out what impact is going to be had for their end users. So, the Insider program is just as important for them, and we gave it kind of a slightly different name because there's—because of the way they can get builds, right? They're not just pulling them down off of Windows Update. There's all the different ways. So there is WUFB, and WSUS, and Intune, and so on and so forth, but we've made it available so they have this same type of ability to find, preview, and potentially influence the development of Windows along the way, just the same as you as individual users do.

So, I realize I kind of, you know, digressed a little bit there for a second, but I don't know if everybody knows exactly what WIP for business is. Hopefully everybody has a little bit better of an understanding.

RAJI RAJAGOPALAN: That’s perfect.

JASON HOWARD: But, having said all that, let me get back on track here. So, I'm going to ask you, what is definitely my favorite question now, that I love asking on-on all the podcasts. So, you don't have to get yourself in trouble, right. We're not trying to, you know, get anybody fired here.

But is there anything on a sneak preview on the roadmap, you know, anything that you're potentially working on that's like what's coming, what's next that you can talk about?

RAJI RAJAGOPALAN: Okay, Jason. You promise this is just going to be between you and me, right?

JASON HOWARD: And every Insider out there listening.

RAJI RAJAGOPALAN: (Laughter.) I can still share some of the roadmap that we have in front of us for Update Staging Lab. So, the—we have, like Rama mentioned, this is a startup. We started a few months ago, about a year ago or so, and we have been working on this MVP for the last few months, right? So, we are at a stage where this is still a little bit of an exclusive club. It's an invitation only program mainly because we're not ready for scale yet.

So, the next set of investments from my team is going to be to take us to the-the level of scale that we expect once this goes public, right? So, we want to have the right-right level of availability, scalability, self-serve—all of that, all of the good stuff that you expect of our world-class service. So, that's going to be a bunch of investments from my team.

But beyond that, we're going to have investments along two different dimensions. One is how do we add more content to the program, meaning web applications, client server applications, all the other different kinds of applications that our customers want to support in the lab, in the Update Staging Lab. We're going to have support for that. On top of that, Microsoft Office, Microsoft Teams. There is a rich application ecosystem on top of all of these different platforms. How do we provide the support and Update Staging Lab to support all those different scenarios? So, that is the content dimension.

The other dimension is the audience dimension where now, we are—we're going to ISVs. We also want to make sure that our partner ecosystem is involved in this program, as well. We want to make sure that enterprise IT professionals are involved in this program, as well. So how do we take on enterprise IT professionals, partners, and ISVs as our direct customers and Update Staging Lab, because they—all of these different customers have different needs.

The enterprise IT professionals have you know, line of business, business critical applications that they may want to test on this program, on this—in this lab. Our partners will have a lot of enterprises that they engage with that they want to bring onboard Update Staging Lab, and we want to continue to serve our ISV community, as well.

JASON HOWARD: I hope I'm not speaking out of turn, but it sounds like you're creating quite the runway here.

RAJI RAJAGOPALAN: Quite the runway, and on top of that, we're also looking at RPA, the-the stuff with the bot-based testing. So one of the things that we are—we're looking at is, you know, when you're listening to our customers as they're giving us feedback, our ISV customers, they're basically telling us that, yes, you know, testing our applications is great, but writing automation, it takes a lot of time for them to write those automations. If we can use things like robotic process automation to make this easier for them, you know, they-they would love that, right?

So, we're looking at robotic process automation and bot-based testing to-to see if we can accelerate the whole testing process even further in Update Staging Lab. Is there anything more you want to add, Rama?

RAMA SHASTRI: I think that, you know, Raji pretty much nailed it. I think the-the important thing is the dimension thing. I think there's so many things we can do. There's so much green field here. There's so much opportunity over here.

I think we—when we talk to customers, they worry about so many different things, like the content, obviously, that Microsoft's putting out, not only Windows updates, but updates from like, .NET, browser updates. There's updates from Office, Teams, Dynamics. So, if you just take Microsoft as a—as a machine that's cranking out updates, we're probably updating a lot.

JASON HOWARD: I'm sure. (Laughter.)

RAMA SHASTRI: On top of it, you layer on things that other companies are putting out updates, and these enterprises and ISVs are sitting there, you know, receiving, you know, Chrome updates, other application updates, security updates that other partners publish. And there's just so many updates that are coming down the pipe, right? How do we help these customers and partners stay ahead of that ecosystem, is what we're trying to help here.

And then, it's not just the ISVs, it's also the system integrator partners. These are primarily the-the consulting companies that help these large enterprise customers take on the burden of a lot of the IT management, and that includes testing, as well, right?


RAMA SHASTRI: And how can we engage with these large side partners? These are global companies. How do we help them become a little bit more efficient with their testing operations? And that's something I think that is well within the reach of Update Staging Lab.

JASON HOWARD: It's interesting, hearing it put into those words, because, with some of the analysis we do related to the Insider program, we start looking at the variety of hardware that's out there. You look at the variety of applications that are available to end users. Then, you start looking at all of the micro-updates that come out, all of the app versions, all of the driver versions, right?

You start looking at that at scale, and we've actually done some math internally, and this is where, it's-it's going to sound implausible, but I promise you, we had some really smart people do the math. There are more combinations of hardware, software, apps, drivers, and firmware, because you can add that in there, as well, then there are grains of sand on the Earth.


JASON HOWARD: It's insane. Mathematically, it explodes so quickly that, it-it just becomes almost impossible to comprehend. And so, when you talk about the—it's almost like a minefield that these partners have to dance through, trying to navigate and not hit any pitfalls along the way, and not create any poor customers experiences, because I will say, you know, IT is—IT can be a whole lot of fun, but at the same point and time, in some organizations, it can be a bit of a rough experience because people, they just want things to work.

And so, when things are going well, you kind of—you know, you don't hear much. People are just kind of happy doing the things they need to do, day-in and day-out, going through their business activities. But as soon as something breaks, then all of a sudden, it's like, “Those pesky IT folks,” right?

And so, this whole effort, everything that sounds like all your—everything that you're doing right now is going to have such a broad reaching impact that, from the-the new IT person who's yet to hit that first, you know, that first broken machine, all the way to the seasoned pros, who are kind of designing and laying out the roadmap for their ISV, or their, you know, their enterprise ecosystem, wherever they may be sitting, you have a giant footprint that you're going to impact with this.

And I've just got to say, like, that's fantastic. Like, I am super excited that this came to fruition. It's like you-you’ve come up with this concept, and I asked earlier, kind of like what spurred this. Obviously, there was a whole lot of things that kind of came together to make this happen, and now you're marching forward with this, and like, there's nothing but upside here.

RAJI RAJAGOPALAN: Absolutely. This is why I started with, I'm so excited to be running this team, not just in-in Redmond, but also in Africa, because the-the product is coming together, the contributions from both Redmond and in Africa, as well. So very excited that so much customer value is coming onboard, but it's also a lot of contributions from a lot of very awesome people.

JASON HOWARD: So, I would say how can ISVs onboard, but from earlier in the conversation, you mentioned, it's-it's a bit of a closed program currently, so let me ask you a different question instead. Where should partners that are interested tune into to keep up with news and updates, and progress along the way?

RAMA SHASTRI: So, aka.ms/JoinUSL. We are inviting nominations from either enterprise customers who want to nominate ISV partners or ISV partners can come and self-nominate themselves.


RAMA SHASTRI: What we want to do is work with the most active and engaged partners to help make our service better. So, that's how we want—that's how we are basically intaking partners into our private preview at this time.


RAJI RAJAGOPALAN: It's our own USL Insider community. (Laughter.)

JASON HOWARD: That's the thing. Everybody at Microsoft wants to have an Insider community. Windows Insider—

RAJI RAJAGOPALAN: Well, it's been so successful. Your Insider community has been so successful, Jason. (Laughter.)

JASON HOWARD: You-you get something going, and people see it, and they're like, “Maybe we could kind of do the same thing here.”

RAJI RAJAGOPALAN: Absolutely, yeah.

RAMA SHASTRI: That’s right. You've got a winning formula.


JASON HOWARD: You know, there's a playbook in place, so you know, it's nice to see other-other teams taking advantage of some of the-the groundwork that was laid. And obviously, you know, the Insiders, especially-especially those of you who are tuning in, because obviously you're active and want to stay up to date with what we're working on, like it's good to have them along for the ride.

And I've got say, it's been fantastic having you here, both Raji, Rama. I-I've loved this conversation. I've really got to say thank you both for joining us, kind of walking through all this new—through all this new information, right, because I kind of knew about this. I heard some of it at Ignite. I've definitely learned a few additional things here, especially related to the background for this.

As I mentioned earlier, it's super exciting for us to see what we're doing with preview builds, WIP for biz, this additional effort with USL, especially as it's bringing along the enterprises, and the winners get to—the Windows Insiders get to kind of hang along for the ride and see what we're doing on this side, which is going to affect them as well, because they also take in some of these updates from these third-party ISVs along the way.

And in the end, really, it's just making our services and the entire environment, the whole Windows ecosystem that much more reliable.

RAMA SHASTRI: Absolutely. We love Insiders, by the way. Did we say that? (Laughter.)

RAJI RAJAGOPALAN: Thank you, Jason.

JASON HOWARD: Absolutely. Again, thank you so much for being here. Thanks for making the time to be in the studio, and no doubt we'll be—we'll be chatting again soon.


RAMA SHASTRI: It was a pleasure to be here, thank you.


JASON HOWARD: Next up, we have Michael Kroll joining us to talk about Bing, the Bing Insider Program, and much, much more. Welcome to the podcast, Michael.

MICHAEL KROLL: Thank you very much.

JASON HOWARD: So, real quick for our listeners, can you please introduce yourself and tell us a little bit about what you do here at Microsoft?

MICHAEL KROLL: Sure. I'm Michael Kroll. I lead the Bing Editorial and Homepage teams, so there's a team of editors that we have in mostly the U.S., but we have some international folks, as well. And we do everything from pick the picture that goes on the Bing homepage every day. Every week, I get to go to this meeting where we look at—talk about pictures. It's pretty-pretty—it’s a highlight for me. It's tomorrow. I'm excited. (Laughter.)

And we also—but there's lots of other things that we do, as well. So, there's the writing of all the strings that are the words to find a—in the—in the answers on Bing, and we write those quizzes that people sometimes take.

JASON HOWARD: Nice. So, for the Insiders out there who are tuning in, right, obviously, we want all of them to be flighting, taking a look at the blog posts that we publish for the Windows releases each week. And of course, for the Insiders that are, you'll see that, pretty much every week, we have a little bit of something in there from the Bing Insider Program. So, can you tell me a little more about what the Bing Insider Program is, and if they're not, why a Windows Insider should become a Bing Insider?

MICHAEL KROLL: Sure. Well, it’s a lot like the Windows Insider Program in the sense that what you get to do is you get to see sort of a little bit ahead of the curve for what-what Bing is up to. And so, we-we will often go on tours where we'll go out and meet folks, one-to-one. And it's actually an opportunity for people to see the people behind the—behind the AI, right, the people who are really working on it, and what I think is kind of exciting is really give feedback to those people that will have a real impact on how the-the search engine works.

We really do listen a lot to the—to the feedback. I mean, I've been on Bing pretty much since it began, and you know, it's an underdog story, sort of like, you know, Dodgeball. We definitely had a long (laughter), long way to go. And one of the ways we got there was really by listening to folks, figuring out what was working for them and what wasn't working for them. And the Insider program kind of gets you on the fast track of-of making your insights known to the people who can change things.

JASON HOWARD: So, as part of those insights that we give out, you know, in our weekly blogpost, one of the highlights form Bing is talking about how you can learn more about specific topics and be prepared for upcoming events, be like holidays or, you know, fantasy football's a big thing here in the U.S., and pretty much anything else you can imagine, right?

So, if you actually also run Bing as your homepage, and regular users of Bing will see that, you know, there's the news highlights. There's the image of the day that you mentioned, because I mean, that just sounds super fascinating by itself. I would love to—maybe you can let me pick one, one day. I can—I can ask. We'll see. (Laughter.)

And then, of course, you know, when you see those images of the day, there's little bubbles you can click to kind of take you, teach you more about the image, the setting, the people—whatever the image happens to be of. And then, of course, one of my favorite things is, like, this day in history, because, I mean, I tend to be a sponge for random knowledge, and talk about a fast way to-to brush up on, like, my Trivial Pursuit stuff.

I've got to ask, given the variety of content here, how does your team come up with I guess the right way to say it, it's like a programming strategy, maybe? How do you come up with a strategy behind laying this out, and doing it far enough in advance so you know what you're doing, you're not scrambling, last-minute, because there's a lot of helpful content here that you're—that you're curating and putting forth for users?

MICHAEL KROLL: Yeah, we-we definitely—we do think about it pretty far in advance, so right now we're working on—so this is about two and a half months into the future, so we're looking at that.


MICHAEL KROLL: But what we-we started out, you know, it was really about having a beautiful image on the page, and then, you know, coming up with a few fun facts about that image. Which is great, and that worked for about eight years, I would say.

But at some point a couple years ago, I—the team and, we-we decided that we wanted to have more of a-a real strategy that pulled everything together, because it could get a little random, also, sometimes. So, we had this idea of, like, what if the day—what if the picture of the day could—or the homepage itself could make you feel smarter about the day, right?

And so, the trending news stories at the bottom is one obvious way in which you can stay-stay up to date with the day. The "On This Day in History" is another one. But then, we started looking at what the picture could actually be. And so, often, you'll find that the picture actually relates to the day in some very specific way. So, you know, Christmas is one that, you know, obviously, we show that on the—on the homepage, but there are other ones that are a little bit more obscure.

World Elephant Day, not everybody celebrates that. I get that, but you know, it's good to know, but it gives us an opportunity to show different kinds of things. And sometimes, it'll just be about the season or things that are happening at this time, so this is the time when the orcas move from here to there, and we can show those kinds of things.

But what we thought was, hey, this-this allows us to tell a story that has-has a point, that has something kind of interesting, and the connect—and can connect people to the day a little bit more directly. And then, once we have that, then we can sort of build that content around that-that topic, and sometimes we can—a lot of people don't know you can scroll below the fold on the Bing homepage. It's kind of a hidden feature, so I'm telling you guys now. (Laughter.) Please—

JASON HOWARD: (Laughter.) Scroll down.

MICHAEL KROLL: Please scroll down, because there's a bunch of other content that we can point to, as well, and sometimes those will go into different kinds of answers within Bing that'll talk about different-different things.

So, for the holidays, for instance, we'll have, you know, recipe carousels that you can point to, and they can type in "holiday recipes," and you'll get some good ideas of varying levels of skills, simple stuff first.

JASON HOWARD: (Laughter.) I was going to say, do you have one for those of us who just burn stuff?

MICHAEL KROLL: Yeah, absolutely, whatever you need. Making drinks is—that's a recipe also.

JASON HOWARD: I haven't burned a drink yet. (Laughter.)

MICHAEL KROLL: Exactly. A punch drink, that's-that's a recipe, you know, but you can probably make that. And if you make it wrong, it's okay, probably, also.

But there's lots of—so there—our strategy, in that case, so we think—we go a couple months ahead. We look at a calendar of events. We look at the month, and we say, "Well, we could—on this day, this was the day that, you know, Tasmania, you know, became part of Australia." I don’t know if we're actually covering that this year, but—and then we can focus on a picture of Tasmania. We can tell that story, and then it allows us to write it.

But we have about 10 people who write homepages every-every couple weeks, and I-I actually also do that, so just to prove that I have some real value to the team beyond as sort of a figure head.

JASON HOWARD: (Laughter.) I can say since I asked for maybe permission to potentially pick an image of the day, for-for all the Insiders out there that-that know me, especially from my online social persona, they know I'm a big fan of tacos.

So, I'm already, like, dreaming up—you're talking about recipes, and I'm talking about pictures. I've got, like, taco recipes, like, floating through my head, and like, I've got this—you know, just this image of a beautiful, delicious plate of tacos. Like, I don't know, I'm getting off track here. (Laughter.) It's like now you've got me thinking about food.

MICHAEL KROLL: I'm a—I'm also a taco fan, so you know, you have a receptive audience here.

JASON HOWARD: Maybe since we share that mutual interest, the-the potential for me getting to pick an image just went up a little bit. I don’t know, I'm just trying to work that angle.

MICHAEL KROLL: Also, we-we've had the Bing Insider—the-the Rewards program has a program where you have this ultimate day at Microsoft, and one of the things you get to do is you come and you help pick pictures for the day. So you know, if you—

JASON HOWARD: Oh, oh, I know exactly where all of my Bing reward points are going now. Oh my goodness, yes. (Laughter.) See, that's another reason I love doing these podcasts, is I always learn something. (Laughter.) So, let me get back on track here because I'm still thinking about tacos. (Laughter.)

With Bing being such an important program for Microsoft, right, both as a product and as a concept of a way that we get out and kind of engage with our user base, a piece of that, kind of what you were just mentioning, is Microsoft Rewards, and you'll see that on your Bing homepage when you're logged in with your Microsoft account.

So, for those who may be unfamiliar with this particular effort, can you share with us a little bit about what Microsoft Rewards is, how Bing connects into that effort, and why should, like, our listeners and Windows Insiders, and kind of everybody in general should participate in that?

MICHAEL KROLL: Absolutely. So, Microsoft Rewards actually started back in the day as Bing Rewards, and it became a Microsoft-wide program a couple years ago. And the idea of it is really to show loyalty back to our customers. The-the search is, again, the way that we make money from search is through ads, and through things like that, and through people using it. And so, we were really like, you know, we could give some of that value back to our customers in some way.

And so, it really is—it's like a classic rewards program that you'd have from an airline or some—any-any other retailer, where if you do stuff with us, you earn points. So, on Bing, that means searching. If you search a certain amount every day, you get a maximum number of points. You could also do it on your mobile phone, so you can earn points in two ways.

And those points accrue, and you can get gift cards, and you can get sweepstakes entries, if you want to join the—see if you can get that ultimate Microsoft day. You can also buy Microsoft products—you know, get money off on a Microsoft product, as well, or even donate some of those points to causes that-that you support. So all of those ways to kind of turn that-that value back around to the users.

And now, it's—you can do it on Xbox, as well. If you go to the Microsoft Store, you can earn points. All these places, you can earn points, and all that accrues under the—your account there, Microsoft account. And we-we try to make it easy for you to earn points. There's, you know, quizzes. There's sort of treasure hunts you can do, things that'll-that'll help you get through those points—max out your points as much as you can every day so you can earn it as fast as you can. I personally have like 100,000 Microsoft rewards points—

JASON HOWARD: Oh my goodness.

MICHAEL KROLL: Because I never—I always forget to use them, so you know, but I-I hear some new things are coming. I can't—I was not allowed to divulge them early, but some interesting things are coming that-that I'll—I should empty out my-my cash pretty quick, I think, once that starts happening.

JASON HOWARD: See, I have to admit, I just used some of mine because, well, I mean, if I'm going to say it, like, I happened to redeem some of mine for the Amazon gift cards, right? So, I just used a big portion of what I had saved up, but it sounds like maybe I should've held on to them.

MICHAEL KROLL: You can always search more.

JASON HOWARD: That is true.

MICHAEL KROLL: You can always search more. There's lots to learn. You like to learn random facts. You know, again, we are really—we know a lot of random facts.

JASON HOWARD: (Laughter.) As a search engine, I would hope so.

MICHAEL KROLL: That's right. It's really our specialty, in some ways.

JASON HOWARD: So, it's funny that you mention the whole quiz thing, because today, while I was eating lunch, I actually went through and did one of the quizzes. It turns out it was about the Velvet Revolution, so super interesting topic, something I won't claim to have known very much about. So, I learned a few things by getting a few of the questions wrong, and then seeing the correct answers.

It’s-it’s interesting to have that piece of engagement, because, like, I would never have thought I go to a search engine and, aside from, you know, I'm going to look something up—oh, tell me what the weather is. Tell me what time it is in this time zone, as I'm trying to do prep for a webcast, or something like that. But I go there to the homepage, because Bing happens to be my homepage, and without even really struggling to, I just, like, randomly start learning things. It's a really cool and engaging experience.

MICHAEL KROLL: I'm glad you think so. We think so. Again, if you come to the meeting, again, the people—the random things I learn in that meeting every day, every week, is pretty-pretty impressive. But it is—it is one of those things where we know people usually come to a search engine with something—a task in mind, so we never want to get in the way of that. But if you do have a minute, or 45 seconds, you know (laughter), that you want to waste, we can—we can help you do that in a way that doesn't feel totally wasteful so that you do actually, you know, pick up something that's interesting that you might share with somebody later down the line, or find edifying in some way.

JASON HOWARD: Yeah. So, there's-there's an interesting connection to be made here, because not only is Bing kind of like this-this search engine that you would think about, right? But there's some handy integrations into other Microsoft products. So, if you look at it from a multimedia standpoint, there's Windows, there's the Office suite. Can you tell me a little bit about how these things kind of coordinate and play together, and how Insiders—especially Windows Insiders—can potentially make the most of it?

MICHAEL KROLL: Absolutely. One of the things that-that Bing got great at first, I think one of the places where we first got known is-is in our multimedia search, looking for images, looking for videos. We have a really cool experience that a lot of people really call out as what they like best about-about Bing. And so, it was natural to think how can we integrate those things into other Microsoft products more directly? So that people have—like, that you don't even have to go to Bing.

One of the—one of the ways that Google thinks about search is, they think everything is search, right? So, your email, they're happy to search your email, things like that to help you—to help you later, you know, find stuff. But you know, I think at Bing, we think that search can go anywhere. It can belong in all kinds of different places. So, for us, it's less about you have to come to Bing every time, but can we bring Bing to you, where you are?

So if you're looking for images, you know, you can probably—you've been able to do this in Office for a long time, but the image search in, like, PowerPoint is all powered by Bing, or the GIF search is all powered by Bing, and video.

Some other stuff that we're starting to do, though, is there's a thing called visual search where sometimes it's hard to describe the thing you're looking for, but you might have a picture of it, right? And so, can you search by picture is one of those things. So, there—you can do this currently on Bing, and one of the things that's getting integrated now is even in your Windows photos app, right, if you right click, you'll be—pretty soon, you'll be able to say, "Look for images like this." So imagine you're—you know, you took a picture of Notre Dame Cathedral, and you're looking for other pictures of Notre Dame. You'll be able to find that from the web in that way.

Similarly, you'll even be able to do a—coming soon is the snipping tool. You can take a snip of one of the—of an image and then search by that, as well. But again, I think that's really super interesting because, you know, a picture's worth a thousand words, so it's a very long query, you know, to type a thousand words. (Laughter.) It's sometimes just-just to have the picture.

JASON HOWARD: Yeah. It's interesting that you brought that up, the searching by image thing, because I will say I don't know how long it's been in there, maybe to my own detriment, but again, and I'm—there's no joking in me saying this; it was something that I noticed for the first time today while I was at lunch, right?

I was sitting there, and I was on this homepage, and I'm going through, like, the little news carousel and kind of clicking through some of the different articles, and just kind of catching up, and whatnot. And I click to go back to the homepage after I had gone through the little carousel wheel, and I looked at the search bar. Normally, I just take the search bar for granted, right? I just click in there and just type whatever it is, and hit enter, and you know, it brings up the search results.

But for some reason I specifically looked at it because something caught my attention, and at the right side of the search box, there's usually a little magnifying glass, which is, "Click me, and I'll start my search," but there was another little icon directly next to the magnifying glass. And so, I hovered over it, and it's this image search thing that you were talking about.

And for like me, it was like, oh my goodness. Like, when did this show up? How long have I been missing this? Is this something brand new, and it's fun to have this conversation because, for me, you know, just one person, like today was actually the day that I found out that it was a thing.

MICHAEL KROLL: Yeah, we just added it in the last, I would say, five months into this homepage, so it's been there a little while.

JASON HOWARD: So, I'm five months behind schedule here. (Laughter.)

MICHAEL KROLL: But this is the challenge of, you know, habitual behavior. Like, we really—like, it's hard to—hard to break the habit. But it is—you know, now it's getting more integrated into more things.

And again, it gets better the more people use it, so again, I would encourage people to use it. Try it out, give us feedback and see how it's going. But it is pretty cool because you can do shopping by that way. You can do—let's say you're looking for a particular kind of shoe, or you know, you can take a picture of it with your phone and then search by that picture. So, there's lots of cool stuff that you can do with that feature that-that makes—you know, can help you tasks—get stuff done, not just waste time but actually get-get some useful stuff done, as well.

I will say I've often thought it's really weird that we use the magnifying glass as the search thing. Like, when was the last time people looked for stuff with a magnifying glass, really? Like, it's really (laughter) kind of Sherlock Holmes. Like, oh, there it is. But it's, you know, now—that's what it means, so we-we use it.

JASON HOWARD: Speaking of funny icons, right, and history of icons, there's something funny I'm going to tangent here for a second on. I'm on Twitter a ton for my job, engaging with Insiders, but I get to see a ton of just the randomness of the internet at that point. And there was a guy the other day that had posted up something very—something I found to be absolutely hilarious.

And his daughter, who’s in her, like, seven, eight, nine—you know, that really young category—she saw an old-school floppy disk on his desk. And she asked him why he printed the "save" icon. I'm like, talk about a way to make me feel old. (Laughter.)

I know, I'm completely tangenting here. It's just kind of funny. It's like, I see that kind of stuff, and I chuckle. I'm like, how much has technology changed? (Laughter.)

MICHAEL KROLL: But I mean, how—I mean, that thing was like—it was the state of the art for not long, like five years, maybe, 10 years, tops, probably less. And it—now it means "save." Like it-it-it is no longer a functional thing, but it has become—like the-the-the you know, the symbol has now transcended the thing itself. So, it's pretty—it's, yeah.

JASON HOWARD: I don't even know what we would try to replace it with. If somebody got the bright idea—well, maybe not-so-bright idea to try to replace it, what would you put in its place?

MICHAEL KROLL: I don't know.

JASON HOWARD: I don't know. I don't even want to try to guess. (Laughter.) Alright, so please let me get back on topic here. I know I've digressed a little bit. It was just kind of a funny anecdote I wanted to share.

So, a lot of our Windows Insiders are engaging in the WIP for business effort, right? They're IT pros. They're working within their organizations, kind of trying to pull their businesses along, making sure they're prepped for updates and things of that nature. And Ignite was held last month, and of course that is the big IT pro conference that happens every year. And there's a lot of announcements that are very relevant to that particular set of the population of the Insiders.

So, Bing also made some important announcements at Ignite. Can you tell us a little bit about what was announced and what the-the Bing Insiders should be looking forward to?

MICHAEL KROLL: Yeah, absolutely. So, we get to use a lot of these features early, working at Microsoft, but it is a really interesting, and amazing, and obvious thing for us to go into, which is search, which is the web search, is, like, that's one giant repository for information, and it's random and filled with who knows what, variable usefulness. (Laughter.)

But on your corporate, you know, network is a whole other universe of information, that's actually pretty relevant, much of it, at least, to you, if you can find it. But it's poorly—often poorly organized, and you know, how you find people and where people are. We—again, at Microsoft, we work on a giant campus with lots of buildings, and stuff like that.

But even if you're in a—at a company with, you know, 100 people or more, finding people is kind of a challenge. And remembering who that was that you shared that document with, or what was that document I was talking about—they were talking about, or what was that acronym somebody was using? What does that really mean? I don't want to seem like I don’t know.

JASON HOWARD: (Laughter.) Not that that's ever happened.

MICHAEL KROLL: No, no, that never happens to me. So, what-what Microsoft Search is, and so, Microsoft Search is this program that happens across all of the sort of Microsoft stack, whether that's Office and SharePoint, and things like that, and Bing, and you can access it through Bing, if you're on your corporate network and you—your company is signed up with it. So, Microsoft Search in Bing is a way to combine your sort of web search and your intranet search into one thing.

I don't think I'm, you know, telling tales out of school to say that, you know, the classic SharePoint search is not always—it's not, it doesn't work in the same way as a web search does. It's just a different animal.

JASON HOWARD: It's designed differently.

MICHAEL KROLL: It's designed differently. And so, what's cool about the Microsoft Search in Bing is it uses sort of the very—the state-of-the-art stuff that Bing is using for commercial or consumer search, and it applies it into this new space of the—of the enterprise. And so, it unlocks all kinds of really interesting things that you can do.

That could be, you know, I said finding a person. So, it used to be like, you know, I've got to remember somebody's name or their alias, and things like that, but now I could be like, "Oh, I'm looking for Jason who works at Studios," and you could put that in, and it's going to be able to use what they—you know, they talk about it as a semantic search engine, and these other things to say, "I know enough about you and about what that means that I can narrow it down to-to you." So, that would be an amazing, new breakthrough for people who are trying to find folks.

Another one is, again, the acronym searches. Those are really interesting. That-that I think is really, really powerful. (Laughter.) Especially if you work at a place like Microsoft where the same acronym could have like three different meanings, at Microsoft.

JASON HOWARD: Oh my goodness, yes.

MICHAEL KROLL: And so, that-that can be a real challenge, so understanding your context.

Another thing that's coming is this idea of semantic search. Like, the example they talk about is like, can I bring my pet iguana to, you know, to work. You know, like is that allowed? And the Microsoft Search in Bing will be able to look through all of the corporate guidelines and see what the pet policy is and understand that your iguana is your pet, and it could say, "No, you can't bring your iguana to work at Microsoft." (Laughter.)

Even if—maybe if it's your, you know, your service iguana, potentially. That would be some—that's a different rule. But as a, you know, the—as a rule, pets are not allowed to come-come to work with you.

JASON HOWARD: I'm going to have an emotional support honey badger.

MICHAEL KROLL: I think that makes sense. (Laughter.)

JASON HOWARD: Let it go and then create havoc in meetings, instead of me having to do it myself.

MICHAEL KROLL: Yeah-yeah, the other thing that's happening is that there's going to be a better browser integration, so like Edge and Microsoft Search in Bing will work together, so you can search directly from the search box in Edge. You don't even have to go to Bing, and you'll get, you know, get the relevant work results, as well.

And the-the other things that—the biggest thing that's coming that's going to unlock a lot of doors is connectors. So right now, we've really focused on making the Microsoft stack work best. But people have data in other kinds of places, as well, box, and we work with a bunch of different companies to build these connectors. So more and more of the data will be unlocked and searchable in the future, which is really, really exciting as well.

JASON HOWARD: I mean, one of the important things with technology on the whole, right, is trying to be where people are or where people want to be. And if you don't have the ability to kind of connect them with what they're trying to do or obtain, right, if you don't create that entire footprint to bring it all together in, you know, like one seamless fashion, right, it makes it harder for them to want to adopt your technology or adopt your service, things like that. And it sounds like how they're doing a lot of work to try to do the broader integration to make sure that whatever it is people are looking for, they end up with it, whatever it may be.

MICHAEL KROLL: Yeah, and I think, you know, Microsoft has a long history of being a partner in that way and understanding that we don’t have to own everything to-to, you know, to be—to make it work for us. Like, we can have partners. We can work with other companies and figure out what really works.

One of the things that makes our-our—I think people and organizations trust Microsoft Search more is, we really respect that it's your data. We understand privacy. We don't—it's not about us trying to own all of the data, and so that we can have some special insight. It's really about making it work better for your company, so that you can—you can achieve the things that you want to do, and we don't need to be, you know, looking over your shoulder at everything that's happening.

JASON HOWARD: Sure, and I mean, what you just said, it lines directly to what Microsoft's mission statement is. And I mean, it's the principle that guides all of us in the work that we're doing, and it's-it's awesome to hear that, you know, the-the great foundation that you're laying right now with that work that you're doing connects right into that bigger picture that we're trying to go forth and tackle.

So, I've got to ask you, right? I know we're kind of winding down here, but this is always my favorite question, right? So, I'm not trying to get you in trouble. Okay, maybe just a little bit, but what's next for Bing, right? What's the next big, cool thing that's not—maybe not quite out yet that you won't get yourself in too much trouble for talking about, maybe?

MICHAEL KROLL: I don't know if you've heard that there-there is an election coming up. Have you heard anything about that?

JASON HOWARD: Uh-oh. You know, I've heard a thing or two, yeah.

MICHAEL KROLL: (Laughter.) I have, too. One of the things that we've looked at over the last year and a half, especially, again, thinking about trustworthiness and how can we be better at satisfying people's needs, we were looking at, you know, the world of the internet and it's—you know, there's not again, you may have noticed there's not full agreement on all topics on the internet. (Laughter.) There's some actual dispute on certain concepts.

And so, one of the things that's tough about a search engine is, it tends to be presented as a big, long list of results, and like, that first one, that carries a lot of weight, and it's like that's the right answer. And sometimes, it's not as simple as there's one answer to a question.

So, we launched a couple, in the last year, this idea of the multi-perspective answer, right, which is, for some questions, there's not just one answer. Sometimes, there's not even just two, and that's not just as easy as yes or no, but there's a-a range of opinions around a topic. And so, we've really looked at how can we start presenting that for people, so that they can find not just the stuff that agrees with them, but they can also be—look at outside their bubble, see what's being said by people who don't share their opinion.

And there's even some—you know, again, you'll notice sometimes with a search engine that if you ask the question, say, "Is coffee good for you?" You'll get a different answer than if you'd asked, "Is coffee bad for you?" And that's part of the way that the search engine works, and it tries to recognize words that are shared in the—in the thing that—the results. So, if it says, you ask if it's bad, its likely going to say, "Yeah, it's kind of bad for you." If you ask if it's good, it's going to say, "Yeah, it's kind of good for you." And so, that's-that's a strange, strange function. So, we are trying to look at how do we bring both sides of that answer together when it—no matter how you ask it.

And so, with the election, I think, you know, one of the things that, there's so many opinions about so many topics, whether they're issues or candidates, and the like, that it's—we're going to have to figure out how do we-we have some ideas of how we're going to do this, but we're going to—we're going to try to present people with a better range of information, so it's not just, "Hey, there's one answer to the question you're asking," but, "Hey, there's a whole rich context for the thing that you're asking about that you can explore.”

And again, if you want that quick answer, it'll be there, too. But if you do want to dig deeper, that's a place where we think we can—we can go further than some other—some other search engines do.

JASON HOWARD: Yeah. So, it sounds like it's, you know, really taken that step to eventually break past—just call it what it is, like confirmation bias of people potentially having a set expectation, right, and showing them the broad range of here's kind of a bigger set of views on this particular thing, whether you may or may not agree with, kind of like you said, that first thing that would pop up, depending on how you ask the question.

There's-there's more to actually educating folks rather than necessarily just giving them an explicit answer that kind of agrees with something they may already have preconceived in their mind.

MICHAEL KROLL: Yeah, and I would say it even works on the other way, too, which is, if you don't show somebody the answer that they, or the opinion that they hold, they're going to call you into question, too.

JASON HOWARD: Yeah, absolutely.

MICHAEL KROLL: They're going to say, "I don't know if I can trust you if you don't have even—you don't even represent this opinion." And so, I think it's really important to-to give people that sense that we-we aren't trying to make—we aren't trying to define what reality is, where it's not clear. We aren't trying to, you know, push some political agenda. We're really trying to say, "Hey, there's a range of stuff here. There's a lot of points being made, and here's a way for you to explore it."

JASON HOWARD: It's like asking if there's—if pineapple belongs on pizza. Woo, if you want to start a fight, that's the way to do it. (Laughter.)

MICHAEL KROLL: Wow. In limited situations, I'd say there's a definitive answer. (Laughter.)

[TCR 00:35:50]

JASON HOWARD: (Laughter.) Oh sure, sure. Pineapple does not belong on pizza.

MICHAEL KROLL: It's just, that's the headline on your—this blog. (Laughter.)

JASON HOWARD: (Laughter.) This episode of the Windows Insider Podcast, does pineapple belong on pizza? No. Thanks for tuning in.

MICHAEL KROLL: Great, this has been great. Awesome.

JASON HOWARD: Oh my goodness. So, I have to say this has been a-a fascinating and enlightening conversation. Like, I-I will say, like, just some of the things that I accidentally learned about Bing today, myself, right, hopefully others out there listening have learned a few things just, you know, from our conversation, and of course the easiest way, just go visit Bing.com, explore it. Check it out. Connect your Microsoft account, so you can take advantage of some of the Rewards programs and some of the additional benefits that that offers. There's a lot to be had there.

But I've got to say, you know, from me to you, Michael, like thank you so much for making the time and joining us here in the studio today. It's been fantastic conversation. I mean, anytime I get to talk about not putting pineapple on pizza, I'm pretty cool with that. (Laughter.)

But you know, for Bing as a whole, right, there's-there's a lot that's out there. There's obviously more to come, and for, you know, for the Insiders out there, don't forget to join the Bing Insider Program, participate in there. You have the same ability to kind of influence and drive the future of that program. And then, of course, keep an eye out for the, you know, the notifications about Bing as we do the Windows releases, because we'll keep you informed that way, as well.

MICHAEL KROLL: Absolutely, and remember to scroll down on the homepage, number one.

JASON HOWARD: Yes, scroll down. No pineapple on pizza. and scroll down on the Bing homepage.

MICHAEL KROLL: And-and please do send feedback. I-I personally read the feedback all the time. It's one of my favorite things to do because, you know, one of the challenges of working at Microsoft is, you're surrounded by people who work at Microsoft, and we're weird. (Laughter.) I don't think it's—like, we're like very, an odd crew, but we're surrounded by people just like us, so we think we're normal.

And so, it's really important to get feedback from actual normal people, so that we can, you know, be—our bubble can keep getting burst in that way, and punctured, and we can see what's really going on, because it's-it's just really refreshing, and I do love it every time. So, please keep it coming.

JASON HOWARD: Awesome. Well hey, thanks again. Really appreciate you making the time.



JASON HOWARD: And with that, Windows Insiders, this episode is a wrap. Thank you to each of our guests and thank you to the listeners for once again tuning into the Windows Insider Podcast.

As we wrap up this, the final episode of 2019, we've entered the holiday season here in the United States. As such, I'll be taking some personal time off, along with many others across Microsoft, and we won't have a podcast episode for the month of January. So, stay tuned in early 2020 for that next episode, and don't forget to subscribe to the podcast on your favorite app to receive it when it becomes available.

A big thank you once again to each of our guests this past year, and happy holidays to all of the Windows Insiders around the world. Until next time!


NARRATION: The Windows Insider Podcast is hosted by Jason Howard and produced by Microsoft Production Studios and the Windows Insider team, which includes Allison Shields, that's me, and Michelle Paison.

Listen to our previous podcasts and visit us on the web at insider.windows.com. Follow us @windowsinsider on Instagram and Twitter.

Support for the Windows Insider Podcast comes from Microsoft, empowering every person and every organization on the planet to achieve more.

Join us next month for another fascinating inside look into Microsoft, tech, innovations, careers, and the evolution of Windows.