(Credit: René Ramos, Bob Al-Greene; Microsoft)
At PCMag, we’re devoted to hands-on testing of products and services, and evaluating security software is a huge part of what we do. We don’t just take the publisher’s word that a product works—we put it up against real-world challenges, including actual malware samples captured in the wild.
We need to test with the same malware samples over time to get an apples-to-apples comparison between different products, but we don’t want to test a product twice with the same samples, lest it get an unfair advantage. Once a year, we go on a hunting expedition for new samples and use them to rebuild the virtual machine testbeds that we use for ongoing evaluations. At some point, we’ll switch those testbeds to Windows 11, but has that time arrived?
In our antivirus reviews, we always consult regularly published reports from four independent antivirus testing labs around the world: AV-Test Institute, AV-Comparatives, MRG-Effitas, and SE Labs. Over many years of covering security and through an association with the Anti-Malware Testing Standards Organization (AMTSO), we've become acquainted with the principals of all four. In 2023, we simply asked them: When are you going to switch to Windows 11? The consensus was, “Maybe next year.” Well, it’s next year—2024. Has anything changed?
How Many People Use Windows 11?
Windows 11 is constantly in the news, but is it on your desktop or your company's systems? If Windows 11 owned the market, deciding to test with it would be a no-brainer. But it doesn’t, not by a long shot. As of this writing, the well-regarded StatCounter shows it has a 28% market share, compared to 67% for the venerable Windows 10.
Peter Stelzhammer, co-founder of AV-Comparatives, pointed out that his company’s yearly survey also covers OS usage. In 2023, Windows 10 was ahead of Windows 11, 50.9% to 38.2%. The latest survey flips those numbers, with 55.0% for Windows 11 and 36.2% still on Windows 10. However, reading the fine print reveals that “the survey’s audience is more tech-oriented and quicker to embrace newer technologies” and that the public adoption rate is much lower.
The MRG-Effitas team reported a lower share for Windows 11 in 2023. “We constantly check the Windows 11 market share, which is 18.13% worldwide,” said Norbert Biro, CTO of MRG-Effitas at the time. “Windows 10 is 68.75%.” This lab’s current stats still show Windows 10 at over 60%.
Asked last year, Andreas Marx, CEO and founder of AV-Test Institute, said, “We're focusing on Windows 10 testing. Most people are still using Windows 10, with a 50%+ market share. Compared with Windows 11, we currently expect about 15–20% market share.” Marx didn't give us statistics this year, but he did say that Windows 10 is still in the lead, as you'll see below.
Does Windows 11 Security Cause Problems Testing Antivirus?
Windows 11, more than any previous Windows edition, has security baked in—you can’t even install the OS if your computer doesn’t support Secure Boot. It requires the presence of a TPM (Trusted Platform Module) chip, which provides hardened storage for the operating system’s essential encryption keys. We worried that these requirements might make Windows 11 too secure, enough so that antivirus programs won’t get a chance to show off their malware-fighting chops. The lab experts quickly disabused us of this notion.
TPM and Secure Boot do not cause any testing problems or other issues so far.
“I can't see any disadvantages to switching [to Windows 11],” said Simon Edwards, founder of SE Labs. “As long as the testing is realistic, it should be fine. I imagine testers that make great assumptions and automate heavily will struggle.”
Stelzhammer agreed, saying “About the TPM stuff: You need hardware (or VMs) which support that, but this does not influence the testing at all.” AV-Test’s Marx also concurred: “TPM and Secure Boot do not cause any testing problems or other issues so far,” said Marx.
Last year, Marx noted that Windows 10 was a more stable platform for testing, though “the protection and false positive rates of Microsoft Defender Antivirus and the third-party products appear to be on the same level.” This year, he explained that Windows 11 remains less stable for testing than Windows 10. “We see some testing issues from time to time, but nothing we were not able to solve yet," he said. “With the platform changing every 2 to 4 weeks (with cumulative and security updates), Windows 11 is more challenging for the AV companies and also the AV testers—and customers, of course.”
Asked if there’s a problem with testing antivirus under Windows 11, MRG-Effitas CTO Biro said, “If you use a desktop PC, yes.” He went on to clarify that it’s simply a practical and financial problem, not any interference with testing itself. “You have to update your PC to a Windows 11-supported CPU version,” he explained, noting that you could also use a virtual machine with a virtual TPM chip to reduce the cost.
When Will We Switch to Testing Antivirus Under Windows 11?
Asked in 2023 when AV-Comparatives would fully switch to Windows 11 for testing, Stelzhammer replied, “We are prepared for testing on Windows 11. Our testing framework supports it already. This year, we will continue testing on Windows 10. Maybe we’ll switch next year.” But the switch isn’t happening. “We will stay with Windows 10 in 2024,” said Stelzhammer. “We might switch to Windows 11 in 2025, at least for consumers. But results from Windows 10 can also be assumed to be correct for Windows 11, as there are hardly any differences in the core of the OS.”
SE Labs echoed that sentiment in 2023. “It’s Windows 10 for us until Windows 11 dominates the market, taking businesses into account as well as consumers,” said Edwards. “I think we're a year away from switching.” As for the present, he said, “We’re sticking with the market-leading OS. That means Windows 10 for now. We’ll keep watching, of course…Maybe we’ll jump to Windows 12 if the market moves in that direction!”
Maybe we'll jump to Windows 12 if the market moves in that direction.
In 2023, Biro didn’t commit to a time frame for MRG-Effitas to make the change, simply saying, “If we see that Windows 11 is taking the lead, we will change it before it happens.” That stance hasn’t changed. Chris Pickard, CEO of MRG-Effitas, said, “Windows 10 still has over 60% of market share, so, for now, we are sticking with it—essentially for those reasons we gave last year.”
When I checked last year, AV-Test was already slotting some Windows 11 tests among its existing Windows 10 tests. That will continue, according to Marx, who said, "We plan to split the tests between Windows 10 and Windows 11 this year, so we will continue testing on both platforms for the time being. As Windows 10 still has a higher market share when compared with Windows 11,” he continued, “we will likely continue with this setup for next year, too, even if Windows 10 gets out of support at the end of 2025.”
We will likely continue with this setup…even if Windows 10 gets out of support at the end of 2025.
We're encouraged to learn that the extra security in Windows 11 doesn’t get in the way of antivirus testing. Based on what the major antivirus labs are doing, though, we're perfectly happy to wait at least another year before switching our own testing to Windows 11.
For more insight into our testing processes, see how we test antivirus and security software and how we collect malware for antivirus testing.
Like What You're Reading?
This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
Sign up for other newsletters
