The IoT: Too big (and buggy) to patch?
The Internet of Things isn’t too big to fail, but is it too big to patch? IoT security experts talk about why and how to upgrade IoT device firmware.
The Internet of Things (IoT) will never be too big to fail, although it is hard to conceive of the entire thing failing at once, unless every power grid on the planet goes down simultaneously.
But it is in danger of increasing incremental failure because it is too big to patch, according to author, encryption guru, and premier blogger Bruce Schneier.
How did the problem get so big?
Schneier, CTO of IBM Resilient Systems, wrote a post focused mainly on the recent disclosure of serious flaws in encryption standards OpenPGP and S/MIME, which are used by numerous email clients to keep communications private.
A team of researchers had published a proof of concept that they could trick any of those vulnerable clients by altering an email sent to it, resulting in a plaintext copy of the email being sent to a server controlled by an attacker.
One reason it’s a serious problem is that the vulnerability is not confined to one client, such as Thunderbird or Apple Mail, but affects any of them that are using a standard that has been around for nearly three decades. And dozens of them are—the researchers said they found that plaintext exfiltration channels exist for 25 of the 35 tested S/MIME email clients and 10 of the 28 tested with OpenPGP.
The other reason is that vulnerable people—journalists, political dissidents in repressive regimes, whistleblowers, and human rights advocates—rely on those clients to protect their privacy, and therefore their personal safety.
And it is all going to take a while to fix since, as he put it, it involves multiple “communities without clear ownership.”
“In this case, there’s nothing wrong with PGP or S/MIME in and of themselves,” he wrote. “Rather, the vulnerability occurs because of the way many e-mail programs handle encrypted e-mail.”
Which led Schneier to what he sees as a much bigger problem, given that “the Internet is shifting from a set of systems we deliberately use—our phones and computers—to a fully immersive Internet-of-things world that we live in 24/7…[where] vulnerabilities will emerge through the interactions of different systems.”
It also suffers, he said, from many vendors not even having the expertise and capability to patch the software in what they sell, because it is frequently designed by “offshore teams that come together, create the software, and then disband.”
Many devices, he noted, aren’t patchable at all—the only way to “fix” a digital video recorder that is vulnerable to being conscripted as part of a botnet is to “throw it away and buy a new one.”
Bottom line? “Patching is starting to fail, which means that we’re losing the best mechanism we have for improving software security at exactly the same time that software is gaining autonomy and physical agency,” he wrote.
Is patching starting to fail?
Which raises the obvious question: What should IoT developers and manufacturers and the software security industry do about it?
Schneier’s view is well-known. He has testified before Congress in favor of government mandates for basic security standards for IoT devices because, as he has written on his blog numerous times, the market won’t do it. “It’s hard to see any other viable alternative [than government intervention],” he wrote.
That sentiment gets mixed reviews from other security experts, in part because not everybody shares such a bleak view of the current state of IoT.
Zach Lanier, principal research consultant with Atredis Partners, says he doesn’t think the situation is as ominous as Schneier does. But he agrees that “the gap between ‘patchability’ of disparate components—from overall firmware to specific components like OS/RTOS, drivers, applications, etc.—is very wide and may certainly be growing, especially with the introduction of niche IoT vendors and their respective devices.”
But Jesse Victors, security consultant with Synopsys Software Integrity Group, said it simply isn’t the case that every device, or even most of them, is built by a team that disbands as soon as it has completed a project.
“I disagree with the premise,” he said. “I see the emergence of IoT devices managed by well-known companies, such as Samsung, Nest, Tesla, Apple, Google, or Amazon. These companies have dedicated teams to their IoT infrastructure, respond to security researchers, and push updates on their own initiative or when pressured to do so.”
And regarding the design flaws in OpenPGP and S/MIME, Larry Trowell, associate principal consultant with Synopsys SIG, said that while “patching the lack of authenticated encryption in the design at this stage would be a herculean task,” the problem can be avoided simply by not using it “in tangent with an automated software retrieval process, but for manual file verification and signature checks.
“Sometimes pieces of software just don’t work correctly together,” he said.
Overall, he said while problems do exist, “patching in IoT isn’t starting to fail, it’s starting to get fixed.
“In the past there was no demand, customers didn’t know better, the developers didn’t think these devices would be around for long, or would be that interesting to attackers. They have since learned, and little by little it’s become a market concern,” Trowell said.
Is government regulation of IoT the solution?
Neither Victors nor Trowell thinks government regulation and oversight will fix the security problems that ail IoT.
Indeed, the federal government has a poor track record of securing its own data, never mind devices. Just two examples are the breach of the Office of Personnel Management (OPM), discovered in 2014, and the compromise of National Security Agency (NSA) hacking tools in 2016.
“Government certification does not work for marking cryptographic libraries secure,” Victors said, “and it will be equally ineffective for IoT security.”
He said he has seen proposals for federal certification bodies, “but I foresee them falling behind in technical understanding, not adapting to new technologies and connectivity relationships, encouraging IoT manufacturers to hide infrastructure, or generally being toothless.”
Trowell added that government involvement could “infringe on the right to repair and the ability to tinker with devices.”
What’s the alternative?
And Victors believes there are other, and better, “viable alternatives” to government regulation.
He said an independent, consumer-friendly organization could rank an IoT device in areas like “whether it transmits user data overseas, whether it self-applies firmware updates, whether it is exposed to the public internet, whether the company is maintaining it, and so on.”
A body like that, he added, could coordinate the sometimes-contentious relationship between security researchers and vendors when it comes to reporting the discovery of vulnerabilities.
Of course, a majority of the security failures that plague IoT products could be avoided by “building security in” from the start of the design phase throughout the development life cycle.
But, even that wouldn’t eliminate every vulnerability, a feat Lanier said will likely take a systemic overhaul. It’s not just vendors and developers who need security expertise but platform manufacturers and service operators as well.
“In some cases they do provide sane and secure defaults, security features, appropriate feedback mechanisms for when something is ‘not okay,’ and robust, usable software/firmware update mechanisms,” he said.
“I don’t know that there’s really a clear answer on how to fix this en masse, but the IoT-device-du-jour building on a platform/stack that ‘doesn’t suck’ is a good start.”
IoT devices must allow easy firmware upgrades
Victors agrees that IoT devices must be designed to allow firmware upgrades easily—which is not the case in most Wi-Fi routers in use today.
A huge percentage of them “are rarely upgraded; their owners are not aware or not technically savvy enough to perform the upgrade, or the device itself cannot download the patches and upgrade itself.”
“This absolutely needs to change,” he said. “We cannot assume that the first production version will be sufficient over the long term.”
And Trowell’s view is that even though the market hasn’t fixed the problem yet, it remains the only viable way to do it. “I don’t think one country or one government mandating the fix is going to do much,” he said. “I think it will only change when the majority of consumers care and demand it.”
Will that happen? Lanier is dubious along with Schneier. “Outside of clued-in organizations or enterprises that actually do some kind of risk analysis on random IoT devices being introduced into their networks, I don’t see most end users—consumers—really making security-conscious decisions anytime soon,” he said.
