How to Safeguard Automotive OTA Updates at Scale

Chris Clark

Jan 04, 2023 / 6 min read

Imagine driving down a scenic country road at night and seeing the gleam of a deer’s eyes ahead. Before you can react, the car slows and steers away from the deer and avoids the car in the next lane. This is the power of driver assist and AI-powered autonomous driving. Now imagine the same scenario a few days later, and the vehicle does not respond. Not only is the vehicle damaged, but there is also wildlife and passenger injury. What happened?

This is where the power of automotive over-the-air (OTA) updates comes into play. Manufacturers can now determine not only how the vehicle reacted but why. Was it a software error, a bad software update, or simply an adjustment in operating parameters that led to the event? Never have manufacturers had such insight into a vehicle’s operation. Never have they had the ability to fix issues at such large scale without a service visit. OTA update capabilities provide great benefits and bottom-line savings, but poor implementation or bad software development practices could lead to costly mistakes.

OTA capabilities also provide a way to mitigate what is foremost on many minds. Cyberattack.

Whether a problem with a vehicle is due to a software bug, a bad update, or a cyberattack, it makes no difference for the consumer. OTA updates can be done at scale, potentially impacting hundreds of thousands of vehicles during adverse events such as a cyberattack or a poorly tested update. OTA is a much faster, more immediate critical path to fix issues compared to legacy recall processes that could take months and lack any guarantee of being done, if ever, or alternatively, those accomplished by chance through a random service visit. Not to mention, OTA updates are much more cost effective than a recall.

In most cases, automotive OTA updates are seamless for vehicle owners. The transparent nature of OTA minimizes required trips to your local service dealer and the time lost to the owner. Aside from safety and security fixes, the updates can add new feature enhancements that were not available at manufacturing, either to improve your driving experience or, in some cases, to extend the life of vehicle components. OTA updates can improve the safety and security of your ride and adapt it as technology continues to evolve in billions of lines of code, increasing interconnectedness and convergence of software-defined, autonomous vehicles.

With these new features and OTA itself, the overall attack surfaces of the vehicle grow, as does the potential for error. In fact, in November, Tesla recalled 40,000 vehicles due to an error in an OTA update that resulted in power steering issues after hitting a pothole or bump in the road. Honda recalled over 600,000 vehicles just two years ago due to software bugs causing incorrect driver speed readout and problems with the rearview camera video. Mercedes recalled over 1.3 million vehicles due to a glitch in the code for an emergency location tracker, and the carmaker enabled the fix through an OTA update.

OTA updates are a great solution when problems arise, but scale and speed make bullet-proofing your hardware and software development practices critical to a successful program. With today’s technical complexity and so much at stake, here are a few ways to better safeguard systems and minimize bugs before OTA updates deploy.

Automotive Digital Twins

Address Systemic Infrastructure and Process Innovation

OTA updates are complex—they don’t come as a single, simple solution wrapped in a neat box ready to be deployed. Because of this, it’s important to focus on the underlying systemic infrastructure for delivering your software, along with the appropriate processes and checks that verify everything works as it should under any circumstance. In fact, a great number of activities need to be in place to have an effective OTA program. These activities tend to fall within three categories:

  • Robust public key infrastructure (PKI)
  • Well-defined software verification and validation practices
  • Effective vehicle software deployment, storage, and activation

Robust Public Key Infrastructure

While key management at scale has been around for a while—think credit card applications in retail—it’s still a relatively new concept for automotive applications. For instance, properly managing PKI for automotive involves working with a wide range of technologies and participants, including automotive manufacturers, OEMs, and the key infrastructure management organizations that help comprise the OEM solutions.

Bad actors can spoof or even steal authentication keys to deliver contaminated software or block service access during a breach. Once a compromise occurs, managing it is no easy task. You’ll need to revoke not a hundred keys but tens to hundreds of thousands of keys throughout the supply chain and reissue them. This is just one reason securing PKI is a multi-disciplinary, multi-organizational undertaking involving the parties’ full participation.

Creating a simulation environment that can be used to test and evaluate a process is critical to success. Because it is not if an attack will happen, it is when. PKI programs should encompass testing for a wide range of fault conditions and corner cases that an attacker could take advantage of. This program should also follow ISO/SAE 21434 practices to ensure the process is part of the development program and evolves with software features and capabilities.

Well-Defined Software Verification and Validation Practices

There are many software development aspects to consider in the software verification and validation process. One area that is rapidly growing is hardware virtualization for software testing. With the rapid changes in software and the volume of developers participating, it is impossible to have enough hardware test beds to test software early in the development cycle effectively. Virtualized hardware provides access to hardware simulants that can be integrated into the CI/CD pipeline and made available to a much broader development audience. This also includes development for a post-production environment. Hardware access is much easier, but due to the volume of hardware, space constraints, and lack of expertise, this begins to become impractical rather quickly.

Leveraging virtualization technologies like Synopsys Virtualizer™ and other simulation technologies enable developers to quickly deliver unit- and system-level tests in a protected environment. This form of testing adds a level of rigor to your software test cycle, reducing the potential for defects or unintended interactions between test software and deployed software that would not be possible in traditional testing environments.

Effective Vehicle Software Deployment, Storage, and Activation

Even with the most capable and extensive levels of testing, there will be issues. Reasons can include poor post-deployment instructions, unintended interaction, or after-market adaptations. The scenarios are endless. What is important is to have safeguards in place and adaptable practices for addressing a wide range of issues and conditions.

Establishing policy for verification and validation of updates is key. Yes, many organizations have these policies in place, but do they take into account partially failed deployments? Do they think “out of the box?” The important takeaway is that while simple steps, such as code signing checks, are important, simple checks for wrong platform deployment are much more difficult.

Once an update has been pushed to a vehicle, the storage and activation of updates must take place perfectly. Storage of software for updates requires architectural decisions, such as the amount of in-vehicle storage and flash memory. How much is enough? Will the amount be suitable for the vehicle’s lifespan? Will it be suitable for the number of reads and writes the specific type of memory will experience? These types of decisions will have an impact on the volume, cadence, and size of software updates. Not enough memory and a copy of the current firmware won’t be stored for rollback. Too much and manufacturing costs may be too high.

Automotive OTA is Just One Piece of a Larger Safety and Security Story

OTA is not a silver bullet. It is one piece of a broader safety and security puzzle. When you deliver updates to a vehicle, you must go through a robust software verification and validation process that ensures the delivery and engagement within the vehicle is as defect-free as possible, both from vulnerabilities as well as a quality and safety standpoint. Whatever you roll out could impact tens of thousands of people, so the verification and validation must be exacting.

Modern software development is a multidisciplinary, multi-organizational endeavor that must keep our cars and roadways safe. Choosing the right partner is key, and Synopsys can help.

Synopsys delivers industry-leading IP, design tools, verification and validation solutions, and software integrity tools capable of addressing your needs. We leverage our deep experience in a wide range of industries to accelerate innovation in the automotive industry, so OTA software updates work the first time and every time.

Continue Reading