Due to the new ISO/SAE 21434 many car manufacturers (OEMs) are expanding their software testing activities. The standard contains regulations for software devices within vehicles, as well as their connectivity to external systems. This ISO recommends OEMs to integrate feedback-based fuzz testing into their DevOps processes and defines new requirements for software security engineering.
Fuzzing or fuzz testing is an automated security testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. One of the main advantages of fuzzing is that it can be done during development, similar to dynamic application security testing (DAST) and interactive application security testing (IAST).
As a Customer Success Engineer that deals with automotive customers on a daily basis, I see three main challenges that developers and security teams face when implementing fuzzing for their automotive software.
This article is a part of a series on automotive security testing in which I will touch all three questions. For more articles on testing automotive software, check out the Code Intelligence - Security Blog. In this article, I will now primarily focus on the question: “Where to start fuzz testing?”
Start simple and proceed to the more complex challenges. Developers (including myself) sometimes tend to start with the most complicated project or the one that promises the most bugs. But I made the experience that when it comes to fuzzing it is often better to keep it simple.
For example, I would always recommend starting fuzzing automotive software with a simple C/C++ library that already got tested by your security experts. This way you can get a better feeling for the fuzzing process. Later, you can work your way up to more complex projects step-by-step.
In many cases, fuzz testing will eventually find a lot of new bugs in these already tested C/C++ libraries, even if you thought this code was safe. (Great, a first success! But please be gentle with the person who tested this code last time. Remember… there is also a human component to automated bug finding.)
I would always recommend automotive developers implementing fuzz testing in three simple steps. With each step, you will become more confident and more proficient in what you do. This way you will get the bests results. And each little success will lead you to even more advanced fuzzing approaches that help you make your software more secure.
*Click to enlarge
When initially setting up fuzz testing in an automotive environment, the easiest way to do this is using pre-existing unit tests as a sort of template. In most projects, these unit tests already exist, since many automotive developers use unit tests to test the functionality of their software.
During the first hours after setting up the fuzzer, you will usually find the majority of security issues, just by testing with these self-contained units. Ideal candidates for these low-hanging fruits are libraries, API functions, and different kinds of parsers.
In this stage, you can also set up a continuous integration very easily, by simply repeating this already defined procedure at every upcoming merge request (similar to DAST or IAST).
On the network level, fuzzing requires more efforts as different network interface protocols have to be taken into account (TCP/IP, Ethernet, CANBus, etc.). To simulate this, the fuzzing engines have to be able to understand certain protocols, such as IPSec, UDS, and Protobuf. That's why, interface fuzzing can take several days to weeks, but as the complexity increases, you will also find more bugs.
If you would start with interface fuzzing before unit test fuzzing you would most likely also detect all the findings that you could detect with unit test fuzzing. However, it might take you more time to debug and analyze the bugs when they are found via interface fuzzing. Therefore, it's usually better to start with unit fuzzing first.
Especially in automotive and complex embedded projects, system fuzzing is the long-term goal. At a certain point all tests should be connected to the hardware to find out what actually happens on the operating system with the commercial compiler. Luckily, it's possible to simulate the input of those systems. The simulations built during unit testing can serve as a fundament for testing the actual hardware.
Long story short: start with the low-hanging-fruits and work your way up. You can already do a lot with a pretty basic fuzzing set-up. But in the long game, you should find a way to implement fuzz testing into your CI/CD.
If you liked this article, feel free to join me at my next live coding session, where I will demonstrate how to fuzz embedded systems with dependencies.
Daniel Teuchert is a Customer Success Engineer at Code Intelligence. He holds a master's degree in IT Security as well as OSWE and OSCP offensive security certifications. Together with the Customer Success Team of Code Intelligence he strives to revolutionize the way the world tests software by supporting numerous companies in implementing modern fuzz testing.