It’s nearly the start of another school year. As the summer comes to an end, I’m working to figure out exactly what I want to do for my school research project. I get to select any topic that interests me as long as the research process for that topic involves the scientific method.
A few months ago, I started to hash out a number of ideas that could work as research projects. From computer graphics to 3D fractals to procedurally generating music to building a whole new OS kernel, I considered all kinds of different options that would allow me to learn something interesting and produce useful research. As I narrowed down my search, I eventually realized that I wanted to focus on some aspect of security research.
Fuzzing is a very simple concept that has been proven to be extremely useful in the context of automatically discovering security vulnerabilities in software. With modern coverage-guided fuzzing, the process is even more efficient. Companies like Google and Microsoft run fuzzers on in-house and third-party software around the clock, finding strange edge cases and critical security vulnerabilities regularly.
At this point, I’m basically certain that I want to incorporate fuzzing into my research project. I’ve been reading dozens of research papers—ranging from the original invention of fuzzing by Barton P. Miller, Lars Fredericksen, and Brian So to state-of-the-art techniques like Driller.
While I know that I want to study fuzzing (and maybe build my own fuzzer to discover previously unknown security vulnerabilities), I have relatively little figured out after that.
Possibilities and where to next
To be able to proceed at this point, I need to decide on two things: a target or targets to look for bugs in and a technique for attacking that target. Simply running AFL or libFuzzer on a new target would not make a very good research project. I want to either come up with a new fuzzing technique or explore ways to make an existing technique more efficient under certain conditions. Whether or not I end up finding any big security bugs, I want to have something meaningful to show for my work.
Some thoughts I’ve had on potential target options:
- Operating system kernels. It’s possible that more obscure and less well-tested OSs would have kernels that are more vulnerable to fuzzing.
- Fundamental system utilities/libraries like various
libcs, init systems (particularly those written in compiled languages), and other libraries that are used throughout a certain system.
- Popular native code libraries that do things like parsing serialization formats and processing images.
- Multiplayer video games written in C or C++ with network-accessible components.
- Text editors and syntax highlighting tools.
- Closed-source system libraries and default apps on iOS. A lot of this is written in Objective-C and could potentially contain remotely exploitable vulnerabilities. The hard part here would be running code on the device that interacts with these libraries and dealing with the fact that compiler-inserted instrumentation for coverage-guided fuzzing is impossible when dealing with closed-source binaries.
- Popular but oftentimes-forgotten web applications like GNU Mailman (even big companies like Apple sometimes forget to update web apps like that; see the 2020-05-15 entry on Apple’s support page where I’m acknowledged for reminding them to update Mailman to fix a vulnerability).
As for potential fuzzing techniques, here are some of my ideas (many of which have already been explored to some extent before):
- Approximating coverage-guided fuzzing without access to source code or the ability to recompile.
- Mutating test cases in ways that are more realistic and tailored to specific formats than the techniques employed by AFL.
- Using alternative metrics (i.e. not code branch coverage) as a fitness function for the evolutionary algorithm that generates test cases.
I’ll keep reading and narrowing down my ideas based on what I find. This blog will serve as a place to document what I’ve been reading and exploring as my research project progresses. Stay tuned!