Fuzzing: what’s behind the automated testing technique

People make mistakes irrespective of the type of work they’re doing, and software development is no exception. Increasingly complex programs that use modules or functions from different vendors contribute to the problem. More often, finished components are continuously integrated. As a result, it becomes hard to predict program behavior in advance. This poses risks, especially if software has to process external data following a manual input or once a file is opened. In the worst-case scenario, if the format of the input does not match the expected format, the program may even crash.

Attackers often use exploits to take advantage of these types of bugs, which is why it’s always a good idea to test software for weaknesses.

Fuzzing, also known as fuzz testing, is an automated method that is ideal for detecting vulnerabilities in software. In this article, you’ll learn what the testing technique involves, its advantages and disadvantages, and the different fuzzing tools you can use.

Fuzzing or fuzz testing was originally developed by computer scientist Barton Miller and is a method used to systematically test software for vulnerabilities. Fuzzing does not attempt to interpret the source code of the program. Instead, it treats the software as a black box and its content as given. In fuzz testing, all possible data input interfaces are automatically invoked and fed random data. This process can take hours or even days, depending on the size of the software project you’re testing.

The aim of fuzz testing is to determine whether the program contains the necessary reactions for all possible input variants. Nonsensical or incorrect inputs should be caught by error-handling routines, if possible. If error-handling routines do not exist for certain inputs or do not work correctly, the program can crash. The technique of testing software for all eventualities has long been an integral part of web development environments. For example, cross-browser testing has been used for many years to test web applications across multiple web clients or browser versions.

There are basically three types of fuzzing:

This fuzzing approach tests functions such as buttons and input fields in graphical user interfaces or options of command line programs. For example, functions are accessed with an abnormally high frequency or speed, or input fields are filled with too much content.

Protocols such as the Hypertext Transfer Protocol (HTTP) are used to exchange data on the web. The transferred data must be in a specific format. Protocol fuzzing is used to test the behavior of the software when incorrectly formatted content is sent. The main goal of this type of fuzzing is to ensure that sent content is not accidentally interpreted as commands and then executed on a server.

Fuzzing tools that follow the approach of file-format fuzzing generate corrupted files and then feed them to the target software for processing. Files typically have a standard format such as .jpg for image files so that they can be exchanged between different applications. If the format of the opened file does not match the expected format, it can cause problems. An advanced version of file-format fuzzing allows you to test implemented features such as compression methods for video files.

Analyzing how a piece of software operates is time-consuming. Software often consists of several thousand lines of source code written in a programming language. Branch instructions and integrated functions make the code difficult to follow. In addition, once the source code has been compiled, it has to be decompiled, which is not always possible.

Fuzzing takes a different approach: It generates all conceivable random data of a given type. The goal is to cover as many input variants as possible. Fuzzing is the interface for automated input. This can happen at different levels. For example, in application fuzzing, different data types are mimicked and passed into the software. Possible lengths and data formats of the inputs can also be tested. Protocol fuzzing uses data packets that you create yourself. Another option involves intercepting, manipulating, and returning contents. File format fuzzing uses randomly generated test files. It tests factors such as size, structure, or set flags.

The primary application of fuzzing is quality assurance in software development. Because fuzzing is highly automated, it allows you to test software regularly in a prepared test environment. You can integrate this environment into existing agile development processes. You can also fuzz-test previously released software as part of a security audit to find out whether the software has security vulnerabilities.

Of course, fuzzing can also help you to detect exploits. However, it only provides clues about errors in program functions, since fuzzing tools ultimately cannot access the internal structures of the software being tested.

A variety of free and paid fuzzing tools with different features are available on the market. Here are just a few of them:

• American Fuzz Lop: Can compile the source code on its own and then test it, which is why this type of program is called a grey-box fuzzer. If the source code isn’t available, it is emulated using QEMU (short for Quick Emulator).
• Fuzzino: Provides a library for generating input data that can be used to test applications using models of protocols.
• LibFuzzer: Library that is part of the LLVM compiler infrastructure.
• ClusterFuzz: Test environment originally developed by Google to test the Chrome browser.
• Sulley: Provides a collection of tools in the Python scripting language; especially useful for simple test cases such as generating random data.
• Peach: Highly automated solution for fuzz-testing hardware and software.
• Powerfuzzer: Provides different attack scenarios, such as SQL injections; features a web-based graphical user interface.

Depending on the intended application, there are alternatives to fuzz testing that take a different approach and deliver different results. We’ve summarized the most important ones for you below.

Reverse engineering is an indispensable tool for security researchers. Reverse engineering is the process of analyzing code after decompilation (reverse translation into programming languages) or disassembly (translation into the low-level assembler language). Reverse engineering is a very complex and error-prone process. When successful, however, it allows you to examine all the details of program functionality.

Debugging is the process of finding bugs during software development. Programs are executed temporarily in the development environment before they are completed. The programmer can set breakpoints to interrupt the program at certain places during execution. Next, the programmer can view the contents of variables and check them for correctness. Debugging is also used in reverse engineering.

Complex software often has a modular design. Module testing is a well-established method for checking individual components in test environments.

As a programmer, bugs and security vulnerabilities in your software can damage your reputation and cause others to lose trust in your work. Expecting a program to be absolutely flawless is unrealistic, and most software users know this. However, by fuzz-testing your software, you have the opportunity to find bugs before others report them. Beyond that, this technique will boost the stability and security of your software. These are all good reasons to make fuzz testing part of your quality assurance process.