Smart fuzzing method for detecting stack-based buffer overflow in binary codes

During the past decades several methods have been proposed to detect the stack-based buffer overflow vulnerability, though it is still a serious threat to the computer systems. Among the suggested methods, various fuzzers have been proposed to detect this vulnerability. However, many of them are not smart enough to have high code-coverage and detect vulnerabilities in feasible execution paths of the program. The authors present a new smart fuzzing method for detecting stack-based buffer overflows in binary codes. In the proposed method, concolic (concrete + symbolic) execution is used to calculate the path and vulnerability constraints for each execution path in the program. The vulnerability constraints determine which parts of input data and to what length should be extended to cause buffer overflow in an execution path. Based on the calculated constraints, the authors generate test data that detect buffer overflows in feasible execution paths of the program. The authors have implemented the proposed method as a plug-in for Valgrind and tested it on three groups of benchmark programs. The results demonstrate that the calculated vulnerability constraints are accurate and the fuzzer is able to detect the vulnerabilities in these programs. The authors have also compared the implemented fuzzer with three other fuzzers and demonstrated how calculating the path and vulnerability constraints in the method helps to fuzz a program more efficiently.