Ph.D. Dissertation Defense

Towards Automatic and Accurate Buffer Overflow Vulnerability Diagnosis For Commodity Software

Jiang Zheng (CS Grad/Pitt)

Friday, September 19th, 2008
2:00pm - SENSQ 6106 - Eli Lilly Room

Abstract

Buffer overflow attacks are still one of the major computer security threats. It has been a computer security threat in software-based systems and applications for decades. Researchers have proposed different techniques to defend against unknown buffer overflow attacks and have also investigated various solutions to protect known vulnerable software systems (e.g. signature generation, patch generation, etc.). However, vulnerability diagnosis is still an open problem in software security research. Currently, this is done manually by security engineers.

This thesis defines automatic buffer overflow vulnerability diagnosis (BOVD) problem and provides solutions towards automatic and accurate BOVD for commercial software. The automatic BOVD problem is: given a buffer overflow vulnerable program P and a working exploit E, we want to automatically and accurately find out where the program is vulnerable, buffer overflow vulnerability point (BOVP) and figure out why the program is vulnerable, buffer overflow vulnerability condition (BOVC). It combines both of dynamic analysis techniques and static analysis techniques to achieve the goal. It extends the existing dynamic-taint-analysis attack detection tool to do the BOVP detection and performs three steps analysis towards understanding BOVC.

The results of the BOVC include the location of the vulnerable buffer, the size of the vulnerable buffer (it can be a variable or a constant), the factors that cause the buffer overflow (e.g. source buffer size or a bound variable) and how the user inputs are related with those factors and the vulnerable buffer size. The three steps approach includes loop analysis, bound analysis and input analysis. Based on the observation of all known buffer overflow attacks that most of them take place in loop context, we perform loop analysis to understand the factors that can cause buffer overflow. We do bound analysis to infer the location and size information of the vulnerable buffer. The input analysis step will tell us how the user inputs are related with them.

Due to our novel bound analysis approach, this thesis work can handle the buffer overflow happens anywhere including stack, heap, Data/BSS. It can also be generalized to diagnose most of the memory corruption attacks including buffer overflow, format string, integer overflow that triggers buffer overflow in the working exploit and double free. We demonstrate the effectiveness of this approach using six real world vulnerable applications.

The evaluation results demonstrate that this approach is effective for most of the real world buffer overflow vulnerability cases and other memory corruption attacks. We also do the complete buffer overflow cases study which may have independent interests to researchers. The results of the buffer overflow cases study can help researchers to design more effective buffer overflow defense systems.

Dissertation Adviser

Prof. Jose Carlos Brustoloni, Department of Computer Science

Committee Members

Prof. Bruce Childers, Department of Computer Science
Prof. Shi-Kuo Chang, Department of Computer Science
Prof. James Joshi, School of Information Science
Prof. Dawn Song, Department of Computer Science, UC Berkeley