An Independent Validation of Vulnerability Discovery Models
Having a precise vulnerability discovery model (VDM) would provide a useful quantitative insight to assess software secu- rity. Thus far, several models have been proposed with some evidence supporting their goodness-of-t. In this work we describe an independent validation of the applicability of six existing VDMs in seventeen releases of the three popular browsers Firefox, Google Chrome and In- ternet Explorer. We have collected ve dierent kinds of data sets based on dierent denitions of a vulnerability. We introduce two quantitative metrics, goodness-of-t en- tropy and goodness-of-t quality, to analyze the impact of vulnerability data sets to the stability as well as quality of VDMs in the software life cycles. The experiment result shows that the\conrmed-by-vendors' advisories" data sets apparently yields more stable and bet- ter results for VDMs. And the performance of the s-shape logistic model (AML) seems to be superior performance in overall. Meanwhile, Anderson thermodynamic model (AT) is indeed not suitable for modeling the vulnerability dis- covery process. This means that the discovery process of vulnerabilities and normal bugs are dierent because the in- terests of people in nding security vulnerabilities are more than nding normal programming bugs.