12.4 - Machine Learning-Driven Test Case Prioritization Approaches for Black-Box Software Testing
- Event
- ettc2018 - European Test and Telemetry Conference
2018-06-26 - 2018-06-28
Nürnberg, Germany - Chapter
- 12. Data Management Applications
- Author(s)
- R. Lachmann - IAV GmbH,Gifhorn (Germany)
- Pages
- 300 - 309
- DOI
- 10.5162/ettc2018/12.4
- ISBN
- 978-3-9816876-7-5
- Price
- free
Abstract
Regression testing is the task of retesting a software system after changes have occurred, e.g., after a new version has been developed. Usually, only a subset of test cases is executed for a particular version due to restricted resources. This poses the problem of identifying important test cases for testing. Regression testing techniques such as test case prioritization have been introduced to guide the testing process. Existing techniques usually require source code information. However, system testing of complex applications often restricts access to the source code, i.e., they are a black-box. Here, a large set of test cases is manually executed. In previous work, we proposed a test case prioritization technique for system testing using supervised machine learning. We designed our approach to prioritize manually executed test cases, i.e., it analyzes meta-data and natural language artifacts to compute test case priority values. In this paper, we apply further machine learning algorithms and an ensemble learning approach. In addition, we evaluate our approach on three different data sets in total, which all stem from the automotive industry and, thus, represent real life regression testing data sets. We analyze the results of our approach in terms of its failure finding potential. Our findings indicate that black-box testing can be improved using machine learning techniques.