A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark - Université Polytechnique des Hauts-de-France
Communication Dans Un Congrès Année : 2019

A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark

Résumé

Automatic program repair papers tend to repeatedly use the same benchmarks. This poses a threat to the external validity of the findings of the program repair research community. In this paper, we perform an automatic repair experiment on a benchmark called QuixBugs that has never been studied in the context of program repair. In this study, we report on the characteristics of QuixBugs, and study five repair systems, Arja, Astor, Nopol, NPEfix and RSRepair, which are representatives of generate-and-validate repair techniques and synthesis repair techniques. We propose three patch correctness assessment techniques to comprehensively study overfitting and incorrect patches. Our key results are: 1) 15/40 buggy programs in the QuixBugs can be repaired with a test-suite adequate patch; 2) a total of 64 plausible patches for those 15 buggy programs in the QuixBugs are present in the search space of the considered tools; 3) the three patch assessment techniques discard in total 33/64 patches that are overfitting. This sets a baseline for future research of automatic repair on QuixBugs. Our experiment also highlights the major properties and challenges of how to perform automated correctness assessment of program repair patches. All experimental results are publicly available on Github in order to facilitate future research on automatic program repair
Fichier principal
Vignette du fichier
1805.03454.pdf (1.25 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03522321 , version 1 (25-04-2022)

Identifiants

Citer

He Ye, Matias Martinez, Thomas Durieux, Martin Monperrus. A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark. 2019 IEEE 1st International Workshop on Intelligent Bug Fixing (IBF), Feb 2019, Hangzhou, China. pp.1-10, ⟨10.1109/IBF.2019.8665475⟩. ⟨hal-03522321⟩
25 Consultations
71 Téléchargements

Altmetric

Partager

More