Parameter tuning for search-based test-data generation revisited: Support for previous results

Although search-based test-data generators, like EvoSuite, efficiently and automatically create effective JUnit test suites for Java classes, these tools are often difficult to configure. Prior work by Arcuri and Fraser revealed that the tuning of EvoSuite with response surface methodology (RSM) yielded a configuration of the test data generator that did not outperform the default configuration. Following the experimental design and protocol described by Arcuri and Fraser, this paper presents the results of a study that lends further support to prior results: like RSM, the EvoSuite configuration identified by the well-known Sequential Parameter Optimization Toolbox (SPOT) failed to significantly outperform the default settings. Although this result is negative, it furnishes further empirical evidence of the challenge associated with tuning a complex search-based test data generator. Moreover, the outcomes of the presented experiments also suggests that EvoSuite’s default parameters have been set by experts in the field and are thus suitable for use in future experimental studies and industrial testing efforts.