Introducing a research foundation for testing relational database schemas

Introduction

Database applications are very common and there has been much attention to testing them and the individual database management systems with which they interact. Yet, there has been very little work devoted to testing arguably the most important artifact involving an application supported by a relational database — the underlying schema! The development of a database schema is a process open to flaws like any stage of application development. Examples of potential flaws in database schemas include incomplete primary keys, incorrect foreign keys, and omissions of NOT NULL declarations. The schema’s cornerstone nature to a database application means that defects need to be found early in order to prevent knock-on effects to other parts of an application and the spiraling bug-fixing costs that may be incurred.

Challenges

In this important area of testing for relational database schemas, there are many challenges that researchers and developers need to address. Some recent advances have focused on automatically generating test data to exercise the constraints in the database schema and assessing the effectiveness of the generated data through the use of mutation analysis.

The paper (Kapfhammer, McMinn, and Wright 2013) introduces a search-based technique for generating database table data with the intention of exercising the integrity constraints placed on table columns. The testing tool described in this paper, SchemaAnalyst, automatically generates data that will both satisfy and negate the constraints in the database schema.

Approach

Of course, the process of automatically generating test data raises the question “well, how good is this data?” The paper (Wright, Kapfhammer, and McMinn 2013) introduces efficient techniques for answering this question. The presented mutation analysis methods insert simple faults into the database’s schema and check to see whether or not the test cases can find them. Intuitively, the tests are not very good if they can not find these faults! In recent studies designed to evaluate the effectiveness of SchemaAnalyst we compared it to a tool for generating table data, DBMonster. With competitive or faster data generation times, our method outperforms DBMonster in both the schema constraint coverage and mutation adequacy scores.

Return to Blog Post Listing