Duplicate-sensitivity Guided Transformation Synthesis for DBMS Correctness Bug Detection

Abstract: Database Management System (DBMS) plays a core role in modern software from mobile apps to online banking. It is critical that DBMS should provide correct data to all applications. When the DBMS returns incorrect data, a correctness bug is triggered. Current production-level DBMSs still suffer from insufficient testing due to the limited hand-written test cases. Recently several works proposed to automatically generate many test cases with query transformation, a process of generating an equivalent query pair and testing a DBMS by checking whether the system returns the same result set for both queries. However, all of them still heavily rely on manual work to provide a transformation which largely confines their exploration of the valid input query space. This paper introduces duplicate-sensitivity guided transformation synthesis which automatically finds new transformations by first synthesizing many candidates then filtering the nonequivalent ones. Our automated synthesis is achieved by mutating a query while keeping its duplicate sensitivity, which is a necessary condition for query equivalence. After candidate synthesis, we keep the mutant query which is equivalent to the given one by using a query equivalent checker. Furthermore, we have implemented our idea in a tool Eqsql and used it to test the production-level DBMSs. In two months, we detected in total 30 newly confirmed and unique bugs in MySQL, TiDB and CynosDB.