The ratio of equivalent mutants: A key to analyzing mutation equivalence

Mutation testing is the art of generating syntactic versions (called mutants) of a base program, and is widely used in software testing, most notably the assessment of test suites. Mutants are useful only to the extent that they are semantically distinct from the base program, but some may well be semantically equivalent to the base program, despite being syntactically distinct. Much research has been devoted to identifying, and weeding out, equivalent mutants, but determining whether two programs are semantically equivalent is a non-trivial, tedious, error-prone task. Yet in practice it is not necessary to identify equivalent mutants individually; for most intents and purposes, it suffices to estimate their number. In this paper, we are interested to estimate, for a given number of mutants generated from a program, the ratio of those that are equivalent to the base program; we refer to this as the Ratio of Equivalent Mutants (REM, for short). We argue, on the basis of analytical grounds, that the REM of a program may be estimated from a static analysis of the program, and that it can be used to analyze many mutation related properties of a program. The purpose/ aspiration of this paper is to draw attention to this potentially cost-effective approach to a longstanding stubborn problem.