Double test coverage? Maybe – but maybe not really.

Some years ago, I inherited an arcane testbed. It was based on a test tool that was already end-of-life, plus no other testers in the company knew anything about it,  and it used a dedicated configuration that could not be regenerated, only backed up and restored.. Also it was not practical to use with  the local automated testing framework. For these excellent reasons, my manager wanted to junk that testbed. But, my Devs really liked the testing I did with it, it was a primo tool for uncovering a certain class of bugs in an important subsystem.

Solution! There’s a another test tool which can provide the same functionality as the end-of-life one, the company owns several units of it and maintains paid support for them, and it is an accepted tool for use with the ATS (multiple testers have written test modules that use it). I should re-implement that testbed that no one else can use, as an ATS-compliant module based on that other tool, and then we can junk the old one.

OK, I did that … and discovered that although the tests appeared to be doing the same thing, the 2 testbeds uncovered some different bugs. The implementation details were just different enough to matter – different testbed configuration, differences in how the calls were established, and especially differences in how the 2 tools implemented “detect an audio tone at a specified frequency”. The arcane testbed remained in service along with the new one.

So before you discard a test that seems to be a duplicate, to a little checking to see if it actually is adding value.