Automated Testing

Automated Testing is the automating of test cases so that tests may be run multiple times with minimal efforts. To do this, the test cases themselves are code. Three advantages to automated testing are:

  1. confidence that integrating functionality into the code base does not break the existing code, and that the new code meets its requirements,
  2. an understanding of the level of completed functionality in the system, and
  3. the compilation of a regression test suite, which ensures the system is meeting all requirements.
However, automating tests can be time consuming and expensive. The test code base can be as large as the project's code base, and test code must be maintained.

Automated tests should follow these guidelines from The Test Automation Manifesto:

  • Concise: As simple as possible and no simpler.
  • Self Checking: Test reports its own results; needs no human interpretation.
  • Repeatable: Test can be run many times in a row without human intervention.
  • Robust: Test produces same result now and forever. Tests are not affected by changes in the external environment.
  • Sufficient: Tests verify all the requirements of the software being tested.
  • Necessary: Everything in each test contributes to the specification of desired behavior.
  • Clear: Every statement is easy to understand
  • Efficient: Tests run in a reasonable amount of time.
  • Specific: Each test failure points to a specific piece of broken functionality; unit test failures provide "defect triangulation"
  • Independent: Each test can be run by itself or in a suite with an arbitrary set of other tests in any order.
  • Maintainable: Tests should be easy to understand and modify and extend.
  • Traceable: To and from the code it tests and to and from the requirements.

Author: Laurie Williams and Sarah Heckman
Maintained By: Sarah Heckman
Last Updated: 2008-08-25