Don't Cry Wolf

• When tests fail, it should indicate a real problem.
• "Expected failures" sap that trust.
  • "Oh, don't worry, that test always fails on Redhat 6.2"
  • If a failure sometimes isn't really a failure, how do you know what's a real failure?

• "Expected failures" make test automation impossible.
  • Programs don't know "well, the test failed but it really passed"
  • Joe CPAN module installer also doesn't know that.

Prev | toc | Next