Testing and maintenance · ⁨测试与维护⁩

Thirty-seven seconds

• On 4 June 1996 the first Ariane 5 rocket lifted off from French Guiana. Thirty-seven seconds later it veered off course and destroyed itself. The payload was four satellites worth $370 million.
• The cause was one line of code, reused from Ariane 4, that converted a 64-bit number into a 16-bit integer. Ariane 5 flew faster, the number was bigger, and the conversion overflowed: a run-time error 运行时错误 in a routine that was not even needed after lift-off.
• The code had never been tested with Ariane 5's flight data. Nobody had chosen test data at the extremes of the new rocket's range.
• This lesson is the three kinds of error, the methods of testing, how to choose test data, and how a program is kept alive after release.

Three kinds of error

• A syntax error 语法错误 breaks the grammar of the language: a missing bracket, a misspelled keyword. It is found at translation, so the program will not run until it is fixed.
• A run-time error happens while the program runs: division by zero, a file that does not exist, an array index out of range. The program crashes or raises an exception; the fix is a check before the risky operation.
• A logic error 逻辑错误 lets the program run and produce wrong results: + for -, an off-by-one loop, conditions in the wrong order. Nothing flags it; only testing and tracing reveal it.

Found at translation, found at run time, found only in the output

Worked example: find and correct the errors

Testing methods: reading the code

• A dry run 手工跟踪 traces the code on paper, writing each variable's value in a trace table after each line.
• A walkthrough 走查 is a team review: the programmer explains the code line by line while colleagues look for faults.
• White-box testing 白盒测试 designs tests from the code's internal structure, so that every statement, branch and loop is exercised. Black-box testing 黑盒测试 designs tests from the specification only: feed inputs, compare the outputs with what was expected, without looking at the code.

Outside looking in, or inside looking at every path

Testing methods: assembling and releasing

• Integration testing 集成测试 combines modules that were tested separately and tests the interfaces between them. A stub 桩 stands in for a module that is not yet written, returning fixed values so the rest can be tested top-down.
• Alpha testing α测试 is done in-house by the developers before release. Beta testing β测试 gives a limited group of real users the program in their own environment.
• Acceptance testing 验收测试 is done by the customer, against the requirements, to decide whether the product is fit for purpose.

Worked example: which method for which situation

Test strategy and test plan

• A test strategy 测试策略 is the high-level approach: which kinds of testing will be done, by whom, when, and what must pass before the next stage.
• A test plan 测试计划 is the detailed list of tests. For each: the purpose, the input data, the expected output, and a column for the actual output when the test is run.
• A test without an expected result is not a test. It only shows what the program did, not whether that was right.

Choosing test data

• Normal data 正常数据: typical valid values inside the range, which should be accepted and processed correctly.
• Abnormal data 异常数据: values that should be rejected, out of range or the wrong type.
• Extreme data 极端数据: the largest and smallest values still accepted, at the edges of the range. Boundary data 边界数据: the pairs that straddle each edge, the accepted extreme and the rejected value just outside it, where off-by-one errors hide.

Inside, outside, and right on the line

Worked example: test data for a mark from 0 to 100

Maintenance

• Most of a program's lifetime cost is spent after release. Corrective maintenance 纠正性维护 fixes faults found in use.
• Adaptive maintenance 适应性维护 keeps the program working in a changing environment: a new operating system, a new API, a change in the law.
• Perfective maintenance 完善性维护 improves a program that already works: faster performance, a new feature users asked for. A program may need all three throughout its life.

Fix it, keep it working, make it better

Amending an existing program

• Read the existing code until you understand the algorithm and the data flow. Find where the change belongs: which subroutine, which lines.
• Make the change as small as possible; do not rewrite working code. Update every related part: each caller of a changed parameter list, each routine that uses a changed data structure.
• Test the new behaviour and the old: regression testing 回归测试 checks that nothing that used to work has broken. Then document the change.

Marks that slip away

You've got it