16. Testing and Debugging

As discussed in the Types of Problems section, once a program compiles cleanly, that is, without any errors or warnings, problems can still exist. The program could fail during execution. It could run without failing, but still not work correctly due to, for example, incorrect output. Once a program compiles, it must be tested and any problems corrected. Only then can it be considered finished.

Testing

Testing involves running a program to verify it does all it is supposed to do and does it correctly. For example, suppose a program is to add five numbers and display their sum and average. If the calculation for the sum only adds four of the five numbers, the program is not working correctly. If the program displays a correct sum but fails to calculate and display the average, it is not doing all it is expected to do. A program must be both correct and complete.

Desk Check Designs

One aid to ensuring a program will work correctly is to desk check its design. Once a design is finished, it should be stepped through to verify its logic is correct and it includes all intended functionality. An error corrected during design is one less error to fix during implementation.

Test All Paths

An important part of testing a program is to ensure all of its statements are executed. Each statement missed during testing is a potential source of problems when the program is made available to users. If a program only uses the sequence control structure, all of its statements will be executed when it is run. There is no possibility of missing any statements.

Most programs, however, use some combination of selection and/or repetition control structures. Testing every statement in these programs is not as simple as just running the programs. Planning must be done to ensure all statements are executed.

For example, suppose a program included the following statements:

double hours_worked;
double overtime_hours;

std::cout << "Enter hours worked: ";
std::cin >> hours_worked;

if (hours_worked > 40) {
    overtime_hours = hours_worked – 40;
} else {
    overtime_hours = 10;
}

If the program was tested by only entering values greater than forty, the assignment statement in the “false” part of the if statement would never be executed. When the program was given to the users, they would probably wonder why employees who worked forty or fewer hours were earning ten hours of overtime.

Here are a few guidelines for ensuring all statements in a program are tested:

if statements

Select data to test with (also known as test data) which causes the Boolean expression or compound condition to test true and test data which causes a false result to ensure the statements in both sections of the if statement are executed. When nested if statements are used, be sure to execute the code in every section.

do loops

Use test data which causes the Boolean expression or compound condition to test false, so no repetition occurs, and test data resulting in repetition. Remember, a do loop always executes once before the Boolean expression or compound condition is first evaluated.

while loops

Use test data which causes the Boolean expression or compound condition to test false immediately, so the code in the loop body never executes, data which causes the code in the loop to execute once but not repeat, and data which causes execution of the loop body to repeat.

Test All Variations of Boolean Expressions and Compound Conditions

Each of a program’s Boolean expressions and compound conditions can evaluate to true or false. When a program is tested, it is important to ensure these results happen when they should. If a compound condition is expected to have a false result for a specific value and its result is true, the program is not working correctly.

The test data selected to verify the correctness of a Boolean expression or compound condition should cover all variations. For example, suppose a program includes the following if statement:

if (number < 0 || number > 10) {
   result = 5;
} else {
   result = -5;
}

In order to thoroughly test the compound condition, the test data should include the following:

• a value less than zero,
• zero,
• a value between zero and ten (i.e., greater than zero but less than ten),
• ten, and
• a value greater than ten

In each case, the result should be the expected ±5 value. If, for example, result should be 5 when number’s value is 10, testing with a value of 10 would show that the compound condition is written incorrectly.

When a Boolean expression covers a range of values, be sure to use test data at the boundary of the range. For example, the Boolean expressions above check for values less than zero or greater than ten. Zero and ten are the values at the boundaries of the ranges being tested so they should be included in the test data. Testing with boundary values often catches problems with the inclusion, or omission, of equal signs. A test which tests a boundary value is sometimes called a boundary case or the testing of a boundary condition.

Choose Test Data Well

As can be seen from the discussions above, it is important to carefully select the data used during testing as the data chosen often determines how well a program is tested. Well-selected test data can result in a well-tested, and therefore high quality, program. Arbitrarily selecting test data can lead to a program with unidentified errors being given to users. It is always best to provide the highest possible quality programs to users.

One last point about test data: it is generally not a good idea to test only with the sample data provided in a programming assignment handout. Instructors do not usually provide a thorough set of test data because it is the students’ responsibility to ensure their programs are tested well.

Make No Assumptions About Program Modifications

One mistake occasionally made is to assume a small change to a program will not introduce any new problems. This is a mistake because even a small change can result in a program failing to compile. Always compile and test a program thoroughly after making any change to it, even when just adding comments.

Debugging

The problems found in a program during testing are called bugs or defects. The process of determining what to do to fix a problem and making the needed changes is called debugging.

Sometimes it is easy to identify and fix the statement in a program causing a problem. For example, perhaps a word displayed on the screen is spelled incorrectly or a value which should be displayed is not. At other times, however, it can take some work to determine the source of a problem and how to correct it.

Print Statements

Suppose a variable is assigned an invalid value at an unknown point during the execution of a program. Or perhaps the error message on the screen after a run-time error occurs is not enough to determine at what point the program failed. Before a fix to the program can be made, the line of code causing the problem must be located.

A common technique used to find an offending line of code is to place print statements in a program at key points. The value of a variable can be printed occasionally throughout the execution of the program. As soon as an invalid value is printed, it is known that the problem occurred somewhere between the current print statement and the one which last printed the variable’s correct value.

To locate the point at which a run-time error occurs, print statements could be placed in a program stating how far execution has progressed. For example, suppose a program has three functions, do_stuff(), do_more_stuff(), and do_last_stuff(), and the program is failing at some unidentified point. Print statements can be placed after each function call to indicate that each function has completed execution successfully:

do_stuff();
std::cout << "Finished do_stuff" << std::endl;

do_more_stuff();
std::cout << "Finished do_more_stuff" << std::endl;

do_last_stuff();
std::cout << "Finished do_last_stuff" << std::endl;

If, when the program is run, “Finished do_stuff” and “Finished do_more_stuff” print but “Finished do_last_stuff” does not, the problem is most likely in the do_last_stuff() function.

Make sure a newline (std::endl) is placed after the text in the print statements. If the newline is omitted, the text may not display on the screen although the error follows the print statement.

Print statements can be used at any time during testing to aid in finding the source of a variety of problems. Their flexibility, along with their effectiveness, make them a very popular debugging tool.

Keep in mind that print statements are used to identify problems so programmers can fix them. Be sure to remove print statements after they have served their purpose. Displaying stray data on the screen, from a user’s perspective, causes confusion and would perhaps make it appear that your program is working incorrectly.

More Testing

Once a correction has been made to a source code file, the program must be compiled and tested again. It is important, of course, to verify that the correction made to the program really did fix the problem identified. It is also important, however, to verify that the change made did not introduce any new problems. Programs can quickly become fairly complex and, although never intended nor desired, it is not uncommon for a fix for one problem to introduce another.

Testing and debugging is often a circular process: testing identifies a problem → debugging is performed to fix the problem → testing verifies the problem has been fixed → testing identifies another problem → debugging is performed to fix the problem → etc. Testing is complete, and a program finished, when testing does not identify any problems with a program.