Location: [CPSC 333] [Listing by Topic] [Listing by Date] [Previous Topic] [Next Topic] Loop Testing
This material was covered during lectures on March 31, 1997.
The goal of loop testing is to test while-do, repeat-until, (or do-while) and any other loops in a program thoroughly - by trying to ensure that each is executed at minimal, typical, and (if this is defined) maximal values - and to try to ``break'' the program, by trying to have a loop executed with a fewer than minimum, as well as a larger than maximal, number of iterations.
Loop testing is discussed, briefly, in recent editions of Pressman's Software Engineering: A Practitioner's Approach.
Three types of loops (or, more accurately, combinations of loops) will be tested.
Simple Loops are loops whose loop bodies contain no other loops (so, these are the ``innermost'' loops if any loops are nested).
Nested Loops are combinations of loops such that each is contained inside the loop body of the next.
Here is a sketch of three nested repeat-until loops:
Concatenated Loops are loops such that each follows the next in the code - that is, the execution of the next loop begins after the previous terminates.
Here is a sketch of three concatenated repeat-until loops:
The ``guidelines'' for loop testing consist of lists of test cases that should be included for each type of loop.
If there is an upper bound, n, on the number of times the loop body can be executed, then the following cases should also be applied.
Apply the following guidelines to ensure that each loop is tested at its boundaries (that is, with both minimal and maximal numbers of iterations of the loop body) while using a number of tests that is only linear in the number of loops:
If the loops are ``independent,'' so that the number of iterations used for one loop doesn't depend on the number of iterations used for any other(s), then it is sufficient to apply the guidelines for simple loops to each of the loops in the sequence.
On the other hand, if the number of iterations used for one loop does depend on the number of iterations used for another, then the following guidelines should be used instead.
Consider, again, the sorting program that is being used as an example for unit testing.
This program contains two nested while-do loops.
In this example, the maximum possible number of iterations of the inner loop (on any execution of that loop) is one less than the value of the variable i, since the integer variable j is initialized to have value i-1 before the inner loop begins, the variable j is decremented in the inner loop's loop body, and the loop test fails as soon as the value of j becomes less than or equal to zero.
The minimal possible number of iterations of the inner loop body is zero, since the loop test can fail immediately (and, it does so if A[i-1] is less than or equal to A[i] when this test is performed).
Finally, it should be noted that when the execution of the inner loop body begins, for a given value of i, the element in position A[i] of the array is the element that was in position when the program began execution, while the elements in positions A[1], A[2], ..., A[i-1] have been arranged (amongst themselves), so that they appear in the array in increasing order at this time.
If k is between 0 and ishould be) executed exactly ktimes at this point, if and only if A[i] is smaller than exactly k of the elements that appeared in the first i-1 positions of the input array: For, when the inner loop is executed for the given value of ``i'', the entries in the first i-1 positions will have been sorted. Therefore, the distance that the element (originally) in position i will need to be moved forward, in order to sort the first i entries of the array, is the same as the number of entries in the first i-1 positions than are smaller than this element - and this is the same as the number of iterations of the inner loop body used at this point.
Figuring out the number of executions of the outer loop body is much easier: This loop will always be executed exactly n-1 times, if n is the size of the input array.
Let's assume that this program is supposed to be able to sort arrays of length at least 100. Then both the inner and outer loop should be tested on inputs of ``moderate'' sizes (say, some size between 40 and 60), as well as on sizes 1, 2, 99, and 100. In order to keep the number of iterations of the inner loop at ``typical'' values while trying to control the number of iterations of the outer loop, it will probably be sufficient to start with an array with distinct elements whose entries are ordered in a random way. In order to keep the number of iterations of the inner loop at a ``minimal'' value, use an input array that is already sorted in increasing order. In order to keep the number of iterations of the inner loop at a maximal value, you should use an input array with distinct entries, that are initially sorted in decreasing order instead of increasing order.
If you examine the tests that were obtained for this program using path testing and condition testing, then you will find that none of the use input arrays whose length is more than 4, so none of them require execution of either the inner or outer loop body more than three times (at any time when the loop is executed). Worse yet, none of them require the inner loop body to be executed more than once, at any time when it is called. Thus, we'll add one more test (with the inner loop body executed twice when the inner loop is executed) in order to complete the testing of the loops at the minimal values, and we'll add tests to ensure that the loops are tested at both ``typical'' and ``maximal'' values, as well.
The test that will be added to ensure that the inner loop body is executed twice is as follows.
Test #8
Inputs: n = 3; A[1] = 3, A[2] = 2, A[1] = 1
Expected Outputs: A[1] = 1, A[2] = 2, A[3] = 3
Now, to complete ``simple loop testing'' for the inner loop, we'll try to develop tests causing the loop to executed 50 times, 98 times 99 times, and 100 times. In order do this while ``keeping the number of executions of the outer loop as small as possible,'' we'll use tests with arrays of length 51, 99, 100, and 101 respectively, such that the smallest element is at the end of the input array in each case. For the sake of a little more variety in the tests, different relative orderings will be chosen for the rest of the elements, in the four tests that are given.
Test #9
Inputs: n=51, A[i] = i for i between 1 and 50, and A[51] = 0
Expected Outputs: A[i] = [i-1] for i between 1 and 51
Test #10
Inputs: n=99,Expected Output:
- A[i] = 1 if i is between 1 and 98 and is even,
- A[i] = 2 if i is between 1 and 98 and is odd,
- A[99] = 0
- A[1] = 0,
- A[i] = 1 if i is between 2 and 50,
- A[i] = 2 if i is between 51 and 99
Test #11
Inputs: n = 100, A[i] = 101 - i for i between 1 and 100
Expected Outputs: A[i] = i for i between 1 and 100
Test #12
Inputs: n = 101,Expected Outputs:
- A[i] = 2 if i is between 1 and 50,
- A[i] = 1 if i is between 51 and 100,
- A[101] = 0
- A[1] = 0,
- A[i] = 1 if i is between 2 and 51,
- A[i] = 2 if i is between 52 and 101
Finally, in order to complete testing of the outer loop, tests should be designed in which the outer loop is executed (approximately) 50 times, 98 times, 99 times, and 100 times, while keeping the number of executions of the inner loop at ``typical'' values.
In order to keep behaviour of the inner loop ``typical,'' we'll simply choose ``random'' inputs. One way to do this, if the array is to have length n, would be choose the inputs A[i] uniformly and independently from the set of integers between 1 and n - assuming that the program really will be applied to ``randomly arranged'' arrays. These details should probably be the changed (so that they reflect the way the program will actually be used) if this in't the case.
You could do this to develop Tests #13, #14, #15, and #16, using the values 51, 99, 100, and 101 for n, respectively. After doing this, the ``inputs'' would have been determined; the ``expected'' outputs would be the same values, rearranged in increasing order.
The tests could then be written down, using (essentially) the same format as has been used for the first twelve tests that have been developed for this program.
A lab exercise based on this material is also available.
Location: [CPSC 333] [Listing by Topic] [Listing by Date] [Previous Topic] [Next Topic] Loop Testing