Self-Study 7, CPSC 331, Fall 2010

home page -  news -  syllabus -  schedule -  assignments -  tutorials -  tests -  java -  references -  Mike Jacobson


Self-Study 1 -  Self-Study 2 -  Self-Study 3 -  Self-Study 4 -  Self-Study 5 -  Self-Study 6 -  Self-Study 7

 Testing a Class

About This Exercise

This exercise can be completed at any time after the previous self-study exercise. While it is far from complete, it does provide some suggestions about how you might develop tests for a class, especially, one that implements an interface.

The Awful Truth: At this time there has not been very much that is written about object-oriented testing that is accessible and widely available, and I cannot guarantee that the suggestions that follow would necessarily be consistent with whatever literature on this subject is available.

Nevertheless, this should help to give you some ideas about how to design tests for classes that are required for assignments in this course.

Expected Background and Preparation

Students will likely find it to be easier to follow the instructions in this exercise if they have attended the lectures or worked through the online notes on testing. Previous self-study exercises on the use of the JUnit testing framework, the use a Java debugger and (as mentioned above) the implementation of ADTs and data structures in Java will also likely be helpful.

Class To Be Tested

Recall that the lecture on data structures, abstract data types, and their implementation made use of a “Simple Counter” example. The previous self-study exercise introduced the following Java programs, which specified requirements for this as an interface and which provided an implementation of it:

While a test file was also provided, this was extremely basic — far too limited to be considered to be sufficient as a test suite for this class.

In preparation for the rest of this exercise, please download and compile these programs if you have not already done so.

A Complication

Virtually any class (that is not simply a collection of static methods — which could presumably be tested independently, by methods we have already discussed) will access and modify data that can normally only be accessed through the methods we are testing.

As a result of this we will make heavy use of tests that use combinations of multiple methods provided by the class to be tested:

A Simplification

The guidelines for testing given so far suggest that an awful lot of extra code might need to be developed, for testing, since stubs and drivers for lots of other methods may be needed as each single method is unit-tested, and as progressively larger subsystems are considered during integration testing.

It might not seem to be necessary — or even sensible — to carry out any kind of extensive “unit testing” for some methods at all, even though the guidelines for testing, that we have seen, do suggest that this is necessary.

This is (I think) a reflection of the fact that most of the literature on testing concerns testing of the kinds of systems that were developed before object-oriented development was as heavily used as it is today. Consequently it assumes that methods are more complicated than they frequently are now.

Indeed, the example that follows will not include extensive testing of modifier methods that are each one-line long and that simply report the value of an instance variable — checking by hand for typographical errors (and making sure that the right instance variable’s value is reported) ought to be enough.

Unit Testing

getLimit and getValue

As explained above, we will not be including unit tests for either of the accessor methods, getLimit or getValue, that are specified as part of the Counter interface or implemented in as part of the SimpleCounter class — each of these methods is only one statement long (and each simply reports the value of an instance variable). Since these methods are so simple we will use them when testing other methods instead of writing stubs that would replace their use.

Constructor

The SimpleCounter class has a single constructor, which receives an integer value as input. If this value is positive then an instance of SimpleCounter is created whose limit is the given integer and whose value is zero; an InvalidInputException is thrown if the input integer is zero or negative.

It makes sense (and consistent with test design guidelnes) to test this constructor when its integer input has a variety of integer values, including zero or negative values that should cause an exception to be thrown. Since surprising things can happen when integer inputs are close to (or at) the maximum or minimum allowed values, tests that use these values can also be helpful.

A Java test class, CreationTest.java, which implements such tests, is now availalble. Please download and compile it and run its tests. If you have set up your Java environment as suggested in the first self-study exercise, then after compilation you should be able to run these tests by executing the command

java org.junit.runner.JUnitCore counters.CreationTest

One criticism of this set of tests is that some of the tests check two things at once! In particular, tests that supply a positive integer argument (and that should cause a new instance of the class to be created) check both that the limit of the new instance is the same as the integer input, and that the value of the new instance is 0.

While both these things certainly should be checked it can be argued that they should not both be checked as part of the same test — because that makes it harder to be sure about what went wrong if the test results in the discovery of an error.

Exercise: Modify the given test file, so that each test that should include the creation of a new instance (with a given positive integer limit) is replaced by a pair of tests — one checking the limit and the other checking the value of the instance that is created.

advanceValue

The SimpleCounter has a single mutator method, advanceValue. This increments the counter’s value, resetting it to zero and throwing a LimitReachedException if the value would otherwise be equal to the counter’s limit.

Since any reasonably comprehensible set of tests of this method should include tests in which the value of the counter has a variety of values (including 0, 1, some value between 0 and limit−1, and, finally, the maximal allowed value limit−1), the testing of this method raises a question whose answer is not, necessarily, straightforward: How do we bring the counter to the point at which we are ready to start the test — since the only method that can be used to increase the counter’s value to something greater than zero is the method we are currently testing?

There are at least two ways to answer this question (and to design tests):

  1. Add new code that can be used to change the counter’s value by other means. In this case this would, most likely, be a second constructor that accepts two integers (the first positive, and the second nonegative and less than the first) and that creates a counter whose limit is the first input and whose value is the second input.
  2. Make use of the methods that are already available, using advanceValue as many times as needed to set the counter’s value to whatever initial value is desired.

Each “solution” to this problem has a different disadvantage(s) (that the other solution avoids).

The first solution requires the addition of code to the class that is being tested — and this new code must be tested too. A proper test suite for the second constructor that has been described here should really be at least as extensive as the test suite for the first constructor that is provided above. Indeed, since the second constructor has two inputs instead of one, it is to be expected that a test suite for it might be more extensive.

The visibility of the new method must be considered carefully: It must clearly be accessible to the test methods that you are providing but should really not be more widely available than that, unless (and until) it is decided that the counter’s abilities should be extended (and the requirements for it changed).

Note, by the way, that you should not get rid of this new code after you have completed unit and integration testing! Regression testing might be needed (and, indeed, might be needed for a long, long time) after this, so that this extra code might be needed again long after you have moved on to other projects.

While the second solution avoids these problems it introduces its own: If an error does occur then it might be harder to determine the source of the error than it otherwise would: The error might be the result of the specific thing that you are are trying to check when you designed this test, or it might be due to something that happened as a result of one of the prior uses of advanceValue that were needed in order to “set up” the counter for this test.

In this case, I choose the second solution. adding additional checks at the end of the “setup” routine as well as code to print out a warning statement if it was discovered that the counter did not have the expected configuration after setup ended.

A java test class, AdvanceValueTest.java, which provides unit tests for the advanceValue method, is now available. Please download and compile this, and run its tests.

A few things should be noticed as you examine this test file.

Integration Tests

This particular class is so simple that there mot very much to do, or discuss, in order to carry out integration testing: Thorough “unit testing” of the advanceValue testing caused the entire class to exercised.

This will not be true in general. The testing of any class that provides multiple mutator methods will, almost certainly, require a process in which you think about how the different mutator methods might get used in combination, and develop even more tests as a result.

Conclusions and Suggested Exercises

You may have noticed that the testing code for this class was more complicated, and longer, than the code for the class itself!

This is (arguably) not usual — it was due in part to the fact that the class being used as an example was extremely simple. However it is not completely atypical either: Code needed for reasonably extensive testing will be more complicated than you might initially suspect.

Consequently, you should allow considerable time for the design and implemenetation of tests when completing assignments in this course!

Here are some suggested questions to think about, and practice exercises to complete.

  1. Create a new directory named counters and move copies of the following files (used in this self-study exercise) into it:

    • AdvanceValueTest.java
    • Counter.java
    • CreationTest.java
    • LimitReachedException.java
    • SimpleCounter.java

    Then execute the following command (inside this directory):

    javadoc AdvanceValueTest.java Counter.java CreationTest.java
            LimitReachedException.java SimpleCounter.java

    You should see that quite a few new files have been created inside of this directory as a result; one of these files should have then name index.html

    Open up this file in your web browser to see the kind of documentation for a package that javadoc can be used to create.

    Note: You can also create this documentaton by executing the following command inside the directory that contains your new counters directory:

    javadoc counters

    However, you might find now that many of the new files (including the file index.html that is of interest) is located in the directory where you executed the javadoc command, rather than your counters directory.

  2. Look for additional assertions (uses of either the assertEquals or assertTrue tests) that could be added, to the tests that have been included for advanceValue, that would make testing more complete!

    There probably are at least one or two of these!

  3. You may have noticed that the execution of the tests provided by AdvanceValueTest.java took a while! Can you explain why this is the case?

  4. If you have time for it, introduce the second constructor (allowing the value of the counter to be set to be a second input value) that is described above and modify the tests that have been given so that (where appropriate) they make use of it.

    What tests are needed to test this constructor adequately?

    How can you set the visibility of this method, to ensure that it can be used for testing without being available for use by methods outside of the counters package — and how must you change the test code, in order to allow testing to proceed?


Last updated:
http://www.cpsc.ucalgary.ca/~jacobs/cpsc331/F10/tutorials/self-study7.html