Location: [CPSC 333] [Listing by Topic] [Listing by Date] [Previous Topic] [Next Topic] Testing Objects
This material was covered during lectures on April 7, 1997.
Much of the material that has been presented on testing so far is most easily applied to ``function-oriented'' development. It seems highly likely that ``object-oriented'' testing will be increasingly important, and material on this is beginning to appear.
Some information on this topic (most of which is not presented here) can be found in the Fourth Edition of Pressman's Software Engineering: A Practitioner's Approach, in a new chapter on ``Object-Oriented Testing.'' For examples, Pressman mentions several black box methods for testing that aren't covered in these notes.
The book by Marick (which was also cited in the notes for integration testing) is also a useful source for material on this topic:
B. Marick,
The Craft of Software Testing. Subsystem Testing Including Object-Based and Object-Oriented Testing
Prentice Hall, 1995
Methods for object-oriented testing are not as well developed as methods for function-oriented testing, and these notes (clearly) reflect this. It's the hoped that the material presented below will be useful - but, also, that it will be improved.
Most of the material that has been presented so far applies to the testing of individual functions, or to subsystems that consist of a collections of functions. A ``test'' has generally consisted of the inputs to be supplied when a single call is made to the function (or subsystem) that is being tested, along with the outputs that are expected to be returned.
It has generally been assumed that these functions are deterministic, but also that they have no ``memory'' or ``state'' - ideally, if you call one of these functions at two different times, with the same inputs, then the same outputs should be returned both times.
Of course, this is not generally true for objects at all: Objects almost always encapsulate data or state information, which can be accessed or changed by using the object's interface (public) functions.
Therefore, when we consider objects (or classes), we'll consider a ``test'' on a single class or object to include
A test that involves several classes or objectmight need to specify the initial and final states of all the classes or objects involved.
In order to perform such a test, you may need to write one or more functions, to be added to (or to bypass) the interface for the object(s) involved in the test; these may be used to set the object's state to be what is specified for the beginning of the test, and also to inspect the object's state after the test is concluded.
A programming language is (roughly) considered to be ``object-based'' if it supports information hiding, so that one can use ``objects'' with interface functions providing access to data or state information (providing that the language prevents access to the data by other means), but the language doesn't support inheritance. For example, (early versions of) Ada is considered to be ``object-based.''
The information given below is appropriate for ``objects'' developed in this way, or using a truly ``object-oriented'' programming language. Additional testing is required if inheritance is used as well, and this will be briefly discussed in the next section.
Now, we'll consider ``unit tests'' to be tests that exercise a single class or object rather than a single function; these definitely might involve multiple calls, to more than one of the functions included in the class's (or object's) interface.
In order to produce a list of cases to be covered by tests, you should apply the methods for unit testing that have already been described for functions, to each of the functions in the object's interface. You may discover that it is necessary for the object to be in a specific (kind of) state in order for a specific path through an interface function's flow graph to be followed, etc., and you can use these requirements to decide what should be specified as the required ``initial state'' of the object, for the test. Once you've chosen the object's initial state, the interface function to be called, and the inputs that are to be supplied, the outputs that are expected and the object's expected final state can be determined from them.
Additional test cases can be obtained by considering the type and/or representation of the data or state information that the object encapsulates. A ``test catalog,'' such as the one contained in the appendix of Marick's book, may provide tests of this kind.
For example, the ``standard version'' of Marick's test catalog, given in Appendix B of his book, describes the following cases to consider, for a ``container'' - that is, an object that contain variable amounts of data:
Appending to a Container's Contents
This includes the case where several elements are being added at once. Some requirements are not applicable when adding elements one at a time.Overwriting a Container's Contents
- Container initially empty.
- Container initially not empty.
- Adding just as many elements as will fit.
- Full container, attempt to add one element.
- Partially full container, add one too many elements.
- Add just enough to leave room for exactly one element.
- Add zero new elements.
This includes the case where several elements are being added at once. Some requirements are not applicable when adding elements one at a time.
- New contents have 1 more elements than will fit.
- New contents just fit.
- New comments leave room for exactly one element.
- Zero elements added (is container correctly emptied?)
- Some elements added, but fewer than in original container (are old contents correctly cleared?)
Marick's catalog includes additional guidelines to test arrays, trees, etc.. As mentioned above, the catalogs themselves are included in appendices; the chapters in the main part of the book include more information about how the catalogs can be used.
Finally, it should be noted that the specification of a class or object may contain more than the specification of each interface function and the type definitions for object data and state information: It may also include a class invariant, which is a condition (or predicate) that is required to be satisfied on initialization of a class, but also when (a call to) each interface function terminates.
This ``class invariant'' may be useful when you're trying to decide what an acceptable ``final state'' of an object might be, for a ``unit test.'' It might also be useful to write additional code - for use in testing only - that explicitly checks that the class invariant is satisfied, just before termination of each function in the class's interface (including a constructor for the class). Writing (and disabling) such code has already been described (for assertion checking, as part of integration testing for ``function oriented'' systems).
As mentioned previously, system integration for a ``function oriented'' (or ``structured'') system can proceed in a top-down or bottom-up way, but this doesn't generally make much sense for an object-oriented system. The alternative, of clustering together components with logically related functions in order to produce useful ``subsystems,'' is likely to be more appropriate for object-oriented development.
It's been noted already that message threads should be identified during object-oriented analysis and used, in analysis and in design, to identify services that classes must provide. As part of ``integration testing,'' you should try to include tests that cause each of the ``message threads'' (that you've identified) to be followed.
Now, it was noted during the discussion of CRC modeling that many ``message threads'' for some kinds of processing of events (especially, ones involving error conditions and cancellations) are ``trivial'' - no messages are sent. While these could be ``ignored'' during CRC modeling, they shouldn't be ignore during testing - that is, you should include tests for these, too.
See Marick's book for a more detailed discussion of ``object-based'' testing.
As mentioned above, an ``object-oriented'' programming language is generally required to support inheritance, as well as information hiding.
A ``specialization'' class inherits the functions and attributes - and, therefore, the ``responsibilities'' - of its generalization, as well. Tests designed for use with a generalization should be performed on all the specializations, as well. Additional tests (for the specialization) should be created by considering the changes - additional classes and functions, and changes in implementation for functions - that have been introduced.
It was mentioned, during the discussion of integration testing for ``function oriented'' systems - that regression testing should be performed whenever a module is changed, during system integration and integration testing. Something similar must be done when changes are made during integration testing for object-oriented systems.
However, additional ``regression testing'' must be performed for object-oriented systems, whenever inheritance is used, as well: Whenever a ``generalization'' is changed, regression testing should be performed on the specializations of the generalization class that was changed - because the use of inheritance allows these specializations to ``bypass the interface'' of the generalization class that's been changed, and so that changes to the generalization class might also introduce changes to the specialization class's behaviour that are hard to predict. Regression testing should (probably) be also performed on all the other classes that bypass the changed function's interface (possibly, through the use of friend functions), as well.
See Marick (and Pressman)'s book for additional information about ``object-oriented'' testing.
A lab exercise based on this material is also available.
Location: [CPSC 333] [Listing by Topic] [Listing by Date] [Previous Topic] [Next Topic] Testing Objects