| home page - news - syllabus - schedule - assignments - tutorials - java - references - Mike Jacobson |
| Self-Study 1 - Self-Study 2 - Self-Study 3 - Self-Study 4 - Self-Study 5 - Self-Study 6 - Self-Study 7 |
| Dynamic Testing of Software |
In this course you will be expected to test the programs that you write and to submit information about testing on assignments. Indeed, markers might even be running your tests on our own — deliberately incorrect — programs, and considering the results (that is, checking to see how comprehensive your testing really was) when grading your work.
The tutorial exercise on software testing focused on test design and the performance of tests that do not require computer programming. The current exercise — which should be worked through during the third week of classes — continues this by considering “dynamic testing,” along with the programming techniques and software tools that can used for this.
This exercise also continues the second self-study exercise, inasmuch as it concerns the testing of programs that were considered in that exercise.
Finally, this exercise introduces a Java package, and is intended to give you a sense of the use of this to organize your software. It also includes an application of the Numbers classes as well as some information about exceptions.
The following problems test comprehension of and ability to apply material presented during the lecture(s) on software testing.
As mentioned above, this exercise builds on each of the two exercises that came before it. You should consider the material in those earlier exercises before spending time on this one.
In the fourth problem on the first self-study exercise, you were asked to modify your computing environment in order to ensure that Java class files were always created (and looked for) in a particular directory. Some of the information about the working of javac and java shown by this exercise will only be clear if you have already solved that previous problem, and modified your computing environment.
Consider the program Fibonacci1.java that was considered at the beginning of the previous self-study exercise. Suppose that — as part of our testing of the method that this provides — we wish to check that the method generates the ith Fibonacci number on input i, for each integer i between 0 and 10 (inclusive). Since the behaviour of a method on an invalid input is also important, we will also check the behaviour of this method on input −1.
It is possible to design and execute a suite of tests using only the tools that we already have.
With that noted, please download the program FibTest1.java and store this in the same directory as the source file Fibonacci1.java. Compile both programs (compiling Fibonacci1.java first) and then run FibTest1. to see that — at the least — the method provided by Fibonacci1.java can be used to generate the first few Fibonacci numbers.
Modify FibTest1.java to produce a test suite FibTest1a.java that can be used to test the second program, Fibonacci2.java, that was considered in the previous self-study exercise. (This should not be difficult.)
Now try to modify FibTest1.java once again, to produce a test suite FibTest1b.java that can be used to version Fibonacci4.java that looked for and reported numerical overflow conditions.
Don’t spend too much time on this — and don’t worry about it (for now) if things don’t turn out as you expect! We will return to this later on in this exercise, when you have a bit more information.
As mentioned in class, a testing framework can also be helpful.
Read the information about JUnit that is available on the course web site. Follow the instructions included there to make sure that you have access to this utility, and look at the information provided there about the programming and execution of test suites.
After you have done that, download and compile the program FibTest2.java.
Finally, execute the command
java org.junit.runner.JUnitCore FibTest2
If you have configured your computing environment correctly, then you should see evidence that the program Fibonacci1.java works correctly in some cases, once again.
Since you will be expected to use JUnit in this course, it is worth your time to correct any problems that you’ve noticed at this point in the exercise.
You may have noticed that things are getting a bit unwieldy, in that we are now dealing with multiple Java source files (and corresponding class files) that are related.
Henceforth we will use Java packages to keep our Java files better organized.
Create a new directory that you will use to store the following files. Download and examine them. They have been produced by modifying parts of the source file Fibonacci3.java that we considered during the previous self-study exercise.
SFibonacci.java — provides a simple Fibonacci number program that runs extremely slowly given integer inputs that are larger than 40.
Fibonacci.java — provides a more efficient Fibonacci number program
FibTest.java — provides a test suite that can be used with JUnit
FibList.java — uses the program provided by compiling the above source file Fibonacci.java to create a listing of the first few Fibonacci numbers.
Classes obtained by compiling the first three of the above source files will be part of a Java package called “Fibonacci1.” The class obtained by compiling the last of these files is not part of this package, but uses it instead.
Compile the above source files. This should not be a problem if you have made sure that you have access to JUnit.
It should be the case that three of the resulting class files are located in the same directory (not the same directory as the one containing the source files, if you have solved the various problems listed above), but the fourth class file is stored somewhere else. Why?
Execute the FibTest program that you have now compiled using JUnit. This time, you will need to do something a little different. Since this version of the FibTest program was declared (and compiled) as part of the Fibonacci1 package, you will need to execute the command
java org.junit.runner.JUnitCore Fibonacci1.FibTest
in order to ensure that this particular set of tests is executed.
In this problem we will consider, and resolve, the problems associated with testing the Fibonacci4 program from the previous exercise that you probably noticed above. Recall that this problem checked for numerical overflow and reported it by throwing a NumericalOverflowException.
If you were trying to include tests that involved this kind of overflow (by including cases where the expected outputs were quite large) then at least two difficulties had to be overcome:
For some test cases the values to be generated by the program being tested are too large to be represented as an int and — it seems — it is not as easy to work with these larger values.
One way to deal with this is to convert these number into “objects,” specifically, into objects of the class Long. This is one of the Numbers classes mentioned above. We can then use the methods provided by these objects to perform the operations that we need.
The NumericalOverflowException exception that we are now using is a different kind of exception than the IllegalArgumentException exception than we had been using before. As a result we are expected to identify those methods that are capable of throwing NumericalOverflowException exception — and the Java compiler objects when we fail to do so.
With that noted, please create another new directory and continue.
Download the following files into the directory that you have just created and compare these with the corresponding files that were considered in the previous question.
NumericalOverflowException.java — provides the NumericalOverflowException exception that will be used by the Java programs that follow
SFibonacci.java — provides a simple Fibonacci number program that runs extremely slowly given integer inputs that are larger than 40.
Fibonacci.java — provides a more efficient Fibonacci number program
FibTest.java — provides a test suite that can be used with JUnit
Compile and run these programs.
Modify the FibList program from the previous question so that it makes use of the Fibonacci2 package that has now been created instead of the Fibonacci1 package that it used before. Make sure that you can compile and run your program.
Finally, try to solve problem 1(c) once again. You should find this to be less surprising or frustrating, this time, if you have worked through this exercise carefully.
This might be a good time to review the online Javadoc documentation for JUnit. Note the variety of different kinds of assertions that be included in JUnit tests.
Can the previous version of FibTest.java be simplified by using JUnit more effectively? If so, how?
The test suite that was used in the third problem was not very good! If you worked through the tutorial exercise on software testing then you should know about additional test cases that should be added to make testing more comprehensive.
Modify the file FibTest.java from the third problem in order to include these additional tests.
Now, in order to “test your tests,” download and compile an alternative version of the Fibonacci.java program — and then run your tests.
The alternative version of Fibonacci.java is, indeed, incorrect. The bug that is (deliberately) included is somewhat obscure, but you will probably be able to figure out what the problem is if you designed your tests well, and it is a bug that you would plausibly introduce if you tried to make your program more efficient.
By all means, please do contact the instructor and ask about this, if you feel that you have made a reasonable attempt to discover the problem with this program but have not managed to find it.
Use the program Fibonacci5.java from the previous exercise to produce a package Fibonacci3 that can be used to compute larger Fibonacci numbers. Make sure to include yet another version of FibTest.java that can used, with JUnit, to test the Fibonacci number evaluator that this program provides!
Note: You might be wondering, by now, whether it is worth the effort to use a test framework like JUnit. I think it is, in part because we have barely scratched the surface here! If you review the information about JUnit that is available online then you will see that this utility has far more features (and supports far more interesting tests) than we have had time to cover here.
|
Last updated:
http://www.cpsc.ucalgary.ca/~jacobs/Courses/cpsc331/W17/tutorials/self-study3.html |