This movie was prepared for the cd-rom publication of the ACM SIGGRAPH MultiMedia93 Conference, Anaheim, California, August 1993.
It is a demonstration of the system described in the paper presented at that conference:-
Open architecture multimedia documents.
The movie demonstrates KWrite being used to develop, publish and use an active document in which the diagrams are knowledge structures in the form of semantic networks. They can be edited from within the document and interrogated by other applications as the knowledge base of an expert system.
The document shown is an example of the actual application of KWrite used to publish an interactive multimedia document in the British Computer Expert Systems Society conference proceedings in 1992:-
which contains an embedded knowledge-based system to solve a room allocation problem as desscribed in the paper in the International Journal of Human-Computer Studies in 1994:-
A situated classification solution of a resource allocation task represented in a visual language
It was published as camera-ready copy by Cambridge University Press in December 1992 in the Proceedings of the British Computer Society Expert Systems Conference. It was also issued for ftp on internet and on CD-ROM as a parallel publication of an active document that could be used for solving room allocation problems.
As the abstract notes:
This paper is written in a document production tool that appears to a user as a word processor but also acts as an expert system shell with frame and rule representations supporting deductive inference. The electronic version of the document is active, providing typographic text and page layout facilities, versioning, hypermedia sound and movies, hypertext links, and knowledge structures represented in a visual language. It can be read as a hypermedia document and also interrogated as a knowledge-based system for problem-solving. The paper version of the document, which you are now reading, is produced by printing the electronic version. It loses its active functionality but continues to act as a record of the knowledge in the document. The overall technology has been developed as an alternative approach to the dissemination of knowledge bases. It also provides a different interface to knowledge-based systems that emulates document interfaces with which many users are already familiar.
KWrite is an open-architecture multimedia word processing and page layout document production and access system. The expert system application was chosen as a demonstration because it illustrates the computational use of the content of multimedia material as well as its audio-visual presentation.
The movie was intended not only to supplement the paper as a live demonstration of the research described, but also to illustrate the potential of Apple's QuickTime for supporting such demonstrations. We pushed the 1993 QuickTime implementation to its limits by adopting a large format, 512 by 342, that enabled us to show accurate screen dumps of KWrite in action.
Given that it plays for 8 minutes with continuous sound and large-screen color video, the size of the movie at some 11 Mbytes is reasonably compact. This is possible through the use of the animation compressor whose data format is well-suited to screen dumps. T
he primary tool used was Camerman from Vision Software, an excellent tool for live screen capture.
Draft sound overlays were added using CamerManEdit which allows one to add sound to a captured movie while it is playing.
This is a good technique for generating a spontaneous commentary, but not for getting smooth voicing and good sound quality. Hence, we transcribed the scripts generated this way into text and re-recorded them separately in a quiet room using a Casio DA-R100 DAT recorder.
The R100 is a lovely machine that slips in your pocket but offers two 16 bit 34Kbytes CD-ROM quality audio channels with 90db signal/noise ratio and no cross-talk between the stereo channels. We normally use this for movie capture to supplement a camcorder giving us three channels of sound capture in the field, with one mike on the camcorder and two on the R100. The peak-holding LCD level meters on the R100 span 80 db and enable one to see the room noise accurately as well as setting microphone levels to avoid clipping.
We put the sound into the Mac (a Quadra 900) through a dbx 1 sound compressor. This is usually necessary with camcorder input from the field where sound levels may fluctuate quite widely.
The dbx 1 with its extensive level controls is also a useful front end to the Mac since the Mac's line inputs are somewhat insensitive and result in unnecessarily low sound levels from standard (1V) line levels.
The compression was not all that important in this application because we recorded the scripts under controlled conditions. However, a dbx compressor is a valuable tool in attaining good sound quality in QuickTime digitization. With 8-bit sound it is worthwhile conditioning the signal on the audio side to be as good as possible before digitization. When 16-bit audio becomes common much of this conditioning can be done post-digitization.
We digitized the audio at 22KHz using Premiere's audio input, and reduced the rate to 11KHz in the compression phase without noticeable loss of quality. We used Premiere to synch up the audio and visual tracks, primarily by adjusting the speed of play of the movie segments.
There is an interesting stylistic problem in putting voice over animation as to whether the voice is the primary focus of attention and introduces the visuals -- in which case vision should lag sound -- or whether the movie is the primary focus, with the sound a commentary on the images -- in which case sound should lag vision. There is no universal rule and we tend to adjust until it feels right, with vision leading sound in most cases since this leads to the most rapid attention switches.
The synchronization problems of the current QuickTime implementation can override these subtle considerations, and it pays to try and detach the commentary somewhat from the visual changes. On slower machines with both QT1.5 and QT1.6 the sound can gradually come to lag several seconds behind the video.
The movie is in six segments: an introduction, an explanation of the KWrite document architecture, a demonstration of the knowledge elicitation tool KSS0/RepGrid, a demonstration of the knowledge editing tools KDraw, a demonstration of KWrite representing the knowledge embedded in a document, and a demonstration of a HyperCard stack being used to solve a room allocation problem by interrogating the knowledge base in the KWrite document.
The four demonstration sequences were captured by CamerMan. The two initial animation sequences were developed as sequences of PICTs in Canvas, loaded into Premiere and adjusted in length to synch with the commentary.
The six segments were pasted together in MoviePlayer and then compressed into a single fork movie in George Cossey's MovieShop. These two programs are issued by Apple as part of QuickTime distribution. We had some problems with MovieShop originally as it had not been used previously with movies with so much sound in relation to the video. George Cossey fixed these in MovieShop release 1.0.4 which is available on AppleLink.
It is important in using MovieShop to compress animations to set the similarity level at which it discards frames higher than for camera video so as to avoiding losing small changes that are significant to the animation. The default is 240, and we found 252 to be ok for the animations in this movie.
We transferred the movie to CD-ROM for testing purposes using a Philips CDD521 recorder and OMI QuickTOPiX software. It plays fine on a Mac IIvx from its internal 300 drive. It may have problems with a standard speed drive such as a 150. These problems seem to manifest differently with QT1.5 where the sound breaks up and QT1.6 where the sound severely lags the video. If you have problems playing from the CD, try playing direct from a hard drive. We pushed QT to its current limits but tried to keep the size small enough that it would be realistic to put the movie up on a hard drive.
The movie is not very glitzy -- effects eat up bandwidth at this size! However, we hope it demonstrates that a black-and-white, inactive, conventional paper can be brought to life by an associated movie. And also that production is neither complex nor expensive -- CamerMan alone at under $100 could be used to produce virtually all of this movie.
We should make the obvious disclaimer, that none of this is intended as a definitive guide to making such movies -- just an account of our experience. In particular, if we wanted to take the quality up another step, we would have used experienced commentators to read the scripts. Sound quality is very important in this kind of production. Technical quality is just one major prerequisite for an effective movie. Then one has to focus on what is said, and how it is said.
Brian Gaines and Mildred Shaw
Knowledge Science Institute, University of Calgary
{gaines, mildred}@cpsc.ucalgary.ca
This is a demonstration of KWrite, an open architecture, multimedia document system.
The application demonstrated shows the use of active documents in knowledge acquisition, representation and inference, with the document itself being the problem solving tool or expert system.
The objective of KWrite is to allow very high quality multimedia documents to be produced which can be issued in printed form for conventional reading, and also put out as active documents on the network and CD-ROM allowing direct user interaction with the embedded data structures.
The notion of a multimedia document component is extended in KWrite to include not only text and diagrams, pictures, movies and sound, but also data structures supporting knowledge structures in concept maps and semantic nets, and animation and simulation.
These components are combined through typography, text layout, page layout, and hypertext and hypermedia links and indexing, to produce the final multimedia, active document.
This is a demonstration of knowledge elicitation for KWrite documents using KSS0, our repertory grid tool.
The problem is one of room allocation.
A knowledge engineer is developing a knowledge base from examples of a domain expert actually allocating rooms.
The knowledge engineer has entered a list of people who have been allocated rooms, and has begun to develop relevant constructs distinguishing them.
The system feeds back the information that there is a close match between the characteristics of two people, and asks whether they can be distinguished.
The user may reduce the match by re-rating people or may decide to enter another construct.
KSS0 creates a new skeleton construct with the two people rated at opposite ends, and asks the user to name each end.
The user types in "non-smoking" for one end and "smoking" for the other.
The user then rates the remaining people in terms of this construct by dragging their names to one end or the other.
At any time during the elicitation the user may ask KSS0 to analyze the knowledge base so far created.
For example, the FOCUS cluster analysis shows which constructs are similar in terms of the way people are distinguished, and which people are similar in terms of the distinctions made.
Induction may also be run to create rules for inferring the position on one construct from logical combinations of the positions on others.
The constructs elicited and the rules based on them may be converted into a variety of knowledge representation formats and exported for loading into a knowledge base used to solve the room allocation problem.
The exported knowledge structures may be loaded into KDraw, a graphic editor for KL-ONE knowledge bases.
The knowledge engineer may edit them freely.
All the interaction is done graphically through the visual language, and the resultant semantic network may itself be compiled into a KL-ONE knowledge structure.
This is a paper written in KWrite that incorporates these knowledge structures.
The printed version appeared as camera-ready copy in the British Computer Society's Expert Systems Conference Proceedings published by Cambridge University Press in December 92.
Picture panes in the document may be dragged to the required position, and the text wraps itself around the new position.
The picture pane is open architecture and available for drawing to other applications, such as this QuickTime movie player.
Making picture panes available to other applications is one way in which KWrite provides open architecture functionality for multimedia applications.
For example this pict is displayed through a simple picture drawing application.
Animation and simulation applications may interact with the user from within the document through such panes.
For example, these pictures are knowledge structures which KDraw is drawing in panes within the document instead of separate windows.
Mouse actions within the panes are reported to Kdraw and the knowledge structures may be edited.
The KDraw editing controls pops up as a floating dialog when a node in the knowledge structure is double clicked for editing.
Thus the user has full KDraw editing capabilities from within KWrite.
Messages may also be sent from the active pictures back to the original applications.
For example, KDraw provides the capability to annotate its structures through popup menus that send messages to other applications such as HyperCard.
This annotation is also available automatically from within KWrite.
Knowledge structures in KWrite documents may also be interrogated by external applications.
For example, this HyperCard stack is being used to solve a room allocation problem by interrogating the KWrite document we have just examined using System 7 inter-application communication protocols.
When the user clicked on Solve, the HyperCard stack sent a message to KWrite to load the embedded knowledge structures into KRS, a KL-ONE knowledge representation server, run inference, and then query that server for advice.
The result of the query is that Thomas D., the group head, should be allocated either room 119 or 117, both central large rooms satisfying the rules expressed in the knowledge structures.
The user selects 117 and clicks on Allocate to send the information to the knowledge representation server and query for more advice.
Since the state of the knowledge base is changed by the allocation the next advice is different, to allocate one of the secretaries, Monika or Ulrike, to room 119, a large office near the group head as again specified by the rules.
The allocation, inference and advice sequence continues under user control until the problem has been solved.
Thus, the knowledge base in KWrite, represented in the KDraw diagrams, may be interrogated and used to drive an expert system.
If the diagrams within the document are changed the recommendations made by the system will also change.
Thus the open architecture capabilities of KWrite may be used not only to support conventional multimedia such as pictures and sound, but also more abstract multimedia such as visual languages for knowledge structures.
In this example the document system has become a knowledge engineering tool, and the document itself is the knowledge base of an expert system.