Brian R. Gaines
Knowledge Science Institute
University of Calgary
Alberta, Canada T2N 1N4
gaines@ucalgary.ca
Abstract: The aspirations and achievements of research and applications in knowledge-based systems are reviewed and placed in the context of the evolution of information technology, and our understanding of human expertise and knowledge processes. Future developments are seen as a continuation of a long-term process of operationalizing the rational stance to human knowledge processes adopted in the enlightenment, involving further diffusion of artificial intelligence technologies into mainstream computer applications, and incorporation of deeper models of human psychological and social processes.
This is the fifteenth year of the KAW meetings. On the cusp of a new millennium it is fitting to look back at what has been achieved and to look forward to the challenges and opportunities that await. There have been 30 KAW, EKAW and PKAW/JKAW/AKAW meetings prior to PKAW'2000 at which over 1,000 papers have been presented and published. Why did we start, what has been achieved, and have we satisfied the original aspirations?
One of the major areas of activity of the Knowledge Science Institute has been tracking the knowledge economy, in particular, modeling and forecasting the evolution of information technology. This has involved projects such as setting the Japanese fifth and sixth generation projects within a historic context (Gaines, 1984; Gaines, 1986), and modeling the convergence of computer and communications technologies in the information highway (Gaines, 1998). This article takes a similar approach to AI, ES and KA, analyzing the expectations and achievements, setting them within the general evolution of information technology, and concluding with an analysis of recent developments in the understanding of human expertise and knowledge processes.
John Boose and I founded the KAW series in 1986 at the peak of the artificial intelligence boom in the context of the industrial acceptance of an expert systems 'breakthrough.' IJCAI'85 in Los Angeles had attracted over 7,500 participants and had the atmosphere of a rock concert with thousands of participants avid to attend presentations in theatres that could seat 500 or less. The exhibition was like a major technology trade show with lavish stands demonstrating AI tools from innovative companies and tables sagging under the weight of a burgeoning AI literature. KAW'86 was intended to be a workshop on knowledge acquisition for some 40 specialists, but some 120 papers were submitted and we had over 400 requests to attend.
Those were heady days after the publicity for the Japanese 'fifth generation' project commencing in 1982 (Moto-oka, 1982; Gaines, 1984), with massive projections for the growth of revenues from the 'AI Industry' as shown in Figure 1.
| Market Area | 1981 | 1982 | 1983 | 1984 | 1985 | 1986 | 1987 | 1988 | 1989 | 1990 |
| Expert Systems | 4 | 9 | 17 | 38 | 74 | 145 | 245 | 385 | 570 | 810 |
| Natural Language | 5 | 8 | 18 | 40 | 59 | 125 | 210 | 320 | 465 | 650 |
| Visual Recognition | 10 | 22 | 51 | 116 | 168 | 260 | 370 | 500 | 660 | 840 |
| Voice Recognition | 5 | 7 | 11 | 20 | 33 | 55 | 85 | 140 | 200 | 270 |
| AI Languages | 3 | 5 | 8 | 12 | 21 | 35 | 45 | 65 | 80 | 105 |
| AI Computers | 28 | 56 | 103 | 217 | 364 | 510 | 710 | 970 | 1250 | 1570 |
| Government Contracts | 20 | 30 | 40 | 50 | 95 | 150 | 150 | 155 | 175 | 200 |
| Total | 55 | 107 | 208 | 443 | 719 | 1130 | 1665 | 2380 | 3225 | 4245 |
Figure 1 Projection of AI Market in 1985
The received wisdom of the early 1980's was captured by Hayes-Roth (1984) in a workshop on AI Applications for Business in May 1983:-
"For the past 15 years, applied work in artificial intelligence has focused increasingly on the use of knowledge to build 'expert systems.' These systems achieve levels of performance in complex tasks that equal or even exceed that of human experts. Because they incorporate much human knowledge, these systems are called knowledge-based expert systems or, simply, knowledge systemsÉThe industrialization of knowledge engineering began in 1981 with the formation of two commercial spin-offs from the Stanford university Heuristic Programming ProjectÉTeknowledge focuses on industrial and commercial uses of knowledge engineering. Sales this year will be $3 million to $6 million."
Hayes-Roth also characterized situations that instigate knowledge engineering initiatives:-
This positive stance to AI/ES applications in the 1980's was a major change from the 1970's when the initial optimism about major advances in, and applications for, artificial intelligence had been undermined by a series of negative reports by influential contributors to the field such as: Bar Hillel's (1964) on the possibility of machine translation; Pierce's (1969) on the possibility of speech recognition; and Weizenbaum's (1976) on the possibility of artificial intelligence. In addition there were highly critical appraisals from influential outsiders such Dreyfus (1972) and Lighthill (1973) with the report of the latter having had a highly negative impact on the funding of AI research world-wide (Fleck, 1982). As shown by the data above, in the mid-1980's there was a strong feeling based on industrial acceptance of expert systems that the critics had been proved wrong and that artificial intelligence research had been successful in creating a major new industry.
From the current perspective, some 15 years later, how have the expectations of AI/ES been fulfilled? The attendance at AAAI/IJCAI conferences has dwindled and the exhibit floors have virtually disappeared. The market projections for an AI industry in Figure 1 do not seem to have materialized. Teknowledge still exists with some 50 employees and revenue growth to $12M a year, which barely keeps pace with inflation. Neuron Data has become Blaze Software largely concerned with supplying technology for personalizing web sites. The expert systems shell FAQ at CMU (ftp://ftp.cs.cmu.edu/user/ai/pubs/faqs/expert/expert_1.faq) lists over 60 products but has not been updated since 1997 and, when one traces the companies listed today, most do not exist and those that do have generally migrated to the ecommerce industry.
Figure 2 characterizes the growth of the literature in AI and ES through to 1999 by plotting the number of books in the library catalog of a world-class university with a strong AI research area. The number of books with 'expert systems' in the title shows a standard sigmoidal learning curve (Crane, 1972), with the peak growth during the 1986 to 1992 period and publication waning thereafter. The number of books with 'artificial intelligence' in the title is still growing and it is difficult to accurately characterize the peak growth period but the data so far is consistent with that being from 1986 through to 1998.
Figure 2 Growth in number of AI and ES books held in a library
It might be reasonable to conclude from this that there was false euphoria in the 1980's and that critical appraisals from the previous decades had been correct. However, the story is by no means that simple and the following sections provide perspectives and examples that elucidate what has happened and provide a basis for predicting and planning future developments.
One answer to the conundrum is that, while the AI/ES industry may not have grown as much as expected, expert systems are still being developed and applied that do satisfy the original aspirations. Gensym was founded in 1986 and had revenues of $36M from a range of AI-related products and services including its G2 expert system shell. Its web site highlights some 25 corporate success stories in ES deployment (http://www.gensym.com/). The continuing health of the applications track at the British Computer Society annual conference on Expert Systems and of the Innovative Applications of AI track at the AAAI annual conference support this position.
More significantly, papers are being published in the professional literatures of the application areas that tell of the success of ES applications exactly as predicted by Hayes-Roth. For example, the April and July 2000 issues of InTech Magazine published by the ISA, the Instrumentation, Systems and Automation Society, has a two-part paper from Eli Lilly on the deployment of an expert system in its fermentation plant. The evaluation in 2000 is in wording that corresponds well to Hayes-Roth's predictions in 1984:-
"Within a few weeks, Phil was satisfied that the expert system reliably came to the same conclusions he would have by looking at the same data (i.e., the system did what it was purported to do, which was an application and validation objective). The expert system then took over this part of Phil's job, freeing up 40 hours per month of his time for other work. Of course, whenever G2 detected a problem fermentor, or one it was unsure of, Phil, or an assistant, would be immediately paged. This application became affectionately known as "Phil in a box." Phil retired from Lilly in 1993 when the company offered an early retirement program. In fact, many of the experienced fermentation personnel at this plant, as well as several at other Lilly plants, also retired." (Alford, Cairney, Higgs, Honsowetz, Huynh, Jines, Keates and Skelton, 2000)
There have also been major advances in the theoretical foundations of artificial intelligence, notably major improvements in the bounds on rational processes of deductive and inductive reasoning such as those originally formulated by Gödel (Davis, 1965), Chomsky (1956) and Gold (1967). The theory of computational complexity (Garey and Johnson, 1979) when applied to formal knowledge representation languages shows that inference in even moderately rich representations is inherently worst-case intractable (Nebel, 1990), and there is now a comprehensive taxonomy of representation capabilities and their complexity implications (Donini, Lenzerini, Nardi and Nutt, 1997). In machine learning, complexity measures have been at the heart of inductive algorithms from the early days (Blum and Blum, 1975) and the intractability of an exhaustive search approach to fitting a model to data is an intrinsic constraint for any reasonable class of models (Gaines, 1977). Algorithmic learning theory has become a well-founded discipline encompassing such results (Natarajan, 1991), and the major theoretical advances have been in formally defining and developing approximately correct modeling approaches that are tractable (Valiant, 1974), and in demonstrating how meaningful learning can take place through socio-cultural processes (Kirby, 1999).
However, a small but reasonably successful industry only captures part of the story. From the earliest days of AI pioneers such as Donald Michie have noted that an intrinsic feature of the field is that problems are posed such that all those involved accept that any solution must involve 'artificial intelligence' but, when the solution is developed and the basis for it is clear, the resultant technology is assimilated into standard information processing and no longer regarded as 'intelligent' in any deep sense. When the magician shows you how the trick was done the 'magic' vanishes. Much of what has been developed through AI/ES research has diffused in this way into routine information technology, the Michie effect.
One example of the Michie effect is the assimilation of expert systems technology into mainstream database technologies. Blaze Software supports the 'business rules' layer in the IBM/Microsoft three-layer client server enterprise model through use of the powerful knowledge modeling tools that were developed for the expert system shell NEXPERT. Teknowledge's patents relating to such applications are being contested in a lawsuit by SAP, the world's third-largest independent software supplier with revenues of over $5B/year employing over 21,700 people in more than 50 countries in which SAP denies violating Teknowledge patents.
There are many books, manuals and white papers now available on business rules and their development. Date, the author of the standard text on relational databases, has one entitled What Not How: The Business Rules Approach to Application Development (Date, 2000). Seiler, the founder of Rule Machines Corporation, has a nice paper on managing business rules which shows their role within an enterprise architecture (Figure 3) and emphasizes that they are not expert systems or database triggers but rather a way in which end-user management can specify activities in terms of "business speak" (Seiler, 1999). In KA terms, the business rules are intended to support knowledge modeling by end-users, a major objective of one line of research at the KAW workshops.
Figure 3 Business rules within n-tier application architecture (Seiler, 1999)
The middle layer in Figure 3 can range from the operationalization of procedures manuals, internal to the company or external such as the tax or building codes, to the incorporation of sales, marketing and financial expertise that is not normally captured in procedures or training manuals. The back-end databases are usually pre-existing relational systems and the client user interface increasingly uses web browsers with HTML as the GUI programming language. The middle layer allows rich ontological models to be incorporated in terms comprehensible to managerial end users such that they can incorporate procedures based on their knowledge and requirements with the minimal of mediation by programmers. An early experiment in encoding an oil company's procedure manual in this way was reported by Kremer (1991) at KAW'91, and noted that the use of rules with exceptions was the most natural way of encoding the constraints in the manual.
A related example of the Michie effect is the ongoing assimilation of AI concepts and frameworks into the mainstream data processing industry in the work of the IEEE Standard Upper Ontology (SUO) study group developing a standard for high-level database integration (http://ltsc.ieee.org/suo/) which draws heavily on the people and research of the KIF and CG communities. The business rules and standard ontology technologies can all be seen as the development of support for knowledge management within organizations, the "management of organizational knowledge for creating business value and competitive advantage" (Tiwana, 2000). The primary Japanese literature on knowledge management emphasizes the knowledge acquisition processes involved in converting 'tacit knowledge' into overt operational knowledge (Nonaka and Takeuchi, 1995; Von Krogh, Ichijo and Nonaka, 2000). The issues of supporting such conversion are strongly reminiscent of those of developing expert systems, and knowledge management web sites link into the KA literature (e.g., http://www.km-forum.org/papers.htm).
Another instance of the Michie effect has been the adoption of rule induction techniques in the scientific community to analyze databases with the results significant for, and reported in, the relevant scientific literature, for example, in research on the carcinogenetic properties of chemical compounds (Lee, Buchanan and Rosenkrantz, 1996). Langley (2000) provides a wide range of examples such AI computational support of scientific discovery. KA tools have also proved useful in helping a research community develop a consensual and comprehensible framework for its research program (Gaines and Shaw, 1994).
A different area of assimilation of AI techniques into mainstream data processing is the routine use of neural networks in conjunction with statistical techniques to model complex datasets. For example, neural networks are being used routinely in geography to develop nonlinear models of ecological (Lek and GuŽgan, 2000) and climatic data (Smolka and Volkheimer, 2000).
Knowledge discovery from databases (KDD, Fayyad, 1996) has clear roots in machine learning, but combines statistical tools, ontology and rule induction with graphic human interaction to provide a new hybrid technology subsuming and merging the other techniques within its own conceptual framework. As KDD techniques becomes clearly defined and classified they will in turn merge with on-line analytical processing (OLAP, Hackney, 1997) techniques for extracting management information from data warehouses, and their AI roots will be primarily of historic interest. The Michie effect is pervasive and inevitable, but is a sign of achievement not failure.
In projecting the future for artificial intelligence research it is also important to recognize that parallel advances in information technology have provided alternative solutions to some aspects of what had been regarded as 'AI problems.' For example, Hayes-Roth's list in Section 2.1 emphasizes the role of expert systems "when organization requires more skilled people than it can recruit or retain," and a classical approach to such labor shortages is through training. E-learning has also developed extensively during the same period as expert systems and there is now a major industry supporting 'corporate universities' (Meister, 1998), and providing on-the-job training and just-in-time learning (Wills, 1998). For example, the Learn4life division of SAIC, a $10B/year company, provides modules targeted on the full range of emergency services, law enforcement, fire service and search and rescue (http://www.Train4life.com/), and Motorola University offers courses in a wide range of core skills areas where recruitment is problematic such as software engineering (http://mu.motorola.com/).
Pace Bar-Hillel, automatic translation is a freely available service on the web. Typing Wiggenstein's famous aphorism:-
"Wovon man nicht sprechen kann, darüber muß man schweigen"into Altavista (http://babelfish.altavista.digital.com/translate.dyn), one gets back:-
"about which one cannot speak, over it one must be silent"which captures the essence quite nicely. Pace Pierce, speech recognition has also become a routine office product from major corporations such as IBM and Fujitsu, again without significant relations to AI developments.
Some of the most dramatic examples of 'machine intelligence' in recent years, arousing massive public interest, have been the Kasparov versus Deep Blue chess games. In 1996, Kasparov won the series but it was clear that the computer program was playing chess effectively at grandmaster level (Newborn, 1997). In the 1997 re-match Deep Blue won the series and, as Schaeffer and Platt (1997) note in regard to game 2:
"If a game such as this were ever used for a Turing Test, few would peg the computer as playing White. In fact, most grandmasters would have been thrilled to have played such a nice a game as White, regardless of who was playing the Black pieces."
Chess playing has been regarded as a benchmark 'AI problem' but the number-crunching search strategy of Deep Blue based on special chess-oriented hardware was not an AI or ES approach, and provides little insight into human chess-playing strategies.
A major advance in information technology that was not even on the horizon at KAW'86 was the development of the World Wide Web. Berners-Lee's (1989) proposal to CERN for managing its documents effectively was still three years away. His first paper about the web was relegated to a poster at Hypertext'93, and it was not until the mid-90's when Andreessen had developed what became the Mosaic browser and eventually Netscape and Internet Explorer that the web exploded into a ubiquitous and revolutionary technology. The web is important not only because it diverted effort from AI activities to communication technologies, but also because it provided alternative solutions to the problem of accessing expertise. The significance of discourse in the human communities collaborating through the net has been underestimated in the stress on 'artificial' intelligence in computer research. Net email and web services provide access to a far more powerful 'expert system' of human agents and their products than any currently conceivable through AI techniques.
Structured search strategies of digitally represented scientific literature have also been used in the automated development of new scientific discoveries in a way that addresses an AI problem without using AI techniques. For example, Swanson (1990) has reported on the success of a methodology that searches for implications of the form A implies B, and B implies C, in two papers from different literatures neither of which generally cites the other. The connection that A implies C has been used to derive significant new results in some medical areas.
Developing search engines for the web has involved the use of text analysis techniques that draw primarily on information retrieval technologies rather than AI but result, in their latest versions such as Google (http://www.google.com/), in such precise access to a massive corpus of knowledge that they should certainly count as an advance in knowledge acquisition techniques.
Web browsers have, as in many other application areas, also provided a convenient interface to AI, ES and KA applications using HTML to program their user interfaces in a standard, and platform-independent, manner. Ontology editors for a range of KR and KA systems have been made available through the web, for example, Ontolingua (Farquhar, Fikes and Rice, 1996), Protégé-II (Rothenfluh, Gennari, Eriksson, Puerta, Tu and Musen, 1996), VITAL (Motta, Stutt, Zdrahal, O'Hara and Shadbolt, 1996), and others, as have personal construct psychology approaches such as WebGrid-II (Gaines and Shaw, 1997).
There is also an interesting convergence between web and AI techniques in the W3 'Semantic Web' framework and its implementation using the Resource Description Framework (RDF). As Tim Berners-Lee notes:
The Web was designed as an information space, with the goal that it should be useful not only for human-human communication, but also that machines would be able to participate and help. One of the major obstacles to this has been the fact that most information on the Web is designed for human consumption, and even if it was derived from a database with well defined meanings (in at least some terms) for its columns, that the structure of the data is not evident to a robot browsing the web. Leaving aside the artificial intelligence problem of training machines to behave like people, the Semantic Web approach instead develops languages for expressing information in a machine processable form. (http://www.w3.org/DesignIssues/Semantic.html)
The KA community has established an international working group to develop technologies for the semantic web (http://www.semanticweb.org/), and launched a Semantic Web journal in the Electronic Transactions on Artificial Intelligence (ETAI, http://www.etaij.org/seweb/) series.
Information technology based on the stored-program digital computer has seen a rate of growth in the past fifty years that is unsurpassed by any other technology. The vacuum-tube based flip-flop memory cell enabled the development of the first generation of computers in the 1947-49 period. Reliability and performance were increased with the advent of solid-state transistors in 1959, and the number of devices on a chip increasingly exponentially since then to some billion currently has induced a similar improvement in computer performance. However, electronic devices and computers could not have been developed over nine orders of magnitude performance improvement without the use of computers themselves to support the design and fabrication of circuits and computers. This is one example of a positive feedback loop within the evolution of computers through which the computer industry has achieved a learning curve that is unique in its sustained exponential growth. Each advance in computer technology has supported further advances in computer technology.
Such positive feedback is known to give rise to emergent developments in biology (Ulanowicz, 1991) whereby systems exhibit major new phenomena in their behavior. The history of computing shows the emergence of major new industries concerned with activities that depend upon, and support, the basic circuit development but which are qualitatively different in their conceptual frameworks and applications impacts from that development. For example, programming has led to a software industry, human-computer interaction has led to an interactive applications industry, document representation has led to a desktop publishing industry, and so on.
Each of these emergent areas of computing has had its own learning curve (Linstone and Sahal, 1976), and the growth of information systems technology overall may be seen as the cumulative impact of a tiered succession of learning curves, each triggered by advances at lower levels and each supporting further advances at lower levels and the eventual triggering of new advances at higher levels (Gaines, 1991b). It has also been noted in many disciplines that the qualitative phenomena during the growth of the learning curve vary from stage to stage (Crane, 1972; De Mey, 1982; Gaines and Shaw, 1986).
The era before the learning curve takes off, when too little
is known for planned progress, is that of the inventor having very little
chance of success but continuing a search based on intuition and faith. Sooner
or later some inventor makes a breakthrough
and very rapidly his or her work is replicated at research institutions world wide. The experience
gained in this way leads to empirical design rules with little foundation except previous successes and
failures. However, as enough empirical experience is gained it becomes possible
to model the basis of success and failure and develop theories. This transition from empiricism to theory
corresponds to the maximum slope of the logistic learning curve. The
theoretical models make it possible to automate data gathering, analysis and associated
manufacturing processes. Once automaton has been put in place, effort can focus
on cost reduction and quality improvements in what has become a mature technology.
The dependent technologies themselves develop along their
own learning curves and come to support their own dependents. Figure 4 shows a
tiered succession of learning curves for information technologies in which a
breakthrough in one technology is triggered by a supporting technology as it
moves from its research to its empirical stage. Also shown are trajectories
indicating the eras of invention, research, product innovation, long-life product lines, low-cost products, and throw-away products for different forms of information technology.
Figure 4 The infrastructure of information technology
The breakthrough in digital electronics leading to the zeroth generation is placed at 1940
about the time of the Atanasoff and Berry experiments with tube-based digital
calculations. The first breakthrough generating a computing infrastructure was
Mauchly's introduction of the general-purpose stored program computer
architecture which led to the transition
from the ENIAC to the EDVAC designs. The next level of breakthrough was in software to bridge the gap between machine and task through
the development of problem-orientated languages. The next level of breakthrough
was in continuous interaction
becoming a significant possibility as the mean time between failures of
computers began to be hours rather than minutes in the early 1960s. These lower
levels of electronics, computer architecture, software and human-computer and
computer-computer interaction define the domain of classical computer
science.
The four learning curves of the tier at the top of Figure 4, of knowledge
representation, acquisition, autonomous agents and socially structured systems constitute the domain of knowledge science where the convergence between artificial
intelligence, expert systems, knowledge acquisition, databases, the web, and so
on, is situated. From an AI perspective, the knowledge level breakthrough
corresponds to the development of DENDRAL (Buchanan,
Duffield and Robertson, 1971)
for inferring chemical structures from mass-spectrometry data and MYCIN (Shortliffe,
1976)
for the diagnosis of microbial infections in the early 1970s. However, it is
important to note that the knowledge level also encompasses the digitization of
the classical knowledge representation media through which typographic text,
diagrams, pictures, sounds and videos became storable, indexable and
retrievable through digital computers. Thus the breakthroughs in the 1970's
represented by the introduction of raster graphics, word-processing software,
MEDLINE, SGML and PostScript, are also critical events for the knowledge level
learning curve.
Similarly, at the acquisition level, the AI breakthroughs may be
seen as AM learning mathematics by discovery (Davis
and Lenat, 1982)
and the successful inductive inference of expert rules for plant disease
diagnosis (Michalski and Chilausky, 1980).
However, developments in scanning, optical character recognition, interactive
graphics and page makeup systems were also significant advances in the
digitization of knowledge in machine processable form. At all levels, research
in robotics and machine vision has been a major source of innovation and a
driving force for technologies at the upper levels involving some degree of
autonomous behavior and social organization.
Figure 4 provides a context within which to model the
assimilation of AI and ES technologies into standard information processing as
discussed in the preceding sections. The deductive and inductive inferences
processes that are seen as core to human rational intelligence, when modeled in
the computer, become data processing capabilities that can be understood as
such and used as computational resources where appropriate in any application.
Similarly, the peripheral perceptual processes when modeled effectively become
statistical pattern-recognition techniques which can again be assimilated as
computational resources. The representation of knowledge at a semantic level
through rich ontological structures is a natural extension of data base
technology and has become assimilated as such. In particular, to the extent
that the knowledge representation is natural and comprehensible to people, it
becomes assimilated as part of the upper level human-computer interface where
the objective is to make the programming and use of computers natural and
comprehensible to people.
From this perspective, what could not be assimilated so readily
would be systems that achieved intelligent behavior in incomprehensible ways.
For example, if the various experiments in electro-chemical perceptron-like
elements in the 1960's had produced effective intelligent systems they might
not have been so readily assimilated except as black-box peripherals. However,
the history of AI research to date has been one of achieving successful
performance at some task by some means, and then afterwards deconstructing that
achievement to rationalize it in algorithmic form. Magic has always been
transformed into science. One can see this process at work in research on
quantum computing where the underlying mechanism is radically different from
that of current digital computers but where science-based engineering design is
being used to develop fresh approaches to the massive search tasks whose
computational complexity undermines current AI algorithms (Grover,
1996).
It is the advances in the understanding of inference algorithms in relation to
knowledge representation schema noted in Section 2.3 that make it possible to
contemplate using such alternative approaches. Both deductive and inductive
inference have become precise computational sciences.
What are the implications of this for the next generation of
AI/ES/KA developments? One can see from Figure 4 that the line of throw-away
products now encompasses the entire arena of classical computing. High-quality
computer hardware, compilers, development environments, and interactive
interfaces, are all now ubiquitous consumer products in the developed nations.
Access to raw knowledge through the web comes at low-cost ranging from the
willingness to tolerate advertising to a few hundred dollars a year for professional
journals. Knowledge acquisition tools through professional services such as
DIALOG are more expensive, but potentially low-cost as techniques such as those
used in Google are applied to electronic versions of journals. Autonomous
agents are proving their practical worth in robotics (Shen
and Norrie, 1998),
and research on social structures of agents is changing our theories of
knowledge processes (Kirby, 1999).
In all these areas the integration of mature AI technologies
such as ontology and ripple-down rules editors and inference engines can be
applied to provide improved performance embodying human knowledge and
expertise. For example, the selective dissemination of information (SDI) has
become critical as the volume of available digitized information has increased
beyond the bounds of individual utility. Current methods based on keyword
searches are crude in their selectivity and difficult to customize effectively.
The selection of an appropriate ontology from a library and its development
through an individualized sub-ontology incorporating rules with exceptions to
manage the retrieval process could be the basis of a next generation of much
more effective SDI systems that are also active awareness agents drawing attention to emerging information and
trends.
Such developments would be consistent with the notion of knowledge
support systems introduced at KAW'87 as a
framework for integrating ES, KA and multimedia knowledge sources (Shaw
and Gaines, 1987).
This was extended at KAW'89 to encompass a wide range of knowledge support
systems shown in Figure 5, a diagram which establishes reasonable targets still
valid today for the assimilation of a variety of information technologies,
including AI, ES and KA, into highly interactive computational systems that
amplify human expertise. It provides the content and human dimension to current
developments of distributed grid architectures (Foster
and Kesselman, 1999).
Figure 5 Computer-based knowledge support processes (Gaines, 1990)
It is interesting to go back even further in time to Shortliffe
and Clancey's (1984) list of desiderata in the early 1980's for the second decade of ES research.
Users surveyed said the systems should:
and system developers said that research should focus on:
We are now entering the fourth decade of ES development, but
this list is as valid today as it was nearly 20 years ago.
Returning to Figure 4, the line of invention leaves the existing
framework, giving no indication of the areas in which breakthroughs in the
current era might be expected. I have considered a third major level, of memetics, reflecting the autonomy of ideas within Popper's
World 3 (Gaines, 1978),
but do not yet have confidence in projections at that level.
The most effective technological forecasting techniques are those
that identify a social need and analyze the state-of-the-art in the technical
pre-conditions for it to be satisfied (Gilfillan,
1937).
Most of our social needs today stem from the continued environmental impact of
exponentially increasing population and the ensuing problems of famine, disease
and social unrest (Meadows, Meadows and Randers, 1992).
Alain Rappaport (personal communication) has drawn my attention to the
migration of AI scientists into genetic engineering projects, and that is
obviously one area focused on addressing current needs related to health and
food. Genetic technology has a tiered structure of learning curves of its own
commencing with the breakthrough in molecular biology through Watson and
Crick's discovery of the double helix model of DNA in 1953 (Gaines
and Shaw, 1986).
We would expect convergence of computing and genetic technology on the basis of
their common foundations in information encoding whether in silicon or DNA.
Perhaps the most significant conclusion to draw from Figure
4, however, is that knowledge representation and acquisition, conceived as
digital computer technologies, are in the late stages of their learning curves.
This may be surprising because, if one looks back to the aspirations of expert
systems research in the 1980's, there is still a major gap in information
technologies despite the assimilation of AI and ES techniques, and that is in
the emulation of human expertise and its transfer from human experts to the
computer. It is not that there has been no progress. The examples in Section
2.3 and the ongoing application of, for example, ripple-down rule techniques to
building effective expert systems demonstrate that the emulation and transfer of
human expertise is feasible in some domains (Compton,
Edwards, Kang, Lazarus, Malor, Preston and Srinivasan, 1992).
However, the large-scale emulation and transfer that fired the industrial
enthusiasm of the 1980's has failed to materialize. The next section provides a
framework for understanding the constraints to achieving such emulation and
transfer of human expertise within existing computing frameworks.
When we moved to Canada in 1982 one of my first tasks was to
return to the UK to act as the neutral chair of a Science Councils workshop
considering funding of UK expert systems research in the light of the Japanese
fifth generation initiative. My recollection of that meeting is of eminent
cognitive psychologists explaining to enthusiastic computer scientists why
modeling human expertise was unlikely to be effective or useful. Dreyfus and
Dreyfus (1986)
have presented the arguments very clearly, and the KAW meetings have from the
beginning had cognitive psychology tracks addressing the fundamental issues. In
particular, Bill Clancey (1997)
has through the KAW meetings and a wide range of publications deconstructed
simplistic notions of the nature and transferability of human expertise with
the credibility of a major pioneering contributor to expert systems
development. What have we come to know of expertise, its computer emulation and
transfer?
Webster's dictionary definition of expert as a noun is:
and as an adjective is:
These definitions capture some significant connotations of
expertise and it is useful to deconstruct them carefully.
First the use of the terms "has" and "possessing" gives
skill and knowledge connotations of a substance that may be possessed. This
association of expertise with substance can lead to a perspective that sees
that substance as something to be transferred to a computer. It may also given
the impression that to possess that substance is to be an expert.
The first association is misleading in the sense that in
many cases the only evidence one has for possession of something is that an
expert is capable of skilled performance in a task. One may reason that there
must be some basis for this performance, and it is a possible metaphor to view
this as possession of a substance. However, the 'substance' is an imputed
hidden variable and hypothesizing its existence gives little insight into the
nature of expertise. The metaphor may also be misleading in locating expertise
within the expert rather than as a process of interaction between expert and
situation.
The association of skill and knowledge in both definitions
is part of this metaphor in implying that knowledge is the substance underlying
skill. Skill is defined as both:
and as
The problem of relating these two definitions of skill, the
first causal and the second phenomenological, involves major ontological,
epistemological and psychological issues.
Knowledge is defined as:
What are facts, truths and principles and how does
acquaintance with them lead to competent excellence in performance? Does
skilled behavior indicate the possession of knowledge?
The impression that the possession of skill is adequate to capture the normal usage of
the term expert is also misleading. One would term someone skilled who can
perform a task well, but to term someone expert has connotations going beyond
mere skill, of being able to perform well in difficult situations, of
maintaining the performance in changing, unexpected and novel circumstances.
These are the connotations which Schön (1983)
emphasizes in his discussion of "reflective practitioners" who do not attempt
to merely preserve their existing capabilities but to extend them continually
in order to match changing circumstances.
The auxiliary terms in the definitions are interesting in
suggesting other aspects of expertise. It is specialist, not a general attribute like intelligence, and
hence can be seen as a situated role that a person can play rather than a
general property of that person. Its being associated with authority suggests that it plays a social role in that others
must allow an expert:
Its association with being trained by practice indicates one, but only one, of the many processes
whereby expertise is acquired.
The problems introduced by attempting to model human action
as derived from knowledge have been extensively discussed in the literatures of
philosophy and sociology. Gadamer, in his critique of Hegel's theory of knowledge,
highlights the fundamental issues underlying the relation of expertise to
knowledge:
However, Gadamer argues:
In the expert systems literature, Clancey has criticized
approaches to expert system development based the assumption that expertise can
be captured in overt knowledge, and comes to similar conclusions:
He argues that overt representations of knowledge are only
partial models of the knowledge processes underlying human behavior:
The nature of human capabilities and knowledge have been a major
topic studied by philosophers from the earliest times, and it is not surprising
that artificial intelligence research has not resolved their nature in its
comparatively short history. Indeed, any fundamental resolution would be highly
unlikely, and any pragmatic technological resolution would be expected to have
limited application. However, the issues and aspirations will not, and should
not, go away. Minimally, the computer is a powerful tool for operationalizing a
theory, allowing us to simulate its application and consequences, and at the
same time testing whether the theory is sufficiently clearly expressed to have
well-defined applications and consequences.
Much of the current thought on the nature of expertise and
knowledge can be seen as stimulated by the later works of Wittgenstein, in
particular, his arguments that the notion of human behavior "following a rule"
is paradoxical:
Given that the majority of expert systems technology
attempts to emulate human expertise through representation as rules, and that
the majority of knowledge acquisition methodologies are concerned to derive
those rules from human behavior, one would expect that attempts to model human
behavior that address Wittgenstein's arguments might be particularly relevant
to AI/ES. Pierre Bourdieu, the French philosopher and sociologist, has
generated a major literature on human psychology, culture and sociology, that
stemmed from just this consideration:
The answer to this question from a wide variety of sources
is that all human behavior is generated within a rich background, to use Searle's (1992)
terminology, that is implicit and not consciously represented, and is
constituted through acculturation processes that internalize the historic
development of a particular society or institution.
Bourdieu builds on the previous analyses of Aristotle,
Hegel, Nietzsche, Husserl, Schutz, Wiggenstein, Heidegger and Merleau-Ponty, to
provide a very detailed analysis of socially-embedded human behavior in terms
of three major constructs: habitus which
is a system of dispositions extending Aristotle's analysis of hexis; field
which is a network of influences and power relations extending Lewin's analysis
of behavior within a social field; and symbolic capital abstracting and generalizing Marx's analysis of
capital formation and Weber's extension of it to cultural domains. Bourdieu's
output in books and papers is prolific, ranging from detailed ethnographic and
statistical studies through sociological models of a wide range of institutions
to deep theoretical analyses--a good starting point is the interviews and essay
in Bourdieu (1990).
Bourdieu's model of habitus is particularly important to the
modeling of human expertise:
Bourdieu has had no interest in artificial intelligence and
little as yet in technology, but Searle (1992)
has used this model of human behavior as founded on an implicit background or
habitus to critique cognitive science and computational analogies of the
operation of the human mind, and it is at the heart of the Dreyfus (1986)
critique of expert systems.
What are the implications of an understanding of human
behavior in terms of habitus for research in AI and ES, apart from suggesting
that the task of developing expert systems comparable in their competence to
people is a difficult, if not impossible, one? It is, perhaps, salutary here to
reverse the analysis and examine the quality of judgement of experts. In a
survey of studies of the accuracy of human subjective probability judgements,
Tversky and Koehler conclude:
In the domain of expertise in scientific research,
Feyerabend (1975) has argued that there is no evidence of a rational methodology, and Fortun and
Bernstein (1998) have provided a compelling account of scientific progress as 'muddling
through.' In Voltaire's Bastards, Saul argues:
How is it that imperfect human capabilities are construed as
expertise and that muddling through is effective? One answer is that human
expertise arises in the context of human action as a pragmatic process of
dealing with present contingencies knowing that there will be further
opportunities to deal with the consequences of our actions at a later stage.
The decision to treat a patient in a certain way is an experiment that entails monitoring the consequences with a view
to planning future treatment. Human action takes place in a control loop with
imperfect information at each decision point, and with the unfolding process
continually changing the state of play.
In many situations it is more important to act in a way that
is not wildly wrong rather than to compute the optimum action, particularly
when available information is inadequate, inaccurate, expensive to obtain, and
so on. It is generally important to know who has the authority to act and who is
accountable for monitoring the consequences, taking follow-up action, and so
on. The giving, or taking, of the authority to be in control in a particular
domain demarcates the abstract role of an 'expert' in that domain relative to
the social norms of the institution that accepts ownership of the domain.
A simple analysis of the phenomenon of such assignment of
authority in a society of learning agents shows that actual expertise, in the
sense of greater capabilities, arises naturally through the positive feedback
processes involved in proto-experts having greater access to learning
experiences (Gaines, 1988).
An extended analysis shows that society can optimize the rate at which the
proto-experts learn without having any understanding of either the underlying
of the domain, the basis of expert performance in it, or the processes of
learning involved (Gaines, 1997).
The management of expertise formation in a society of learning agents can be
highly successful while being remarkably knowledge-free in all its aspects.
Figure 6 is a diagram from KAW'88 of the processes of
expertise formation through a variety of feedback processes (Gaines,
1989).
The central loop showing the client-expert dialog derives from studies by
Hawkins (1983)
of industrial experts in mineral exploration, and emphasizes that the
generation of advice is a feedback process of discourse and modeling. The upper
and lower ovals showing the expert's interaction with his or her professional
and client communities is what I would now want to describe in terms of the
development of the expert's habitus, using Bourdieu's term deliberately to
avoid any implication of the development within the expert of explicit
knowledge (and disliking the adjective 'explicit' in this statement because the
implicature of thus allowing the term 'implicit knowledge' may be highly
misleading). That is, I would say today that the process shown in Figure 6
captures much of the dynamics of expertise formation but would want to make the
matters of 'knowledge acquisition' and 'knowledge formation' the subject of a
different level of discussion.
The client community in Figure 6 constitutes the domain of
practice for the expert, and the role of knowledge-level explanation in that
community might be expected to be very different from that in the professional
community which, among other things, constitutes the domain of reflection. The
conditions of satisfaction in the client community are ones of achievement in problem-solving,
not necessarily success but at least the assessment of 'as well as might be
expected.' Discourse is at the level of potential outcomes, contingency plans,
risk management, about what might happen and how the contingencies may be
managed under different action plans rather than why questions involving
foundational considerations of underlying models.
The conditions of satisfaction in the professional community are
ones of effective expertise development and transmission, of access through apprenticeship,
case reports, evaluation of procedures, rationalization through links to
existing models, related literatures and so on. Discourse is at the level of
managing the formation of expertise and this may involve reflective processes
raising why questions and addressing foundations, but note that the objective
of these is to develop expertise, a coaching function, rather than to discover
'truth' or uncover 'reality.' Rationalizations are valid to the extent that
they help the development of expertise, and that development does not
necessarily leave any residue of the rationalization in the expert's mind. It
is possible to have an effective knowledge-level approach to expertise
development without basing it on a knowledge-level approach to expert performance
(Vickers, 1990).
Figure 6 Expertise formation through expert-community interaction
Bourdieu's other dimensions of field and social capital may
also be exemplified in terms of Figure 6. The expert acts in a specific
situation within a social network of power relations with clients, colleagues,
regulatory agencies, and so on, and competes within that field for symbolic
capital that will affect his or her ongoing and future status within such
fields. That is, the decisions and recommendations made are not just an outcome
of the problem situation and the expert's dispositions through his or her
habitus, but also reflect the interaction of habitus and field, in particular,
the impact upon the expert's symbolic capital of the possible outcomes. The
solution of any particular problem is situated within the processes of
developing the overall competence of the community as a social network. Shapin (1994)
has documented the importance of the power relations and symbolic capital in
the development of science.
Figure 7 is the latest version of an evolving model that we have
used at many KAW meetings and in many publications in an attempt to capture the
entire conceptual framework for human psychology, sociology, action and
knowledge in a simple diagram.
Figure 7 Levels and worlds of being
The central region presents a three-layer model of human
entities, whether roles, people, groups, institutions or societies. At the
bottom are the processes of interaction with the environment, of percepts,
acts, reflexes, sensation, transducers, and so on. This is the level that is
being emulated and extended with increasing effectiveness through neural
networks (Elman, Bates, Johnson, Karmiloff-Smith,
Parisi and Plunkett, 1999).
At the top are the processes of reason, of rationality, reflection, planning
and so on. This is the level that is being emulated with increasing
effectiveness through digital computation. In the middle are the processes of
practice, of culture, habitus and field characterizing the mental and the
social, action, mimicry, reward and punishment. This is the level where neither
neural networks nor digital computation have so far provided adequate
emulation, and lack of such emulation is the greatest impediment to the
development of expert systems.
The four surrounding boxes set human entities within the context
of Popper's (1968)
three worlds, as we have done in many
previous papers, but adding a fourth world at the top to balance the
presupposed World 1 of physical reality with an equally presupposed World 4 of
transcendental a priori
presuppositions and ideology. Popper would probably have placed our World 4 in
his World 3, as a human artifact, but we separate it here to emphasize its
psychological and cultural status as something presupposed not constructed.
Friedman (1999)
has presented a reconstruction of the work of the logical positivists,
particularly Carnap, suggesting that their contribution is best understood as
offering a new conception of a priori
knowledge and its role in empirical knowledge, the link between our Worlds 4
and 1. Searle (1998)
has argued that realism is based on a presupposition of a real world underlying
all our further discourse and hence is not itself subject to empirical study,
and there are other such presuppositions.
The box on the left of the central core attempts to situate
in relation to the three layers of the core a hierarchy of World 2 levels of
construction similar to those we previously derived from Klir's (1976)
epistemological hierarchy generated
through a system of distinctions (Gaines
and Shaw, 1984),
and have used to model various forms of knowledge transfer in individuals and
organizations (Gaines, 1994).
The box on the right of the central core attempts to situate in relation to the
three layers of the core some major World 3 products, with Giddens' (1986)
locales of practice in the center, and
Gibson's (1979)
affordances at the bottom. One feature
of this representation of World 3 in relation to World 2 is that it stresses
how human activity is not just culturally situated in its habitus and socially
situated in its field, but also artifactually situated in a humanly built world
that exists in major part to trigger off the dispositions within a habitus. Our
being is essentially embedded not only in the being of others with whom we
interact but with that of others who have left artifacts from their activities
within which ours take place.
There are many implications for research in the diagram
above, far too many since research in AI, ES and KA cannot be expected to take
on the problematiques of each and every
discipline represented in Figure 7. However, one can delineate some realistic
research agendas.
There are two major research areas currently concerned with
eroding the central territory of practice in Figure 7 by extending the areas of
interaction below it and reason above it. Connectionist research has had major
practical achievements in emulating human pattern learning capabilities at the
interaction level, and is seen by many researchers as capable of emulating
higher brain functions including the domain of practice. Spitzer's (1999)
The Mind Within the Net: Models of Learning, Thinking and Acting is a good exposition of the state of the art.
Lenat's Cyc project may be seen as an attempt to emulate human practice by
extending the domain of reason downwards and developing a rich habitus based on
a massive knowledge base coupled with a range of inference methods from logical
deduction, through statistical induction, to speculative reasoning based on
analogy (Lenat and Guha, 1990).
DARPA continues to fund the development and application of Cyc through Cycorp
(http://www.cyc.com/), and it is the core system in a range of well-funded
DARPA projects such as High-Performance Knowledge Bases (HPKB,
Cohen, Schrag, Jones, Pease, Lin, Starr, Gunning and Burke, 1998).
It is early days to forecast how far connectionism may move
up or Cyc-like systems may move down. One would expect success in domains where
the habitus is strongly circumscribed, such as highly specific roles that
people play that, given the state-of-the art of emulation of human
sensory-motor systems, also involve strongly circumscribed interaction with the
world. An example domain of this nature that has been extensively studied is
that of pronunciation of words from text. DECTalk is a text-to-speech expert
system with human capabilities modeled through rules with exceptions, and one
of the achievements of connectionism has been to show that a neural net, NETtalk,
can learn to speak better than the expert system (Sejnowski
and Rosenberg, 1987).
Later, Dietterich, Hild and Bakiri (1995)
found that better performance than both DECTalk and NETtalk could be achieved
through standard machine learning algorithms. This is an example of a
significant but highly circumscribed habitus being modeled through approaches
from below and above, and a bridge being created between the modeling of human
practice in the expert system, connectionism from below, and machine learning
from above. This is also a domain where there are major literatures on child
development, educational practice, psychological studies, cognitive models, and
so on, where it might be reasonable to expect an exhaustive synthesis to be
feasible.
One form of habitus that should be more amenable to modeling by
rules is where the behavioral regularities are induced by normative rules such
as government codes, company operating procedures or equipment operation
manuals. These examples characterize major areas of successful application of
expert systems technology. Business rules are generally imposed, not induced
from behavior, and it is notable also that Gensym's list of success stories
largely relates to industrial process control. In such applications the expert
system is put in place not so much to model the habitus but to manage it.
However, knowledge acquisition from experts is still relevant because normative
rule sets are rarely complete and require interpretive guidelines and
extensions often derived from practice.
The role of field and symbolic capital is also significant
for expert system development. One needs to situate the experts in their
institutional setting and analyze their roles within their social networks. What
is their organizational function, how are they recruited, how do they acquire
expertise, to whom do they report, on whom do they rely for support, and how
does all this play out in terms of tasks, action, monitoring and control? A
cognitive stance attempting to look within the experts' minds needs to be
complemented with an institutional stance examining their situations. In
conventional systems development, task and situational analyses are routine
techniques and often lead to organizational redesign that simplifies or
by-passes the need to incorporate existing roles and expertise.
What technologies might be most effective in modeling habitus? It
is interesting to note that the Wittgenstein-derived literature on the
incoherence of rules (e.g. Kripke, 1982),
and the Goodman-derived literature on the incoherence of inductive inference (e.g.
Stalker, 1994)
both use exceptions to rules in their counter-examples, that is breakdowns in
rules are fixed by adding exceptions. Paul Compton and I (Compton
and Jansen, 1990; Gaines, 1991a; Compton et al., 1992; Gaines and Compton,
1995; Gaines, 1996; Richards and Compton, 1998)
have long promoted the representation of expertise through rules with
multi-level exceptions as one that arises naturally, is easy to acquire, update
and understand, provides a pragmatic fit to complex human action but supports
reflection to extract the principled knowledge that corresponds to insight. One
can embed such rule-based model in as rich a representational schema as one
wishes, derived from pre-existent ontologies, just-in-time extensions to them
or pattern-formation in neural networks.
The major extension I would see as necessary to use such systems
to more richly model habitus is that multiple, prioritized rule sets need to be
used and the conclusions need to be general constraints not specific values so
that the output is more a structured constraint system than a single outcome (multiple-classification
RDR go some way towards this, Kang, Compton and Preston, 1995).
This would allow for the resolution of conflicting constraints which no
possible action satisfies but where there is a set of admissible actions that satisfice the constraints, and where
the selection of a particular action among them is indeterminate. This
indeterminacy is realistic in terms of human practice and desirable since the
role of randomness in breaking out of sub-optimal behavioral loops and learning
outcomes has been known since the early days of AI (Gaines,
1969).
What theoretical developments are promising for modeling
human expertise? I have already discussed Pierre Bourdieu's work on habitus,
field and symbolic capital. John Searle (1998)
seems to me to be providing the richest and most operational framework for
modeling human intentional behavior that is consistent with the notion of
habitus. Niklas Luhman (1995)
has provided a complementary framework for institutions based on his appropriation
of the notion of autopoiesis in the context of social systems. The appropriate
mathematical foundations are to be found in the literature on chaos theory and
its application in the social sciences (Vallacher
and Nowak, 1994; Eve, Horsfall and Lee, 1997).
This paper has had the pragmatic objective of attempting to
provide some perspectives on research in artificial intelligence, expert
systems and knowledge acquisition that will be useful in formulating future
research agendas. It has recalled the initial excitement, expectations, and
aspirations, reviewed what has happened to date, shown the extent of the Michie
effect whereby AI developments, once understood, are assimilated into mainstream
information technology, and suggested research opportunities for knowledge
support systems within the current ethos of convergence and integration.
It has addressed the continuing impediments to the computer
emulation of human expertise that stem from inadequate theories of the nature
of that expertise, and has surveyed developments in psychological, cultural and
sociological research that promise greater understanding of human practice. It
has suggested further research opportunities that bring those developments into
the ambit of artificial intelligence and support new approaches to expert and
knowledge support systems.
I subtitled this article, operationalizing the
enlightenment, because it seems to me that
computer technology is the latest of many powerful tools that have been
developed to further the processes that we associate with Greek enlightenment's
invention of new modes of thought and argument (Solmsen,
1975),
and the seventeenth century enlightenment's application and extension of those
intellectual tools, together with material tools resulting from advances in
technology, to create modern science (Cohen,
1994).
The notion of enlightenment has been a focus of discussion for some centuries
with many responses, reactions and evaluations. Zšllner's question, what is
enlightenment?, in the Berlinische Monatsschrift of December 1783 prompted a range of distinguished replies. Moses
Mendelssohn saw it as "related to theoretical matters: to (objective) rational
knowledge and to (subjective) facility in rational reflection about matters of
human life." Karl Reinhold saw it as "the making of rational men out of men who
are capabable of rationality." Immanuel Kant saw it as "mankind's exit from its
self-inflicted immaturityÉthe inabilility to make use of one's own
understanding without the guidance of another" and added the aphorism "If it is
asked 'Do we now live in an enlightened age?' the answer is 'No, but we do live
in an age of enlightenment.'" (Schmidt, 1996).
The notions of rationality, and the freedom to be rational, are
still with us as enlightenment objectives, and Kant's aphorism is as valid
today as it was over two centuries ago. The enlightenment is a project of which
we all, as scholars and researchers, are part. The computer is par
excellence a tool for making rationality
operational, for mechanically developing the consequences of our postulates in
an environment that ruthlessly exposes sloppy definitions and invalid
derivations. It is the ultimate tool of the enlightenment as we have conceived
it so far.
However, from the discussion in Section 4 and the literature
cited it should be clear that human beings and their institutions are not
naturally rational in this sense--enlightenment rationality is a stretch goal,
not a natural consequence of our being. And it may be a dangerous goal.
Horkheimer and Adorno (1972)
have argued "the fully enlightened earth radiates disaster triumphant."
Wojciechowski (1983)
has exemplified this in the way that the majority of the world's problems now
stem from knowledge, yet can only be solved by developing more knowledge, the
ultimate escalatory positive feedback loop. Bickerton (1990)
has argued that our higher level capabilities may not be survival traits for
the species. Bourdieu (1988)
has turned the spotlight of his analysis of habitus on homo academicus and shown how scholarly practices conform to the
same principles as other behavior which we would not regard as rational by our
idealistic canons. Rationality is not a path to utopia but, in the developed
world at least, it has become one of those presuppositions that is core to the
habitus created by our educational systems. We could only attempt to reject it,
in most spheres of our society, within a framework that accepts it.
I believe these deep discussions at the species level parallel
significant discussions that need to take place at the institutional level. Why
should knowledge management that attempts to derive explicit knowledge from
implicit knowledge be expected to improve some evaluative measure of an
institution? Our habitus leads to this intuition, but that is as much a source
of blindness as insight. The entire conceptual framework needs deconstruction:
what do we mean by 'implicit knowledge'; does it exist; what is it to make it
explicit; can we do this; how should we proceed; what outcomes should we
expect; how can we measure the cost of doing all this and the benefits, if any,
that result? In practice, as with expert systems, some organizations will
experiment, claim benefit, and use this to advance their competitive position
through their marketing stance, true or not. That is the nature of practice,
and the engineering of rationality is embedded in the socio-economic practices
of those responsible for it, like any other engineering project.
This is not to pour scorn on those who advocate some form of
knowledge management. The social practices that are described by major authors
in this area are often interesting, innovative and attractive, advocating more
open and sharing institutions promoting the emergence of leadership and teaming
appropriate to changing contingencies. One can well imagine that the processes
advocated can be effective in improving performance, and that the rationale
provided is comprehensible, meaningful, acceptable and motivating. However,
none of that connects the rationale to the underlying processes that lead to
these outcomes in any rational, scientific way. Research on knowledge
management does, however, as did that on expert systems, provide an
experimental playing field in which scientific research on those underlying
processes might be conducted. There are important opportunities to be grasped.
In conclusion, I think the field of knowledge acquisition
research is as exciting, challenging and rewarding as it was twenty years ago.
It is far more daunting for the young researchers entering the field because of
the accumulated literature of many thousands of papers with links to other rich
literatures. It is less fashionable because industry's focus of attention has
moved elsewhere, and start-up fields with small literatures are easier to enter
and promise more rapid chances for establishing one's reputation. However,
there are rich opportunities for major scientific and technological
contributions, and I hope this article has helped to indicate some of them.
Financial assistance for this work
has been made available by the Natural Sciences and Engineering Research
Council of Canada. I am grateful to Paul Compton for the opportunity to make
this presentation and for his stimulating collegial support and generous access
to his research materials over many years.
Alford, J., Cairney, C., Higgs, R., Honsowetz, M., Huynh, V., Jines, A., Keates, D. and
Skelton, C. (2000). Real rewards from artificial intelligence. InTech (April) 52-55.
Bar-Hillel, Y. (1964). Language and Information:
Selected Essays on Their Theory and Application. Reading, Massachusetts, Addison- Wesley.
Berners-Lee, T. (1989). Information Management: A
Proposal. CERN, Geneva. http://www.w3.org/History/1989/proposal.html.
Bickerton, D. (1990). Language and Species . Chicago, University of Chicago Press.
Blum, M. and Blum, L. (1975). Toward a mathematical
theory of inductive inference. Information and Control 28(2)
125-155.
Bourdieu, P. (1988). Homo Academicus. Cambridge, Polity Press.
Bourdieu, P. (1990). In Other Words: Essays Toward a
Reflexive Sociology. Oxford, Polity.
Buchanan, B.G., Duffield, A.M. and Robertson, A.V.
(1971). An application of artificial intelligence to the interpretation of mass
spectra. Milne, G.W.A., Ed. Mass Spectrometry Techniques and Applications. New York, John Wiley.
Chomsky, N. (1956). Three models for the description of a
language. I.R.E. Transactions on Information Theory 2
113-124.
Clancey, W.J. (1989). Viewing knowledge bases as
qualitative models. IEEE Expert 4 (2) 9-23.
Clancey, W.J. (1993). The knowledge level reinterpreted:
modeling socio-technical systems. International Journal of Intelligent
Systems 8 (1) 33-49.
Clancey, W.J. (1997). Situated Cognition: On Human
Knowledge and computer Representation.
Cambridge, UK, Cambridge University Press.
Cohen, H.F. (1994). The Scientific Revolution: A
Historiographical Inquiry. Chicago,
University of Chicago Press.
Cohen, P., Schrag, R., Jones, E., Pease, A., Lin, A.,
Starr, B., Gunning, D. and Burke, M. (1998). The DARPA High-Performance
Knowledge Bases Project. AI Magazine 19 (4) 25-49.
Compton, P., Edwards, G., Kang, B., Lazarus, L., Malor,
R., Preston, P. and Srinivasan, A. (1992). Ripple down rules: turning knowledge
acquisition into knowledge maintenance. AI in Medicine 4(6)
463-475.
Compton, P. and Jansen, R. (1990). A philosophical basis
for knowledge acquisition. Knowledge Acquisition 2(3) 241-258.
Crane, D. (1972). Invisible Colleges: Diffusion of
Knowledge in Scientific Communities.
Chicago, University of Chicago Press.
Date, C.J. (2000). What Not How: The Business Rules
Approach to Application Development.
Reading, MA, Addison-Wesley.
Davis, M. (1965). The Undecidable: Basic Papers on
Undecidable Propositions, Unsolvable Problems and Computable Functions. Hewlett, N.Y., Raven Press.
Davis, R. and Lenat, D.B. (1982). Knowledge-Based
Systems in Artificial Intelligence. New
York, McGraw-Hill.
De Mey, M. (1982). The Cognitive Paradigm. Dordrecht, Holland, Reidel.
Dietterich, T.G., Hild, H. and Bakiri, G. (1995). A
comparative study of ID3 and backpropagation for English text-to-speech mapping.
Machine Learning 18 51-80.
Donini, F.M., Lenzerini, M., Nardi, D. and Nutt, W.
(1997). The complexity of concept languages. Information and Computation 134(1)
1-58.
Dreyfus, H.L. (1972). What Computers Can't Do: A
Critique of Artificial Reason. New York,
Harper & Row.
Dreyfus, H.L. and Dreyfus, S.E. (1986). Mind over
Machine: The Power of Human Intuition and Expertise in the Era of the Computer. New York, Free Press.
Elman, J.L., Bates, E.A., Johnson, M.H., Karmiloff-Smith,
A., Parisi, D. and Plunkett, K. (1999). Rethinking Innateness: A
Connectionist Perspective on Development.
Cambridge, MA, MIT Press.
Eve, R.A., Horsfall, S. and Lee, M.E. (1997). Chaos,
Complexity, and Sociology: Myths, Models, and Theories. Thousand Oaks, CA, Sage.
Farquhar, A., Fikes, R. and Rice, J. (1996). The
Ontolingua server: a tool for collaborative ontology construction. Gaines, B.R.
and Musen, M.A., Ed. Proceedings of Tenth Knowledge Acquisition Workshop. pp.63-1-63-7
(http://ksi.cpsc.ucalgary.ca/KAW/KAW96/farquhar-demo/farquhar-demo.html).
Fayyad, U.M. (1996). Advances in Knowledge Discovery
and Data Mining. Menlo Park, CA, AAAI
& MIT Press.
Feyerabend, P., Ed. (1975). Against Method. London, NLB.
Fleck, J. (1982). Development and establishment in
artificial intelligence. Elias, N., Martins, H. and Whitley, R., Ed. Scientific
Establishments and Hierarchies.
pp.169-217. Holland, D.Reidel.
Fortun, M. and Bernstein, H.J. (1998). Muddling
Through: Pursuing Science and Truths in the 21st Century. Washington, Counterpoint.
Foster, I. and Kesselman, C. (1999). The Grid:
Blueprint for a New Computing Infrastructure.
San Francisco, Morgan Kaufmann.
Friedman, M. (1999). Reconsidering Logical Positivism. Cambridge, Cambridge University Press.
Gadamer, H.G. (1972). Wahrheit und Methode. TŸbingen, Mohr.
Gaines, B.R. (1969). Stochastic computing systems. Tou,
J., Ed. Advances in Information Systems Science, 2. pp.37-172. New York, Plenum Press.
Gaines, B.R. (1977). System identification, approximation
and complexity. International Journal of General Systems 2(3)
241-258.
Gaines, B.R. (1978). Computers in world three. Proceedings
of the International Conference on Cybernetics and Society. pp.1515-1521. NY, IEEE (78CH-1306-0-SMC III).
Gaines, B.R. (1984). Perspectives on fifth generation
computing. Oxford Surveys in Information Technology 1
1-53.
Gaines, B.R. (1986). Sixth generation computing: a
conspectus of the Japanese proposals. ACM SIGART Newsletter 95
39-44.
Gaines, B.R. (1988). Positive feedback processes
underlying the formation of expertise. IEEE Transactions on Systems, Man
& Cybernetics SMC-18 (6) 1016-1020.
Gaines, B.R. (1989). Social and cognitive processes in
knowledge acquisition. Knowledge Acquisition 1(1) 251-280.
Gaines, B.R. (1990). Knowledge support systems. Knowledge-Based
Systems 3 (3) 192-203.
Gaines, B.R. (1991a). Integrating rules in term
subsumption knowledge representation servers. AAAI'91: Proceedings of the
Ninth National Conference on Artificial Intelligence. pp.458-463. Menlo Park, California, AAAI Press/MIT Press.
Gaines, B.R. (1991b). Modeling and forecasting the
information sciences. Information Sciences
57-58 3-22.
Gaines, B.R. (1994). The collective stance in modeling
expertise in individuals and organizations. International Journal of Expert
Systems 7 (1) 21-51.
Gaines, B.R. (1996). Transforming rules and trees into
comprehensible knowledge structures. Fayyad, U.M., Piatetsky-Shapiro, G.,
Smyth, P. and Uthurusamy, R., Ed. Knowledge Discovery in Databases II. pp.205-226. Cambridge, Massachusetts, AAAI/MIT Press.
Gaines, B.R. (1997). Knowledge management in societies of
intelligent adaptive agents. Journal for Intelligent Information Systems 9(3)
277-298.
Gaines, B.R. (1998). The learning curves underlying
convergence. Technological Forecasting and Social Change 57(1)
7-34.
Gaines, B.R. and Compton, P. (1995). Induction of
ripple-down rules applied to modeling large databases. Journal for
Intelligent Information Systems 5 (3) 211-228.
Gaines, B.R. and Shaw, M.L.G. (1984). Hierarchies of
distinctions as generators of system theories. Smith, A.W., Ed. Proceedings
of the Society for General Systems Research International Conference. pp.559-566. Louisville, Kentucky, Society for
General Systems Research.
Gaines, B.R. and Shaw, M.L.G. (1986). A learning model
for forecasting the future of information technology. Future Computing
Systems 1 (1) 31-69.
Gaines, B.R. and Shaw, M.L.G. (1994). Using knowledge
acquisition and representation tools to support scientific communities. AAAI'94:
Proceedings of the Twelfth National Conference on Artificial Intelligence. pp.707-714. Menlo Park, California, AAAI Press/MIT
Press.
Gaines, B.R. and Shaw, M.L.G. (1997). Knowledge
acquisition, modeling and inference through the World Wide Web. International
Journal of Human-Computer Studies 46 (6) 729-759.
Garey, M.R. and Johnson, D.S. (1979). Computers and
Intractability: A Guide to the Theory of NP-completeness. San Francisco, W. H. Freeman.
Gibson, J.J. (1979). The Ecological Approach to
Perception. Boston, Houghton Mifflin.
Giddens, A. (1986). The Constitution of Society:
Outline of the Theory of Structuration.
California, University of California Press.
Gilfillan, S.C. (1937). The prediction of inventions.
Ogburn, W.F., Ed. Technological Trends and National Policy, Including the
Social Implications of New Inventions.
Washington, U.S. Government Printing Office.
Gold, E.M. (1967). Language identification in the limit. Information
and Control 10 (5) 447-474.
Grover, L.K. (1996). A fast quantum mechanical algorithm
for database search. Proc. 28th Annual Symposium on the Theory of Computing. pp.212-218. New York, ACM Press.
Hackney, D. (1997). Understanding and Implementing
Successful Data Marts. Reading, MA,
Addison-Wesley Pub. Co.
Hawkins, D. (1983). An analysis of expert thinking. International
Journal of Man-Machine Studies 18 (1) 1-47.
Hayes-Roth, F. (1984). The industrialization of knowledge
engineering. Reitman, W., Ed. Artificial Intelligence Applications for
Business. pp.159-177. Norwood, New Jersey,
Ablex.
Horkheimer, M. and Adorno, T.W. (1972). Dialectic of
Enlightenment. New York, Herder and
Herder.
Kang, B., Compton, P. and Preston, P. (1995). Multiple
classification ripple down rules: evaluation and possibilities. Proceedings
of the 9th AAAI-Sponsored Banff Knowledge Acquisition for Knowledge-Based
Systems Workshop. Banff, Canada,
University of Calgary.
Kirby, S. (1999). Function, Selection, and Innateness:
The Emergence of Language Universals.
Oxford, Oxford University Press.
Klir, G.J. (1976). Identification of generative structures
in empirical data. International Journal of General Systems, 3
89-104.
Kremer, R.C. (1991). Experience in applying KRS to an
actual business problem. Boose, J.H.G., B.R., Ed. Proceedings of the Sixth
AAAI Knowledge Acquisition for Knowledge-Based Systems Workshop. pp.11-1-11-12. Calgary, Canada, University of
Calgary.
Kripke, S.A. (1982). Wittgenstein on Rules and Private
Language: An Elementary Exposition.
Oxford, Blackwell.
Langley, P. (2000). The computational support of
scientific discovery. International Journal of Human-Computer Studies 53(3)
393-410.
Lee, Y., Buchanan, B.G. and Rosenkrantz, H.S. (1996).
Carcinogenecity predictions for a group of 30 chemicals undergoing rodent
cancer bioassays based on rules derived from subchronic organ toxicities. Environmental
Health Perspectives 104 (Suppl 5) 1059-1063.
Lek, S. and GuŽgan, J.-F. (2000). Artificial Neuronal
Networks : Application to Ecology and Evolution. New York, Springer.
Lenat, D.B. and Guha, R.V. (1990). Building Large
Knowledge-Based Systems. Reading,
Massachusetts, Addison-Wesley.
Lighthill, J. (1973). Artificial intelligence: a general
survey. Artificial Intelligence: a paper symposium. UK, Science Research Council.
Linstone, H.A. and Sahal, D., Ed. (1976). Technological
Substitution: Forecasting Techniques and Applications. New York, Elsevier.
Luhmann, N. (1995). Social Systems. Stanford, CA, Stanford University Press.
Meadows, D.H., Meadows, D.L. and Randers, J. (1992). Beyond
the Limits: Confronting Global Collapse, Envisioning a Sustainable Future. Mills, Chelsea Green.
Meister, J.C. (1998). Corporate Universities : Lessons
in Building a World-class Work Force. New
York, McGraw-Hill.
Michalski, R.S. and Chilausky, R.L. (1980). Knowledge
acquisition by encoding expert rules versus computer induction from examples--A
case study involving soyabean pathology. International Journal of
Man-Machine Studies 12 63-87.
Moto-oka, T., Ed. (1982). Fifth Generation Computer
Systems. Amsterdam, North-Holland.
Motta, E., Stutt, A., Zdrahal, Z., O'Hara, K. and
Shadbolt, N. (1996). Solving VT in VITAL: A study in model construction and
knowledge reuse. International Journal of Human-Computer Studies 44(3/4)
333-371.
Natarajan, B.K. (1991). Machine Learning: A
Theoretical Approach. San Mateo, CA,
Morgan Kaufmann.
Nebel, B. (1990). Terminological reasoning is inherently
intractable. Artificial Intelligence 43 235-249.
Newborn, M. (1997). Kasparov Versus Deep Blue:
Computer Chess Comes of Age. New York,
Springer.
Nonaka, I. and Takeuchi, H. (1995). The Knowledge-Creating
Company. Oxford, Oxford University Press.
Pierce, J.R. (1969). Whither speech recognition? Journal
of the Acoustical Society of America 46 1049-1051.
Popper, K.R. (1968). Epistemology without a knowing
subject. Rootselaar, B.V., Ed. Logic, Methodology and Philosophy of Science III. pp.333-373. Amsterdam, North-Holland.
Richards, D. and Compton, P. (1998). Taking up the
situated cognition challenge with ripple down rules. International Journal
of Human Computer Studies 49 (895-926)
Rothenfluh, T.E., Gennari, J.H., Eriksson, H., Puerta,
A.R., Tu, S.W. and Musen, M.A. (1996). Reusable ontologies,
knowledge-acquisition tools, and performance systems: PROTEGE-II solutions to
Sisyphus-2. International Journal of Human-Computer Studies 44(3/4)
303-332.
Saul, J.R. (1993). Voltaire's Bastards: The
Dictatorship of Reason in the West.
Toronto, Penguin.
Schaeffer, J. and Plaat, A. (1997). Kasparov versus Deep
Blue: The Re-match. Journal of the International Computer Chess Association 20(2)
95-101.
Schmidt, J., Ed. (1996). What is Enlightenment? Berkeley, University of California Press.
Schšn, D.A. (1983). The Reflective Practitioner. New York, Basic Books.
Searle, J.R. (1992). The Rediscovery of the Mind. Cambridge, MA, MIT Press.
Searle, J.R. (1998). Mind, Language and Society:
Philosophy in the Real World. New York,
NY, Basic Books.
Seiler, H. (1999). Managing business rules: a
repository-based approach. Rule Machines Corporation. www.RuleMachines.com.
Sejnowski, T.J. and Rosenberg, C.R. (1987). Parallel networks
that learn to pronounce English text. Complex Systems 1
145-168.
Shapin, S. (1994). A Social History of Truth: Civility
and Science in Seventeenth-Century England.
Chicago, University of Chicago Press.
Shaw, M.L.G. and Gaines, B.R. (1987). KITTEN: Knowledge
initiation and transfer tools for experts and novices. International Journal
of Man-Machine Studies 27 (3) 251-280.
Shen, W. and Norrie, D.H. (1998). An Agent-Based Approach
for Distributed Manufacturing and Supply Chain Management. Jacucci, G., Ed. Globalization
of Manufacturing in the
Digital Communications Era of the 21st Century:
Innovation, Agility, and the Virtual Enterprise. pp.579-590. Dordrecht, Kluwer.
Shortliffe, E.H. (1976). Computer-Based Medical
Consultations: MYCIN. New York, Elsevier.
Shortliffe, E.H. and Clancey, W.J. (1984). Anticipating
the second decade. Clancey, W.J. and Shortliffe, E.H., Ed. Readings in
medical artificial intelligence : the first decade. pp.463-472. Reading, MA, Addison-Wesley.
Smolka, P. and Volkheimer, W. (2000). Southern
Hemisphere Paleo- and Neoclimates: Key Sites, Methods, Data and Models. Berlin, Springer.
Solmsen, F. (1975). Intellectual Experiments of the
Greek Enlightenment. Princeton, N.J.,
Princeton University Press.
Spitzer, M. (1999). The Mind Within the Net: Models of
Learning, Thinking, and Acting. Cambridge,
MA, MIT Press.
Stalker, D.F. (1994). GRUE!: The New Riddle of
Induction. Chicago, Open Court.
Swanson, D.R. (1990). The absence of co-citation as a
clue to undiscovered causal connections. Borgman, C.L., Ed. Scholarly
Communication and Bibliometrics.
pp.129-137. Newbury Park, Sage Publications.
Tiwana, A. (2000). The Knowledge Management Toolkit. NJ, Prentice-Hall.
Tversky, A. and Koehler, D.J. (1994). Support theory: a
nonextensional representation of subjective probability. Psychological
Review 101 (4) 547-567.
Ulanowicz, R.E. (1991). Formal agency in ecosystem
development. Higashi, M. and Burns, T.P., Ed. Theoretical Studies of
Ecosystems: The Network Perspective.
pp.58-70. Cambridge, Cambridge University Press.
Valiant, L.G. (1974). A theory of the learnable. Communications
of the ACM 27 1134-1142.
Vallacher, R.R. and Nowak, A. (1994). Dynamical
Systems in Social Psychology. San Diego,
Academic Press.
Vickers, J.N. (1990). Instructional Design for
Teaching Physical Activities: A Knowledge Structures Approach. Champaign, Illinois, Human Kinetics.
Von Krogh, G., Ichijo, K. and Nonaka, I. (2000). Enabling
Knowledge Creation: How to Unlock the Mystery of Tacit Knowledge and Release
the Power of Innovation. Oxford, Oxford
University Press.
Weizenbaum, J. (1976). Computer Power and Human
Reason: From Judgement to Calculation. San
Francisco, W.H.Freeman.
Wills, G. (1998). The Knowledge Game: The Revolution
in Learning and Communication in the Workplace. London, Cassell.
Wittgenstein, L. (1953). Philosophical Investigations. Oxford, Blackwell.
Wojciechowski, J.A. (1983). The impact of knowledge on
man: the ecology of knowledge. Hommage a Francois Meyer. pp.161-175. Marseille, Laffitte.
3.2 Emergence of Knowledge Science
3.3 Knowledge Support Systems
3.4 Trends and Limitations
4 The Nature of Expertise and Knowledge
4.1 What is an Expert?
"a person who has special skill or knowledge in some particular
field; specialist; authority,"
"possessing special skill or knowledge; trained by practice;
skillful or skilled."
"the ability, coming from one's knowledge, practice,
aptitude, etc., to do something well,"
"competent excellence in performance; expertness; dexterity."
"acquaintance with facts, truths, or principles, as from
study or investigation."
"the power to determine, adjudicate, or otherwise settle issues
or disputes; jurisdiction; the right to control, command or determine."
"For Hegel, it is necessary, of course, that the movement of
consciousness, experience should lead to a self-knowledge that no longer has
anything different or alien to itself. For him the perfection of experience is
'science', the certainty of itself in knowledge." (Gadamer, 1972)
"The nature of experience is conceived in terms of that
which goes beyond it; for experience can never be science. It is in absolute
antithesis to knowledge and to that kind of instruction that follows from
general or theoretical knowledge. The truth of experience always contains an
orientation towards new experience. That is why a person who is called 'expert'
has become such not only through experiences, but is also open to new
experiences. The perfection of his experience, the perfect form of what we call
'expert', does not consist in the fact that someone already knows everything
and knows better than anyone else. Rather, the expert person proves to be, on
the contrary, someone who is radically undogmatic; who, because of the many
experiences he has had and the knowledge he draws from them is particularly
equipped to have new experiences and learn from them." (Gadamer,1972)
"The new perspective, often called situated cognition, claims that all processes of behaving,
including speech, problem-solving, and physical skills, are generated on the
spot, not by mechanical application of scripts or rules previously stored in
the brain. Knowledge can be represented, but it cannot be exhaustively
inventoried by statements of belief or scripts for behaving. Knowledge is a
capacity to behave adaptively within an environment; it cannot be reduced to
representations of behavior or the environment." (Clancey, 1989)
"A representation is not equivalent to knowledge
A representation of what a
person knows is just a model of his or her knowledge, a representation of a
capacity. Knowledge cannot be reduced to (fully captured by) a body of
representations. Knowledge cannot be inventoried.
The meaning of a representation cannot be made explicit
Meaning can be represented, but
it cannot be defined once and for all, captured fully by representations. The
meaning of a representation is open, though there are culturally stable
representations of meaning (e.g., word senses).
The context in which a program is used cannot be made explicit"
Context can be represented, but
the world cannot be objectively and exhaustively described; cultural or social
circumstances cannot be reduced to a set of facts and procedures." (Clancey,
1993)
4.2 What is the Basis of Expertise?
"This was our paradox: no course of action could be
determined by a rule, because every course of action could be made to accord
with the rule...'obeying a rule' is a practice...If I have exhausted the
justifications I have reached bedrock, and my spade is turned. Then I am
inclined to say: 'This is simply what I do.'" (Wittgenstein,
1953, 201, 202, 217)
"I can say that all my thinking started from this point: how
can behaviour be regulated without being the product of obedience to rules? (Bourdieu,
1990, 65)"
"I am talking about dispositions acquired through
experience, thus variable from place to place and time to time. This 'feel for
the game', as we call it, is what enables an infinite number of 'moves' to be
made, adapted to the infinite number of possible situations which no rule,
however complex, can foresee. (Bourdieu, 1990, 65)"
"Action guided by a 'feel for the game' has all the
appearances of the rational action that an impartial observer, endowed with all
the necessary information and capable of mastering it rationally, would deduce.
And yet it is not based on reason. (Bourdieu, 1990, 65)"
"The evidence reported here and elsewhere indicates that
both qualitative and quantitative assessments of uncertainty are not carried
out in a logically coherent fashion, and one might be tempted to conclude that
they should not be carried out at all. However, this is not a viable option
because, in general, there are no alternative procedures for assessing
uncertainty." (Tversky and Koehler, 1994)
"Among the illusions which have invested our civilization is
an absolute belief that the solution to our problems must be a more determined
application of rationally structured expertise. The reality is that our
problems are largely the product of that application." (Saul,
1993, 8)
4.3 The Dynamics of Expertise Formation
4.4 An Overall Framework for Human Activity
4.5 Implications for Research
5 Conclusions--Operationalizing the Enlightenment
Acknowledgment
References