Managing prototype knowledge-expert system projects.

MANAGING PROTOTYPE KNOWLEDGE/EXPERT SYSTEM PROJECTS As guidance to those who are investigating knowledge/expert systems (K/ES) technology for the first time, we propose that the most efficient way to investigate this technology is to build a demonstration prototype. This prototype is the first step in a three-stage process that is typically followed for building K/ES. It makes sense to use this first step to prove the technology as well as to acquire system buildng skill. The demonstration prototype stage differs from the two subsequent stages in that it results in K/ES tha behave intelligently in only a very limited segment of the domain.

The second stage of system building is full prototype phase that incorporates both breadth and depth across the problem domain, but is not ina deliverable form for daily use.

The third stage is the delivered system, usable on a day-to-day production basis. This stage requires the conceptualization of the remaining system building activity as development of a product. This is true even if the final system will only be used internally. Thinking product, instead of project, means paying attention to such factors as performance, usability of the interface, documentation, training, support and reliability.

While it is difficult to generalize, it will typically take three to six months to develop the demonstration prototype, six months to a year to develop the full prototype, and eighteen months to two years to finish the delivered system. The finished system will not be static but will be expected to change over time. Hewett and Sasson [10] state that a delivered system takes an order of magnitude more time to build that its corresponding demonstration prototype; an estimate quite in line with the system buildng times already mentioned.

It is important to recognize the strategic importance of building a demonstration prototype. It is the vehicle used to demonstrate to senior management, domain experts, and other colleagues the intelligent behavior (and hence, economic value) of these systems, the rapid protyping properties of this technology, and the ability of your project team to deal effectively with this new technology. Once this is accomplished, it should be easier to convince senior management of the merits of prototype development for eventual delivery to users. One might evern argue for the expansion of the initial group and the development of additional applications.

Development of a final, deliverable product will usually involve significant product enhancement and software engineering activities and not be a simple extension of the demonstration prototype. As noted, this may involve two to three times the effort required to develop the prototype and is thus a significant investment. The management of expectations such as these is an important component of project management for K/ES.

IT i counterproductive to attempt to teach the details of K/ES technolog to senior management. Mr. Tim McCullough of 3M Company recently commented, in a user's group meeting, about the importance of "building faith where there is funding and understanding where there is curiosity." In other words, convince senior management of the wisdom of building a prototype based on the ability to deliver what is promised, and convince the user community based on an intelligent understanding of how the technology can assist them in their work.

From here on, we will use the term K/ES to refer to large computer systems that are either model-based or rule-based knowledge systems or the hybrid combination of both. Traditionally, such systems have used rule-based reasoning in which human knowledge is captured in the form of IF-THEN rules such as, "IF a person's right hand is signifiantly larger that his left hand, THEN his job type is or was manual labor". Fortunately, we are now at a stage where intelligent systems can be buit using model-based reasoning alone, or in conjunction with rule-based reasoning.

Model-based reasoning involves building a computer model of the world to be represented. This model might contain rules, but it is also likely to contain many other mechanisms for representing knowledge such as frames, methods, and demons. We call these model-based systems knowledge systems, a term which encompasses pure rule-based expert systems as a subset.

Operative Principles

This articles proposes that K/ES technology is both complex and extremely easy to mismanage. In addition, we suggest that K/ES:

* cannot be managed by traditional Management Information Systems (MIS) organizations as they now exist in most Fortune 500 companies, although we trust that traditional MIS organizations will incorporate this technology eventually;

* should not be managed by techies with graduate training in artificial intelligence (AI) but no practical management experience;

* will not be properly managed by nontechnically trained MBA's or other nontechnical management personnel.

We argue that K/ES technology should:

* be managed by technically trained personnel (e.g., engineers, scientists) with proven management skill and appropriate AI training;

* report directly to an executive-level individual with some or all of the corporate responsibility for the technical direction of the company; and

* be acquired by a technology-transfer process that involves training a company's best computer scientists in the tools and techniques of AI. Modest doses of external hiring are probably wise, but not until the technology has executive support as a result of prototype demonstrations.

Much of the advice offered here may not apply to smaller knowledge systems. Theefore we will clarify what is meant by K/ES.

We assume the knowledge that is captured in software must be obtained from one or more human experts through a process of interviews and discussions. …