Detecting deception: adversarial problem solving in a low base-rate world.

1. Introduction

Information in organizations and in society generally is often communicated in a context of conflict of interest, and with awareness of potential decision consequences (Feldman & March, 1988). Deception is one particularly powerful form of information manipulation that occurs when an opportunistic agent induces a misrepresentation that is designed to influence the behavior of another agent (Hyman, 1989; Johnson, Grazioli & Jamal, 1993; Russow, 1986). Deception poses a problem to individuals and organizations because those who fail to detect a deceiver's malicious manipulation take action on the basis of misrepresented information and, as a result, behave in ways that are more favorable to the deceiver than otherwise warranted. The problem of detecting deceptions is solved by identifying the dements of the manipulation and by taking corrective action.

The need to detect possible manipulations arises frequently since adversarial situations with potential for deception are found in virtually all domains of activity characterized by a conflict of interest between cognitive agents (e.g., in business, politics, warfare, games, intelligence and counterintelligence (Bowyer, 1982; Ekman, 1992; Mawby & Thompson, 1986). Yet, the occurrence of specific deceptions must be relatively rare, especially in domains where interactions are repeated and feedback is available: if a particular form of deceit is frequent, victims learn how to circumvent it (Akerlof, 1970; Kraut, 1980).

The relatively infrequent occurrence of specific deceptions, and the adversarial nature of deception as a social interaction, constrains the nature of the knowledge that cognitive agents develop to solve the detection problem (Thagard, 1992). Moreover, the element of intentionality introduced by the presence of an adversary sets the problem of detecting deception apart from the larger class of tasks where evidence is independent of the nature of the process employed to identify and analyze it (Johnson et al., 1981). Deception works because the Deceiver, 1) intentionally exploits the weaknesses of the Target's information processing, and 2) is aware of the Target's possible detection efforts, and acts so as to thwart them.

Under these conditions it is not surprising that naive Targets have high average rates of failure at detecting specific forms of deception (e.g., Ekman, 1992). What is surprising is the fact that the rate of failure often remains high, even when the Targets have been trained at the task within which deception occurs and are motivated by high stakes (e.g., Johnson, Grazioli & Jamal, 1992). For example, failure to detect forms of deception in the financial markets entails monetary losses amounting in some cases to hundreds of million of dollars, as well as the loss of careers and reputations (e.g., Albrecht, Wernz & Williams, 1995). Yet, the detection problem is not hopelessly unsolvable: small percentages of individuals in experimental studies of deception (e.g., secret service agents, judges, psychiatrists, auditors) have been found to consistently succeed at detecting forms of deceit within tasks in their domains of expertise (e.g., Ekman 1992; Johnson, Grazioli & Jamal, 1992).

The study reported here investigates success and failures in detecting deceptions in one such task. Specifically, we examine how twenty-four practicing auditors cope with the task of detecting deception (fraud) in cases in which financial information was manipulated by management for the purpose of making companies appear more profitable than they actually were.

Using the auditors' data, we develop a theory of detecting financial deceptions based on the proposition that successful individuals adopt a variant of Dennett's intentional stance strategy (1989). This strategy consists of the application of deception-detection heuristics developed outside the context of auditing (Johnson, Grazioli & Jamal, 1993). The theory is evaluated by building a `competence' model of deception detection that embodies these heuristics, and by testing the model against the cases given to the auditors.

We expand the theory to account for the observed behavior in our sample of auditors, inclusive of successes and failures. Because the detection of fraud is relatively infrequent (Hansen, 1996; Loebbecke, Einning & Willingham, 1989), most auditors do not receive feedback from which they can develop skills in fraud detection. As a result, learning how to apply detection heuristics in the domain of auditing is incompletely specified (underconstrained). Knowledge is generated that is compatible with past experience, but fraught with the potential for failure.

Accordingly, we explain failure at detecting deception by means of imperfections in the knowledge of the domain in which the deception occurs (i.e., accounting). Specific hypotheses about these imperfections are derived from the errors observed in the behavior of a subset (five) of the twenty-four auditors. We evaluate the theory of success and failure at detecting financial deceptions by seeding the hypothesized imperfections in the competence model of deception detection and using the modified model to predict detection outcomes for the remaining nineteen auditors.

2, The problem of fraud detection

Most companies whose securities are publicly issued and held are required to release audited financial statements annually. These statements are a means of informing various interested parties about the company's economic conditions. Investors, creditors and regulators use financial statements as evidence in making decisions such as investing in a company's stock, granting a loan, and assessing compliance with laws and regulations. Since these decisions have consequences for the company, management may opt to misstate its financial conditions for the purpose of favorably affecting the outcomes of these decisions.

The public has incurred substantial losses due to financial misrepresentations (Albrecht, Wernz & Williams, 1995). Through the legal system, society has charged professional auditors with responsibility for detecting fraudulent misrepresentations and determining whether the financial statements issued by a company are fairly presented. Financial statements are prepared for companies on the basis of the Generally Accepted Accounting Principles (GAAP), a set of principles that specify how to fairly present the financial condition and results of operations of a company.

In the study reported here, we focus on the detection of fraudulent manipulations of financial reports. Such manipulations cover intentional misstatements or omissions of amounts or disclosures in financial statements, which are intended to deceive the users of those statements. The context for the study is the auditing task called second-partner review. In this task, a partner of an accounting firm reviews draft financial statements of a client company, footnotes to the statements, and other descriptive financial materials. The objective of the review is to evaluate the `engagement' partner's conclusion as to the appropriate audit `opinion.' The engagement partner is the auditor who directs the audit fieldwork and develops the opinion as to the fairness of the company's financial statements.

Several audit opinions are possible. If the auditor feels that the financial statements are fairly presented, an `unqualified' (or `clean') opinion is issued. If the auditors feels that the statements `are fairly presented, yet some areas of concern need to be highlighted, an `unqualified opinion with additional paragraph' for items such as doubt as to continued financial viability (going concern), material uncertainties or lack of consistency, is issued.

If the auditor disagrees with the accounting methods used, or with the manner in which they are applied, and the violation has a material effect on the financial statements, the auditor renders either a `non-GAAP' or an `adverse' opinion. A `non GAAP' opinion is issued when there is a departure from GAAP as to the accounting principles selected or their application, and where disclosures are inadequate. An adverse opinion is issued when the departures from GAAP are judged to be very material.

In the experiment reported here, twenty-four practicing auditors who were partners in a major international accounting firm were asked to perform a second partner review of four cases on which an unqualified opinion was issued. Subjects in the experiment had on average 19.7 years of experience as auditors, with a range from 12 to 40 years. One-half of the auditors had experience in the high technology medical products industry, the other half had experience in the retail industry.

Each of the four cases is based on the annual report produced by a publicly traded firm, abridged to a length of approximately 15 single-spaced pages. All cases describe real companies. Two cases (Pharmaceuticals, Inc. and Surgical Products, Inc.) describe medical products companies. The other two cases (America's Family Video Store and Big John's Electronic Sales, Inc.) depict retail companies. The names of the companies and other identifying information were altered to avoid recognition.

Subjects were not informed that the cases they would examine contained financial statement frauds. The frauds present in the four cases were constructed by management and were missed by the accounting firm responsible for conducting the audit of the company's financial statements. The frauds were designed to increase reported income and assets, and range from overstating the value of the inventory to improperly capitalizing costs that should be expensed. A brief description of the cases and their manipulations is presented in Appendix A.

In addition to the four fraud cases, each subject was also given a nonmanipulated (`clean') case as an experimental control designed to check for the frequency of `false positives,' that is, cases where the subject feels that a fraud is present when there is none. Audit partners with experience in the medical products industry received a clean medical industry case (Vascular Products) as their first case, followed by the four fraud cases. Audit partners with experience in the retail industry received a clean retail industry case (Southern Retail) as their first case. The four fraud cases were given to all subjects in the same order: America's Family, Pharmaceuticals, Big John's, and Surgical Products.(1)

All subjects worked on the cases in individual sessions held in their offices. Each subject was given a set of written instructions in which they were asked to conduct a second-partner review of each case and to evaluate the appropriateness of the engagement partner's conclusion that the case presented no material issue (an unqualified audit opinion).(2) Upon completion of the five cases (four fraud and one clean case) each subject was asked to rate the appropriateness of issuing the six alternative audit opinions listed above on each case using a 7-point scale. The `unqualified' opinion is the correct solution for the two clean cases. Non-GAAP is an acceptable solution and `adverse' is the best solution for the four fraud cases.

Since we are interested in understanding how auditors detect deception, as opposed to modeling technical differences among different auditing opinions, we assigned the opinion chosen by the auditors to one of three outcomes: `Misleading,' `Unqualified+,' and `Unqualified.' `Misleading' is used when the auditors chose an `adverse' or `non-GAAP' opinion as most appropriate for the case. `Unqualified+' is used when the auditors chose to issue a lack of consistency, going concern, or material uncertainty paragraph. `Unqualified' is used when the auditors chose the clean opinion.(3)

The process of thinking used by the auditors was captured by instructing each subject to think-aloud (Ericsson & Simon, 1993) while performing each review task. All sessions were tape-recorded, transcribed and coded according to a scheme developed in previous research on the fraud detection process and described in Appendix B (see also Jamal, 1991; Johnson, Grazioli & Jamal, 1992).

The category of outcome for the opinion given by each subject on the four fraud cases and the clean case is presented in Table 1. Inspection of the data in Table 1 reveals that detecting financial deception is a difficult task. Two subjects failed all fraud cases, and eighteen (out of twenty four) failed at least two cases. Using a strict definition of success (i.e., giving an adverse opinion) no subject was successful on all four fraud cases and twenty subjects failed at least three cases. Seven subjects also failed to give an unqualified opinion on the clean cases. The responses of these subjects are discussed in a later section of the paper.

 
Table 1 
Audit opinions given by twenty-four auditors 
on four fraud and two clean cases(a) 
 
                                             Retail fraud 
               Medical fraud cases              cases 
 
                              Surgical 
Subject(b)   Pharmaceutical   products       Big John's 
 
S1           Misleading       Misleading     Misleading 
S2           Misleading       Misleading     Misleading 
S3           Misleading       Misleading     Unqualified+ 
S4           Misleading       Misleading     Unqualified+ 
S5           Misleading       Unqualified    Misleading 
S6           Misleading       Unqualified+   Unqualified 
S7           Misleading       Misleading     Misleading 
S8           Unqualified      Misleading     Unqualified+ 
S9           Unqualified      Unqualified+   Misleading 
S10          Misleading       Unqualified+   Unqualified+ 
S11          Unqualified      Misleading     Unqualified 
S12          Unqualified+     Misleading     Unqualified+ 
S13          Misleading       Unqualified+   Misleading 
S14          Misleading       Unqualified+   Misleading 
S15          Misleading       Unqualified    Misleading 
S16          Unqualified+     Misleading     Misleading 
S17          Unqualified      Unqualified+   Misleading 
S18          Unqualified      Unqualified+   Misleading 
S19          Misleading       Misleading     Unqualified+ 
S20          Misleading       Unqualified+   Misleading 
S21          Misleading       Unqualified    Unqualified+ 
S22          Misleading       Unqualified+   Unqualified+ 
S23          Unqualified+     Unqualified+   Unqualified+ 
S24          Unqualified      Unqualified    Unqualified 
 
             Retail fraud        Clean cases 
               cases 
 
             America's      Vascular       Southern 
Subject(b)   family         products       retail 
 
S1           Misleading     Misleading 
S2           Misleading     Unqualified+ 
S3           Misleading     Unqualified 
S4           Unqualified    (c) 
S5           Unqualified+   Unqualified+ 
S6           Misleading     Unqualified 
S7           Unqualified+   Unqualified 
S8           Misleading     Unqualified 
S9           Unqualified+   Unqualified+ 
S10          Unqualified+   Misleading 
S11          Unqualified    Unqualified 
S12          Unqualified    Unqualified 
S13          Misleading                    Unqualified 
S14          Misleading                    Unqualified 
S15          Misleading                    Unqualified 
S16          Unqualified+                  Unqualified+ 
S17          Unqualified                   Unqualified 
S18          Unqualified                   Unqualified 
S19          Unqualified+                  Unqualified 
S20          Unqualified                   Unqualified 
S21          Unqualified                   Unqualified 
S22          Unqualified                   Unqualified 
S23          Unqualified                   Unqualified 
S24          Unqualified                   Unqualified 
 
(a) `Misleading' indicates that the auditor rendered either an 
`Adverse' or a `Non-GAAP' opinion. `Unqualified +' indicates that 
the auditor felt that the case is fairly represented, yet there are 
areas of concern that need to be explicitly highlighted. `Unqualified' 
indicates a case judged to be fairly represented. Specific 
definitions of these audit opinions can be found in the main text. 
 
(b) Subjects S1-S12 specialize in medical companies, S13-S24 
specialize in retail companies. 
 
(c) Subject S4 judged equally likely a Misleading (Adverse), an 
Unqualified+, and an Unqualified Opinion on this case. 

The data in Table 1 also show that detecting financial deception under the experimental conditions of this study is not impossible. Two subjects (S1 and S2) were successful on all fraud cases. Four subjects were successful on three out of the four fraud cases. Most subjects succeeded in at least one case. Analysis of the think-aloud protocols of the individuals who detected the deception reveals that these successes were not chance events; rather, they were the conclusions of insightful financial analyses. Table 1 also shows that industry specialization does not appear to have a clear effect. While some individuals who have experience in the industry of a case (medical products or retailing) were able to conclude an adverse opinion, so also were individuals who did not have such experience.

The variability present in the data in Table 1 poses a challenge to our ability to develop a single theory that explains the behavior of the subjects who perform the fraud detection task successfully, as well as those who fail. We tackle this problem in two steps: first, we construct a theory of successful detection of financial deception, and second, we expand the theory to account for failures to detect.

3. A theory of detecting deception in financial information

The nature of financial statement fraud poses two constraints on the knowledge that is developed to support success at detecting this form of deception: 1) infrequent occurrence of successful detection, and 2) the presence of an adversary (i.e., management). Building a theory of the knowledge that supports successful performance on infrequently occurring tasks is challenging because a low rate of occurrence rules out explanations of performance based on direct and repeated experience (e.g., Ericsson, 1996; Larkin, McDermott, Simon & Simon, 1980). Since fraud detection occurs infrequently in the professional life of a typical auditor (Loebbecke, Einning & Willingham, 1989), we must propose a theory of success based on knowledge that is partially independent from specific experience.

The presence of an adversary distinguishes fraud detection from tasks in which a cognitive agent generates and tests hypotheses about a malfunction in some system of interest (e.g., Clancey, 1988; Johnson, Kochevar & Zualkernan, 1992). While most diagnostic tasks (e.g., medical diagnosis, electronic troubleshooting) entail determining faults generated by unintentional events (e.g., a heart condition, or a faulty component), fraud detection is characterized by the presence of an adversary (management) who intentionally causes the misrepresentation of a company's economic conditions that is, a financial deception (Johnson, Grazioli & Jamal, 1993; Thagard, 1992). Frauds are difficult to detect because they are manipulated so as to appear to be the result of a fair financial reporting process.

We assume that cognitive agents (auditors included) have frequent and varied experience with the phenomena of deception, both as a Deceiver and a Target-that is, victim-of deception (e.g., Ceci, Leichtman & Putnick, 1992; Ekman, 1992; Grover, 1993). As a way to cope with potentially deceiving adversaries, cognitive agents develop knowledge for …