Formative assessment of classroom concept maps: the reasonable fallible analyser.

Concept mapping is a powerful learning technique that can be enhanced by computer technology. Software tools are already available for the preparation of concept maps but as yet, few systems provide feedback on their content. The claim made by this article is that by enlisting the student as an assessment partner, computer-based feedback becomes both feasible and useful. The article reports on the development of a semi-automated software analyser for formatively assessing students' concept maps. The analyser is fallible: many of its assessments are initially incorrect. Crucially however, it is also reasonable: a student is able to argue for a more favourable assessment. The article reports classroom trials which indicate that high-school students' experience of concept mapping is enhanced by use of the analyser. The system's wider significance is that it points towards a new genre of educational software based upon a more genuinely dialogic model of human-computer interaction.

**********

A concept map is a graphical representation of knowledge that uses a network of symbols to describe relationships between ideas. Typically, each linked pair of symbols specifies a proposition. Based on the constructivist theory of meaningful learning (Ausubel, Novak, & Hanesian, 1978), concept mapping was developed as a learning technique (Novak & Gowin, 1984). By articulating their understanding and rendering it in visual language, learners engaged in making a concept map are encouraged to reflect upon and refine their knowledge.

Like most kinds of documents, concept maps can benefit from technological support. Currently available software includes Conception (http://www.parlog.com), CmapTools (http://cmap.ihmc.us/), and Inspiration (http://www.inspiration.com). Such tools enhance the preparation, revision, and utility of concept maps in many ways, for example by enabling a concept map's symbols to act as hyperlinks to other concept maps, web pages, and multimedia resources. Until recently, however, programs have done almost nothing to provide students with feedback on the quality of their maps. Given the general importance of feedback in learning, this is a significant omission.

Obviously, a fully intelligent or infallible analyser for unrestricted concept maps--one that can evaluate a student map on any topic and generate unerringly authoritative and insightful suggestions--is beyond the state of the art in artificial intelligence. Such a program would amount to a linguistic prodigy and a universal sage. The claim made by this article however, is that useful forms of computer-based feedback do become feasible when a student's role is promoted from a passive recipient of assessment to an active partner in the process. The article reports the development of a semi-automated software analyser that shows some promising results. The analyser, titled the Reasonable Fallible Analyser (RFA), compares a student's concept map to a presupplied expert's map for the same domain. It generates a score for students' maps and offers hints for improvement. Owing to its limited intelligence, the RFA's initial assessments are not expected to be highly reliable. However, a design goal has been to make a virtue of the system's fallibility. The system's misconstrual of a student's map is taken as an opportunity to initiate an argumentative dialogue, which may benefit the student's learning as well as improving the accuracy of the assessment.

The remainder of the article is in five parts. First, the motivation and general aims for this research are described. Second, a brief review of relevant previous work is provided. Third, the design and functionality of the analyser is reported. Fourth, some classroom experiences are reported that are based on trials with the RFA in an Edinburgh high-school. The trials indicate that high-school students' experience of concept mapping is enhanced by the RFA. Students enjoy arguing with the system, accept its scoring as fair, welcome its hints, and are frequently stimulated to revise their maps to accommodate the feedback obtained. Finally, some discussion is offered around the content of this work, including its relationship to the wider context of assessment, pedagogy and concept mapping.

MOTIVATION AND AIMS