Multilevel models in model-driven engineering, product lines, and metaprogramming.

INTRODUCTION

Over the last decade, there has been an increasing desire in both research and practice to abandon the manual development of programs in favor of more automation. (1) Work on software product lines is an example. (2,3) A product line is a family of similar programs. Individual programs differ by the features that they support, where a feature is an increment in program functionality. By modularizing features, programs in a product line are produced by composing features (4); that is, the process of developing a complex program can be reduced to the comparatively simple activities of feature selection and composition. Software tools automate the composition process.

More broadly, research on product lines and generative, transformational, and component-based programming (5) are progressing toward the goal of making programming a computation. This requires a fundamental shift in perspective on program design and development. Programs themselves become objects, and operations on programs are methods of such objects. Metaprogramming is the concept that programming is a computation.

Model-driven engineering (HIDE) is an emerging approach to software development that centers on higher-level specifications of programs in domain-specific languages (DSLs), greater degrees of automation in software development, and the increased use of standards. (6) Among the tenets of MDE is the use of models to represent a program. A model is a specification written in a DSL that captures particular details of a program. As an individual model represents a limited range of information, a program specification is often defined by several models. A model can be computed from other models, and the process of building programs is one of transforming high-level models into executables (considered as yet other models).

Although MDE and metaprogramming are not identical, they do share concepts and goals; namely, that programs are first class (i.e., as objects or models), operations can be performed on them (i.e., as methods or transformations), program development can be a computation, and programs have multiple representations.

In this paper, it is argued that product lines enable both MDE and metaprogramming to converge on a multilevel paradigm of program design. This paradigm not only uses object oriented (OO) design techniques to represent programs that manipulate everyday objects (e.g., employees, books, ledger sheets), but also uses OO techniques to represent the metaprograms that produced these programs, and the meta-metaprograms that produced these metaprograms, recursively. The paradigm is based on a small number of simple and well-known ideas, scales to the synthesis of applications of substantial size, and helps clarify concepts of MDE.

MULTILEVEL DESIGNS

Long before MDE, software engineers realized that a program has many different representations, a simple Java ** application, . class files, and . html files (produced by the javadoc tool). (7) A more elaborate application might have performance models (represented as Mathematica files), makefiles (in Extensible Markup Language [XML] format), formal models (as a state machine in an XML Metadata Interchange [XMI] file), and so on. Each representation is written in a language specific for its purpose--Java is good for source code, HyperText Markup Language (HTML) is good for documentation, and so forth.

Representations can be derived from other representations (e.g., a . class file is derived from its corresponding . java file by javac), or a representation may express unique information about a program (e.g., Mathematica files define a performance model that is not automatically derivable from source files). From this perspective, the software engineering community has been practicing a primitive form of MDE for years.

If we treat files (i.e., representations) as objects that are instances of file types, an OO design emerges. Figure 1 identifies the file types that are encountered in the development of a Java program. File methods are implemented by Java tools that either transform a file into another representation (e.g., .javac is a method that maps a .java file to a .class file) or that modifies the file (e.g., reform is a pretty printing tool that transforms unruly .java files into beautifully formatted files (4)). Even inheritance relationships exist: operating systems provide a standard set of operations (move, copy, delete) on files of all types. Specialized file types are distinguished by different file extensions, and have their own methods (tools).

[FIGURE 1 OMITTED]

A makefile is a program that operates at this level of abstraction. It consists of one or more scripts that create objects (i.e., files) by invoking methods (i.e., tools) in a particular order, and whose goal is to maintain the consistency of these objects whenever an object is modified. (In effect, a makefile is a metaprogram--a program that produces a program.) Makefiles are written in a special language that is not object-oriented, but that could be given a class structure. Figure 2A shows the skeleton of an ant build script, and Figure 2B shows its corresponding class structure (i.e., an ant project is a class, ant tasks are methods, and property definitions are members). Although this correspondence is loose, it is not difficult to recognize an OO class structure in other non-Java artifacts as well. We will return to this observation later in the section entitled "AHEAD."

[FIGURE 2 OMITTED]

Given the above, there are clearly two different levels of abstraction in program design. The meta-application level deals with the construction, manipulation, and synthesis of application level artifacts (files, etc.). Although there are OO languages to express programs at the application level, there is no language that unifies the concepts of files with objects, file instances with file types, tools with methods, and execution scripts (e.g., makefiles) as bodies of methods. (Perhaps Smalltalk and Lisp are exceptions, as they were both programming languages and programming environments.) Not surprisingly, programs at the meta-application level are developed by using tools and design techniques that are reminiscent of those used at the application level in the 1960s. The primary reason that OO techniques now dominate programming is that they impose more structure on programs. More structure means program complexity is better controlled, accidental complexity is reduced, and greater opportunities for automation and analysis arise (e.g., tools that restructure programs using design patterns). On the other hand, programs at the meta-application level are very simple--almost trivially so--compared to their counterparts at the application level. Perhaps this, for no other reason, explains why tools and design techniques at the meta-application level have not progressed.

MDE may be a driving force to change this. MDE explicitly embraces the idea that programs have multiple representations, thus complicating activities at the meta-application level. There are potentially many more program representations, called models, to keep track of; there are many more operations that can be performed on models to modify them or to …