Subjective ROI. (return on investment)

Subjective ROI

The concept of "return on investment" (ROI) is old hat in the business world. But until recently, its use in evaluating the effectiveness of management-training programs had been rare.

More and more, management is asking for evaluation that measures not only the practical utility of a program, but its financial utility and proven benefits as well. A growing need to justify the extensive management-training budget of Hughes Aircraft Company prompted its technical trainers to find a way to measure the overall effectiveness of training.

The following case study describes our evaluation and assessment of two existing training programs at Hughes. We asked supervisors to rate their employees' performance in each. We then used the results, which were quite impressive, to evaluate improvement and to estimate ROI. Estimating ROI posed a new challenge for our company, as it has for so many others. Described here are the procedures and rationale for data collection, along with three methods of calculating ROI.

How to measure on-the-job

performance

Training professionals differ on the difficulty of developing job-performance measures. Some say it's easy to develop objectives for skills training, but harder to objectify and measure learning in management-development programs. Having to justify costs and benefits of management-training programs can make forming realistic and quantifiable measures of on-the-job performance tough, they say.

The general literature, unfortunately, largely disagrees with this view and offers many simplistic, sometimes trivial, examples of how "easy" it is to measure job performance. Becker, for one, in an article in the book Designing and Delivering Cost-Effective Training-- and Measuring Results (1977), says that such things as leading, problem-solving, deciding, analyzing, coaching, and goal-setting are "learnable, measurable managerial skills." Still, certain categories of managerial job performance are difficult to measure specifically, objectively, and quantifiably.

Therefore, some approaches for estimating ROI are based on subjective data, such as supervisors' opinions of employee performance. While less than ideal, subjective data can provide credible estimates of ROI. In these situations, one must describe clearly all assumptions and procedures that apply to the data collection, analysis and interpretation. Given such information, the people reviewing and evaluating the results may decide how credible the findings are, using their own criteria to determine the practical and financial utility of the particular training program.

Assessing employee

performance

Participants. For this study, we selected 20 participants from non-management personnel in …