Theory: The Phillips ROI Methodology Theorist: Jack J. Phillips Biography:
Jack J. Phillips is a “world-renowned expert in accounting, measurement, and evaluation” (ROI, 2008). 2008). “Phillips has undergraduate degrees in electrical engineering, physics, physics, and mathematic mathematics; s; a master’s master’s degree degree in Decision Decision Sciences Sciences from from Georgia Georgia State State University, and a Ph.D. in Human Resource Management from the University of Alabama” (ROI, 2008). “This background led Phillips to develop the ROI Methodology, Methodology, a revolutionary process that provides bottom-line figures and accountability for all types of learning, performance improvement, human resource, technology, and public policy programs” programs” (ROI, (ROI, 2008). 2008). Description of Theory:
Donald Kirkpatrick’s Learning and Training Evaluation Theory was developed from Donald Kirkpatrick’s desire to “clarify the elusive term evaluation,” with the purpose of evaluations being to measure the effectiveness of a training program (Kirkpatrick, 1975). In 1997 Jack J. Phillips expanded on Kirkpatrick’s four level sequence which Kirkpatrick used in describing his Learning and Training Evolution Theory, by adding a fifth level which Phillips termed Level 5 – ROI or Return on Investment (Figure 1), a traditional financial measure based on historical date. Phillips added the the fifth level because corporations needed a way of measuring the monetary value of the training (Global Learning Alliance, 2004). Phillips used the framework of the Kirkpatrick’s Learning and Training Evaluation Theory as the evaluation framework for his ROI. A data collection, processing, analyzing
and reporting process was needed; Phillips was able to use the ROI process model. Phillips used the ROI methodology in his development of operating standards. It was the combination of all of these components that allowed Phillips to develop the Phillips ROI Methodology composed of 5 major elements: 1. An evaluation framework 2. A process model 3. Case applications and practice 4. An operating standards and philosophy 5. Implementation The diagram used by Phillips to illustrate the above methodology is a jigsaw puzzle (Figure 2). All 5 pieces must be utilized otherwise the methodology is incomplete. “The Phillips ROI Methodology model provides a step-by-step process for collecting data, summarizing and processing data, isolating the effects of programs, converting data to monetary value, and capturing the actual ROI” (ROI Methodology, 2008). It is essential that when business results and ROI is desired that every step of the process be evaluated. If not it becomes difficult to conclude that the desired results occurred increasing the difficulty of tracking the monetary value of the training. Theory Measurement/Instrumentation:
Phillips uses similar tools to those of Kirkpatrick had a measurement tool for each level. In Level 1 – Reaction: Reaction/Smile/Happy Sheets are used along with verbal reaction and post-training surveys and questionnaires (Kirkpatrick, 1975). Level 2 – Learning: classroom performance, paper and pencil tests, as well as interviews and/or observations (Ely, Plomp, and Plomp, 1996). In Level 3 – Behavior: interviews and
observation performed overtime are required to ensure that a change has indeed taken place (Ely, Plomp, and Plomp, 1996). Level 4 – Results: data collection and retention instruments should already be in place at the company or organization, the data from the process just is simply entered into the entity’s instrument (Kirkpatrick, 1975). Level 5 – ROI: use of a control groups if possible, trending lines, forecasting models, and project estimations performed by the participants, supervisors of participants, senior managers and various experts. (Phillips 2003) Report Prepared By: Jason Prather
References:
Ely, D., Plomp, T., and Plomp, TJ (1996). Classic Writings on Instructional Technology, Volume I. Connecticut: Libraries Unlimited. Global Learning Alliance (2004). The Human Capital Return on Investment. Retrieved September 16, 2008, from http://www.trainingreference.com.uk/training_roi/roi1.htm Kirkpatrick, D. (1975). Evaluating Training Programs. San Francisco: Berrett-Koehler. Phillips, Jack (1997). Handbook of Training Evaluation and Measurement Methods, 3rd Ed. Texas: Gulf Publishing Company. Phillips, Jack (2003). Return on Investment in Training and Performance Improvement. Butterworth-Heinemann. ROI Institute (2008). Jack Phillips, Ph.D. Retrieved September 16, 2008, from http://www.roiinstitute.net/about/roi-team/jack-phillips/ ROI Methodology (2008). The Phillips ROI Methodology. Retrieved September 18, 2008, from http://www.villanovau.com/Home/Content/VU/Program_ROI_Meth.aspx
Figure 1
Figure 2
