EVALUATION: OUR APPROACH
Usability at All Stages of a Project
PRAIRIE’s primary aim is to produce evaluation work that clients and stakeholders will find applicable and useful in improving the quality of their programs. We are interested in evaluating programs across a range of stages:
-
proposal stage
-
early pilot stages
-
scale-up implementation
-
upon project completion.
While the types of evaluations we design vary greatly depending on the stage of a project, we believe that evaluation is useful and usable at all stages. The PRAIRIE group is committed to the direct use of evaluation products and processes. We are interested in working with clients who are likewise committed to the use of evaluation at any and all stages of project development.
Multiple Approaches to Evaluation
To date, some of our most satisfying and fruitful evaluations have taken the approaches of formative evaluation, developmental evaluation, participatory action and collaborative evaluation, and program theory evaluation.
Formative evaluation
PRAIRIE is committed to evaluating programs for purposes of program improvement. Scriven (1996) uses the term formative to describe evaluations that seek information to be used by program leaders, typically with pilot or demonstration projects, to allow improvements to be made and to also serve to set the stage for summative evaluation of the worth of the program. PRAIRIE has evaluated many programs with the foremost aim of helping project leaders improve the specific project.
Developmental evaluation
Multi-year evaluations provide the opportunity for the PRAIRIE group to engage in developmental evaluations. As conceptualized by Patton (1996), developmental evaluation involves ongoing work with an organization for continuous improvement. In a developmental evaluation, evaluators assist the organization and program design team to monitor processes and outcomes over time. The role of an evaluator in a developmental evaluation is to facilitate conversations about the evaluation, data, logic, and decision-making about the program and specific projects within the program.
Participatory action research and collaborative evaluation
The PRAIRIE group is also involved in participatory action research and evaluation. In external evaluations, outside researchers collect and analyze data. In participatory action research and evaluation, the evaluators recognize that participants are the experts on their program; the role of the partnering evaluators is to provide training and support in the evaluative process carried out by program participants.
Program theory-driven evaluation
In this type of work, evaluators uncover and develop the program theory that drives (consciously or unconsciously) a program’s activities. Attending to the maturity of a program, evaluators work to understand the underlying mechanisms that link program activities to program outcomes.
Theoretical Orientation
In addition to the influences of Scriven (1996) and Patton (1996), PRAIRIE evaluations draw from the three major branches described by Alkin and Christie (2004) in their family tree of evaluation theory: Use, Method, Valuing.
Use-oriented evaluations focus on the use of evaluation in decision-making. Method-oriented evaluations emphasize obtaining generalizability and knowledge construction through evaluation. Valuing-oriented evaluations consider the perspectives being represented in the chosen outcomes and in the findings. As Mark (2007) has noted, there are many evaluations and evaluators that belong in tree houses situated and supported by all three of these branches. While the PRAIRIE tree house is most closely aligned with the use-oriented branch, it requires and welcomes the foundational support from method- and valuing-oriented evaluation theory.
REFERENCES
Alkin, M.C. (Ed.) (2004). Evaluation roots. Thousand Oaks, CA: Sage.
Chen, H-T. (2005). Practical program evaluation: Assess and improve planning, implementation and effectiveness. Thousand Oaks, CA: Sage.
Cousins, J.B. & Earl, L.M. (Eds.) (1995). Participatory evaluation in education. London: Falmer Press.
Mark, M. (personal communications, 2007). The Evaluator’s Institute course Informing Practice Using Evaluation Models and Theories. April 23-25, 2007. Chicago, IL.
Patton, M.Q. (1997). Principles of participatory evaluation. Utilization-Focused Evaluation: The New Century Text (3rd ed.). Thousand Oaks, CA: SAGE.
Patton, M. Q. (1996). A world larger than formative and summative. Evaluation Practice, 17(2), 131-145.
Scriven, M. (1996). Types of evaluation and types of evaluator. American Journal of Evaluation, 17 (2), 151-161.
|
|