The publication of the new 5th edition of Utilization-Focused Evaluation provides an opportunity to revisit the interconnection between Blue Marble Evaluation and Utilization-Focused Evaluation. In essence, Blue Marble evaluations should be utilization-focused so Utilization-Focused Evaluation provides the overarching framework for Blue Marble Evaluation. This blog summarizes how we address the interconnection in the new book.
The new edition integrates Michael’s 50 years’ experience with Charmagne’s perspective as a second generation evaluator, bringing attention to emergent trends and issues with new questions and perspectives. Together we are committed to engaging in evaluation in ways that contribute to a more just and sustainable future. That has led to a complete rewrite and revision of the book incorporating Blue Marble Evaluation for the first time.
The Evolution of Utilization-Focused Evaluation
The 1st edition of Utilization-Focused Evaluation published in 1978 featured early research on factors consistently affecting use: (1) political considerations and (2) "the personal factor” -- the presence of an identifiable individual or group of people who personally care about the evaluation and the findings it generates. Where such a person or group was present, evaluations were used; where the personal factor was absent, there was a correspondingly marked absence of evaluation impact.
The 2nd edition (1986) offered for the first time a definition of utilization: “intended use by intended users.” This focus proved to have implications for everything done in an evaluation and every decision made along the way. Working with primary intended users to enhance intended uses became the hallmark of utilization-focused evaluation.
The 3rd edition (1997) introduced process use. Up until that time the entire field had narrowly focused on use of findings. This idea of process use draws our attention to individual changes in thinking and behavior among those involved in the evaluation as a result of the learning that occurs during the evaluation process.
The 4th edition (2008) gave significantly increased attention to international and cross-cultural factors that affect use and brought systems thinking and complexity theory into utilization-focused evaluation.
The Ongoing Evolution of Utilization-Focused Evaluation
This new 5th edition for the first time presents and elaborates principles of utilization-focused evaluation. Principles guide application and adaptation. Principles connect theory with practice, questions with methods, and findings with follow-through actions. The 21st century has brought major changes in the world and, correspondingly, in the evaluation profession, like the global adoption of Sustainable Development Goals for Agenda 2030 that expresses a commitment to equity and sustainability. This new edition presents for the first time the role of utilization-focused evaluation in addressing issues of equity and sustainability in the context of the COVID-19 pandemic, the climate crisis, and the global social justice uprising. As evaluators, we have a stake in having our evaluations used—and we have a stake in a more equitable and sustainable world. The implications for utilization-focused evaluation are substantial, dramatic, controversial, and forward-looking.
The Ongoing Evolution of Evaluation Practice
Utilization-focused evaluation as part of the profession and transdiscipline of evaluation has been both influenced by the evolution of evaluation practices and contributed to those practices. Evaluators have come to excel at evaluating projects and programs. We know how to specify SMART goals (specific, measurable, achievable, relevant, and time-bound) and develop performance indicators. We have become skilled at doing evaluability assessments, developing logic models and theories of change, conducting meta-evaluations. We know the importance of distinguishing monitoring from evaluation and how to do so. New approaches to evaluation are proliferating along with traditional uses. The field has standards for what constitutes evaluation quality and checklists for what should be included in an evaluation. There is general agreement about the importance of specifying intended use by intended users, being culturally responsive, and adhering to high ethical standards. Evaluation has largely moved beyond the qualitative–quantitative paradigm debate and come to value mixed methods. Evaluators have a variety of ways of reporting findings with increased emphasis on data visualization. This is by no means a comprehensive or exhaustive list, but, hopefully, it provides a sense that evaluation has developed into a knowledge-and-skills based profession and discipline.
The evaluation profession continues to develop. Evaluators recognize the importance of being competent at more than methods, though that remains the focus of most training. We’re still figuring out how to train for other professional competencies like interpersonal skills, reflective practice, project management, adherence to standards and guiding principles, and building capacity for cultural competence and responsiveness. Likewise, substantial attention is being devoted to incorporating systems thinking and complexity concepts, facilitating diverse and conflicting values, providing rapid feedback and real-time data, and treating failure as an opportunity for learning. We know that evaluations should search for unanticipated consequences and side effects, but too few designs include adequate resources and open-ended fieldwork to actually do so. For too long, the field of evaluation has centered white, male, heteronormative values while purporting to be neutral. Evaluators are getting better at naming and elevating ethical frameworks and making underlying values explicit, but these directions need improvement and further development. Again, this is by no means a comprehensive or exhaustive list, but the field is continuing to develop, innovate, and adapt to deepen and expand our relevance, utility, and excellence as a profession.
The emergent challenges for evaluation have to do with new units of analysis and broader areas of focus for evaluation. Beyond project and program evaluation, evaluators are being asked to evaluate community impacts, regional and sector-wide initiatives, cross-sector interventions, networks and collaborations, social innovations, and collective impact, all of which pose new conceptual and methodological challenges. Evaluators are being challenged to develop new approaches to scaling innovations, assessing the effects of social media, and using “big data” to examine large and open systems. We are also being challenged to reckon with the legacy of colonization and whiteness in the field and to decenter whiteness and decolonize our practice. In essence, evaluation has evolved into a diverse and vibrant field with a great variety of approaches, methods, applications, and uses.
Updating Utilization-Focused Evaluation
As the profession has developed so has utilization-focused evaluation. U-FE offers a comprehensive decision framework for designing and implementing an evaluation to fit a particular situation and, in that situation, meeting the information needs of primary intended users to enhance their intended use of the evaluation primary intended users. This means that utilization-focused evaluation encompasses every evaluation option methodologically, conceptually, theoretically, analytically, and process-wise. Any evaluation methods and approaches can be used in a utilization-focused evaluation. In essence, the utilization-focused evaluation doesn’t prescribe what to focus on but rather prescribes a process for determining what to focus on based on an unwavering attention to intended uses by intended users.
Blue Marble Evaluation emerged from U-FE as an approach to evaluating global systems change efforts. Blue Marble Evaluation is developmental and principles-focused, incorporating global and glocal thinking in support of systems transformation. Blue Marble Evaluation also positions evaluators as having skin in the game where the game is the future of humanity. Having skin in the game means that where issues of equity and sustainability are concerned, evaluators are not outside the global system looking in at what is unfolding; rather, evaluators play a role in how the future unfolds and have a stake in that future. That makes Blue Marble Evaluation utilization-focused and requires that utilization-focused evaluations treat global (Blue Marble) issues as context for all evaluation designs.
Bottom line: Utilization-Focused Evaluation and Blue Marble Evaluation are interdependent and mutually reinforcing in support of a more just and sustainable future.
Official launch of the 5th edition on November 17
We invite you to join us for the official launch of the new book on November 17. Click here to register.