Evaluating Agriculture Systems Change
Introduction
A seminal 2019 report by the Lancet Commission entitled Food in the Anthropocene: the EAT–Lancet Commission on healthy diets from sustainable food systems states:
Because much of the world’s population is inadequately nourished and many environmental systems and processes are pushed beyond safe boundaries by food production, a global transformation of the food system is urgently needed.
The CCRP has a mission of “helping smallholder farmers feed their world”. It is deeply interested in how this mission contributes to the global need for and efforts on food systems transformation, which is a Blue Marble concern.
Background Information
While the CCRP’s history dates back to 1983, it didn’t start using an explicit evaluation strategy until 2008 when the new program director, Jane Maland Cady, decided to begin a utilization-focused and complexity-aware approach to evaluation that was dubbed “Integrated Monitoring, Evaluation, and Planning”, IMEP for short. Marah Moore, the director of i2i Institute, was brought in to lead the initiative, with Michael Patton and Glenda Eoyang joining as thought partners who helped shape IMEP.
Over the years, different strategies have been used to ground IMEP in the regions where the CCRP works. Specifically, the CCRP has worked in 7-9 countries in Africa and three countries in Andes, with each region organized into a Community of Practice. Initially, local evaluation consultants were hired to work with CCRP projects. However, this approach effectively siloed evaluation from all the other work that the projects were doing, especially their participatory research efforts. Eventually a model emerged where each region’s CCRP representative took on the responsibility of facilitating IMEP at the project and CoP levels (micro and meso), in addition to their roles related to grantmaking and grantee support. The main author of this case is the regional representative for the Andes, and also works directly for i2i on IMEP activities.
Click here to view a map of all CCRP sites.
The CCRP underwent a major shift in the early 2000s as it transitioned from a plant biology program that connected plant breeders in Northern and Southern countries, to one formed around communities of practice that emphasized participatory research with smallholder farmers. This shift was informed by a 1998 evaluation which inspired a more inclusive and decentralized approach to grantmaking and support.
The timeline of the program shows how regional, program, and external/global levels (micro, meso, and macro) interact to exchange ideas and learning: CCRP Timeline
Figure 1: CCRP ToC 2014-2018
The new ToC (Figure 2) more clearly incorporates transformational concepts (BME principle #3) at a global scale (#1) and showing how those are anchored to local processes (#6).
Figure 2: CCRP Program ToC 2019–
The new ToC makes explicit how local initiatives can contribute to something bigger. This, in turn, has allowed the CCRP be more intentional in its grantmaking and grantee support.
One example is a long-term project (since 2000) on quinoa in Bolivia. In the last iteration of its project ToC, it began to identify how its work connects to global initiatives to sequester carbon through breeding for perennial quinoa and reduction in agrochemical use through research on biopesticides. Just as importantly, the infinity symbol in the center of Figure 2 illustrates how global efforts also benefit from local learning and knowledge. For instance, many wild parents of modern crops are perennial and have been used for centuries as a source of food in times of famine. It is important that the global not be seen as just extractive, or in a “scaling-up” relation to the local, but rather in a symbiotic relationship.One challenge is that there is often a tradeoff between working with global agroecological frameworks, such as those promoted by CIDSE or FAO— or even the SDGs, versus facilitating a participatory approach to constructing local and endogenous systems of meaning. A strategy for negotiating this tradeoff is to use evaluation to translate between micro, meso, and macro models. So, in the example above, the quinoa project did not explicitly mention contributing to SDG #13 on climate change. Rather the CCRP representatives brought the need to connect to global frameworks to the dialogue and the project team and the CCRP discussed and negotiated methods that could be used to measure carbon sequestration for local, regional, and global audiences and outputs.
This case shows the importance of BME, and in particular, the GLOCAL (global-local) principle in terms of being able to ground change in actual practice. Change, by definition, is always manifested among local people and places. Evaluation practices, such as connecting local initiatives to global frameworks, help to visualize and shape local change so it can contribute to global outcomes through the use of shared indicators or questions. This case explores the role of the Blue Marble Evaluator as a facilitator and knowledge broker between different levels and knowledge paradigms; the Blue Marble Evaluator is a connector. An essential part of network theory is that not every actor needs to be connected, there are different nodes and bridgers. Thus, not every local stakeholder might use the same language of “global systems change”, but there are learning loops that knit together the local and global.
The CCRP does not ask grantees to use any specific indicators or frameworks in their work and instead defers to the projects’ understanding of their needs and resources. Increasingly, the CCRP is trying to curate and foment sharing of frameworks and indicators across projects and other global efforts — to help local initiatives understand their contribution to global impacts and be more efficient. This involves sharing promising resources with grantees, encouraging them to test them out, and share their reflections with and across communities of practice.
Simultaneously, the IMEP team applies global frameworks to current and past projects to analyze our evolving portfolio in order to better assess our contribution to global agroecological change and also shape our grantmaking and support. For example, the CCRP portfolio since 2008 was analyzed using a global rubric developed by Biovision and based on the FAO agroecology elements. One finding is that, in general, there have not been dramatic shifts in the portfolio over the years in terms of levers, or transition level as used by Biovision. The elements that have always been high (between 71-85% of projects) are knowledge sharing and diversity (72%-82% of projects). This evaluative insight led us to conclude that in terms of fidelity to a model, CCRP projects are fairly agroecological. The next thing the CCRP wants to evaluate is if these activities actually lead to increased agroecological performance at the farm, community, landscape, foodshed, region, and even global level.
The evaluation also revealed that across 54 Biovision indicators, the CCRP has had projects that touch all of them except for three: reducing energy use through renewables, introducing domestic pollinators, climate mitigation through redesigned system. This has pushed the CCRP to reflect on how it can contribute to global climate change mitigation through carbon sequestration and reducing GHG emissions.
The CCRP also used the results from the Biovision assessment to conclude that while the portfolio might have many agroecology elements, they are often siloed and not part of a systems perspective. In response, an internal AEI research capacity index was developed to assess to what degree projects “explore feedback loops and/or interactions between natural and human systems, thresholds, and non-linearity.” The results of that evaluation show that the capacity to do this has increased with time, which, in turn, validates and informs the CCRP’s commitment to capacity strengthening.
Continuing with the example of carbon sequestration, the CCRP’s models and projections could help to build Payment for Ecosystem Services programs for farmers who are helping to keep carbon in the ground, as many smallholder farmers do, and even sequestering additional carbon. This, in turn, could contribute to more adaptation and use of these practices to transform agriculture from a massive emitter of greenhouse gases to a source of mitigation.
Change is constantly happening; the program needs evaluation to capture that change and learn how it happens under different circumstances, in order to accelerate positive change. The CCRP contributes to the knowledge about how a diversity of smallholder farmers in Latin America and Africa are transforming or have transformed agriculture systems, and, at the same time, the CCRP participates in global conversations, with actors like the Global Alliance for the Future of Food and FAO, about how to use these models to stem the damage of industrial agriculture practices.
Conclusion
To move beyond the threats that the Anthropocene presents to our planet, the agriculture system at a global level needs to change. There is much that can be learned and leveraged at the local level, and a Blue Marble Evaluation stance helps to envision new ways that growers and consumers, as well as all the actors in between, can participate in a sustainable and productive agriculture system.
