What ‘Good’ Means

Evaluation is integral to social change, but it must evolve. I first encountered the problem at a youth shelter, where “impact” was measured in beds filled and youth served. But conversations with the young people revealed far more was at stake: disrupted schooling, health struggles, family conflict, survival jobs, identity questions, and friendships through it all. Making a difference in their lives meant a lot more than a temporary place to sleep. I began to wonder: How could evaluation not just measure, but shape systems change and redefine success?

One answer is to listen and incorporate the values of those affected. Though my first evaluation taught me that this is essential, but not enough. In Illinois, we assessed a coal education program involving teachers, government staff, environmental activists, and industry representatives. Their values diverged sharply—some even wanted the program shut down. That experience showed me that evaluation must go beyond inclusion to create space for deliberation about values themselves.

This excerpt from our Evaluative Inquiry for Systemic Change A Guide to Shift Beyond Fixes to Lasting Value introduces the idea of value deliberation and why it matters for systemic change. While conceptual, it challenges evaluation’s technical legacy and sets the stage for the rest of the book, which offers practical guidance on leading systemic change and embedding evaluative inquiry, a way to explore, evidence, and develop value.

We wrote this book for funders, practitioners, and evaluators seeking to reposition their roles in large-scale systems. As scholars and practitioners—Emily in the United States and Pablo in Spain—we work internationally across education, public health, and food systems. This book shows how evaluation can evolve to address the complexity of social-ecological problems, offering practical processes for shaping and evaluating systemic change collaboratively.—Emily F. Gates and Pablo Vidueira

* * *

We engage in systemic change to understand the world around us and make it better. Value is central to systemic change. Value is about why something matters, its worth, and its significance within a context. Evaluation means understanding and assessing value. Yet, among the many known systemic approaches to evaluation, nearly all sidestep how to assess value. Many practitioners conceptualize, model, describe, and/or explain systems with few or no suggestions on appraising the value of a system or changes to a system.

While it may be tempting to ask whether a system has changed, it is a question that makes no inherent sense. Systems are always changing. Whether it has changed for the better depends on what changes are considered better (and worse). This can be likened to a road trip without an inherent end destination.

What a “good” systemic change means is an open question, often asked but rarely answered. The United Nations Development Program has a Measurement and Evaluation Sandbox to share novel ideas and practices. One blogpost asks, “What is ‘good’ systems change and how do we measure it?” The author, Søren Vester Haldrup (2023b), discusses the importance of this question, while showing the trickiness of answering it. The trickiness lies in the need to inclusively deliberate what good and bad mean in the context of a systemic change process and the difficulty of measuring and gathering data that truly captures what matters.

We take this further to contend that defining “good” systemic change raises a different kind of question and depth of trickiness than defining a good policy or program. As explored in Chapter 1, when an intervention is considered a fix to a social problem, evaluating becomes a matter of determining value in relation to whether the intervention fixed the problem. With systemic change, we need to know how good our efforts are to adjust amid complex and shifting circumstances. This requires a shift in what value means and how we approach defining and measuring it.

This chapter explains why questions of value are central to systemic change and how evaluative inquiry can facilitate a collaborative process to develop value. Evaluative inquiry is an ongoing process of asking and answering questions about value. We argue for a critical deliberative stance involving discussion of and across multiple values, perspectives, and ways of knowing, as well as careful consideration of inclusion, exclusion, and marginalization. We dispel misconceptions about evaluating systemic change.

Shift From Determining to Developing Value

Here we contrast a typical approach to evaluating value that supports “fixing” social problems with evaluative inquiry, as centered in this book. Whereas determining value serves a point-in-time judgment about an intervention (Scriven, 2012), developing value serves open-ended, collaborative inquiry. Both are supported by criteria that define what value means and evidence to assess whether and how well criteria are met. The difference lies in deeper assumptions about the nature of value and who and what evaluating value serves. Our discussion of this contrast draws on the work of Thomas Schwandt, a philosopher of evaluation with whom Emily coauthored a book. In that book, they proposed and argued for expanding “value” beyond something determined by experts to assess interventions (Scriven, 2012) via continuous, collaborative process to amplify the value of change efforts in unfolding circumstances (Schwandt & Gates, 2021; Gates et al., 2024). We also draw on work that positions value as something to be created—not extracted (Mazzucato, 2018)—through deliberative processes that involve individuals and groups clarifying, and sometimes shifting, personal values to co-shape shared value (Milstein, 2008; Kenter et al., 2019).

Four features anchor our vision of evaluative inquiry to develop value together: (1) from value as predefined to we-defined, (2) from fixed and singular to open and multidimensional value; (3) from retrospective to prospective; and (4) from technical to moral-political process. In Table 2.1, we contrast these with assumptions about value traditionally made when evaluating interventions as “fixes” to social problems.

From Value as Predefined to We-Defined

In a “fixed” approach to social change, interventions target specific social problems and needs. The value of an intervention is predefined, typically by people and organizations with power in relation to an intervention’s goals and design. A good intervention accomplishes its goals, achieves intended outcomes, and ameliorates or contributes to amelioration of the targeted problem or need.

In systemic change, value development is a collaborative process among people (and often nonhuman species and places) bringing inherently diverse views regarding what value currently means and should mean. Developing value begins with an open stance around what value means and intention to include multiple perspectives, human and nonhuman, with critical awareness and transparency regarding choices about who or what to privilege and exclude (Schwandt, 2018).

Phrases that capture this diversity include Hannah Arendt’s (2018) “plural we” and Arturo Escobar’s (2018) “pluriverse.” Arendt (2018) asserts, “We are every one unique, not variations of a kind” (p. 220). Plurality is fundamental to what it means to be human. Escobar (2018) further recognizes the layers of relationship between humans and non-human species and ecologies: “The pluriverse, a world where many worlds fit, as the Zapatista aptly put it . . . includes difference in the biological realm (biodiversity), epistemic difference, cultural difference, and ontological difference” (p. xvi).

Plurally defining value in systemic change offers a way to restore change processes to the public realm, a way to enact what it means to live together as a democracy by engaging in collaborative social practices (Schwandt, 2018). Those with the most resources and power do not get to decide the direction, scope, and nature of change for all of us. They have no legitimacy to define what counts as success or progress. Engaging in questions of value together is a way to take back the spirit and practice of the public realm (Arendt, 2018). Escobar (2018, p. 7) calls this “a transition toward plural ways of making the world.”

Seeing value as something by and for plurality aligns with well-established democratic orientations to evaluation, as envisioned by Jennifer Greene (2005), “Evaluation conducted for the public good, that is, aims to enlighten the relevant policy conver sation and especially to include therein diverse voices, perspectives, and experiences (p. 8). It also aligns with value-based evaluations for transformative change which centers what matters to the people and places directly involved and affected (Aronsson & Hassanain, 2019).

This suggests a relational mode in which, according to Thomas Schwandt (2019b), “politics returns to the people, to the sphere of everyday practices, interactions and understandings to viewing democracy not as composed of all the features of formal politics but as a way of life—as John Dewey put it” (p. 10).

Acknowledging plurality and committing to engage with difference is easier said than done. Power dynamics are unavoidable, as are value conflicts. “We need to address the question of ‘value from whom, created by whom, for whom,’ with an understanding of the issues of power and equity underlying the investment, or our investment might fail or even do harm. We need to take great care when defining what is valuable to whom” (King, 2021, p. 5). Balancing the idea of a plural process that explicitly names whose and what values inform criteria (Gates et al., 2024) with critique around potential exclusion and harm is central to what we call “a critical deliberative stance.”

From Fixed and Singular to Open and Multidimensional Value

In a “fixing” orientation to social change, value often gets appraised through quantitative indicators and financial calculations. It is common to isolate specific outcomes and then assess changes before and after an intervention or between a control and treatment group. Whether differences in the outcome are statistically significant, in turn, practically indicates the value of the intervention. In economic analyses, cost benefit analyses and returns on investment consolidate into financial gains or losses. Value gets equated with dollars.

We restore “value” to normative terminology, rather than designate “some actions or outcomes as good, right, or desirable and others as bad, wrong, or undesirable” (Schwandt, 2019b, p. 1). As a normative term, value is inherently open and multi dimensional. To engage in and evaluate systemic change, we must let go of any idea of value as an inherent, fixed attribute that can be discovered or assessed. There is no such thing as a better world, a just system, or a resilient community in an incontestable, objective sense that remains stable over time. Rather, a change process necessarily involves figuring out what value means and should mean. This is a continuous process.

Never static, value is always, by nature, in flux. And yet, pursuits of systemic change require us, individually and collectively, to make judgments about what we envision “better system” to mean. Shaped by a plural we, value is and should be comprised of multiple dimensions. As Davies and Goldie (2023) in “Navigating System Change Evaluation” say, “. . . system change efforts usually demand a richer, potentially more complex definition of success, which may include some tension and contradiction” (p. 6).

Multidimensional value is distinguished by the consideration of multiple dimensions that may or may not be complementary (e.g., social, health, economic, and environmental dimensions); also called blended value. Milstein (2008) contends that a sustained questioning orientation to what value means allows multiplicity to (co)exist. In the context of public health, “standards for judgment tend to examine how simultaneous values like health, dignity, security, equity, satisfaction, justice, prosperity, and freedom are upheld in both means and ends” (p. 2).

We can practice the idea of multidimensional value by intentionally selecting criteria to define multiple dimensions that matter. Rather than top-down conceptualizations, set by funders, criteria should be agreed upon through/during open discussions driven by those directly involved in and affected by change efforts fit for their contexts. See Chapter 5 regarding why we translate value into criteria, options for criteria well aligned with systemic change, and other considerations (e.g., justifying these, given potential alternatives).