People have framed the conversation about measurement in the social sector in terms of monitoring and evaluation for decades. They shorthand it as “M and E” and serve it up as a generic, two-dimensional description for measuring nonprofit performance. Monitoring is the routine data collection and analysis conducted by an organization about its own activities, while evaluation typically means the kind of data collection and analysis conducted by an independent third party.
In many respects, these two complementary parts of measurement have matured and strengthened over time. Aided by technology, monitoring has developed to collect information about who is being served and with what level of frequency and intensity, and even to track short-term outcomes—all of which can inform decision making. And the evaluation field is now more nuanced, with new approaches to answer a wide array of questions about outcomes, impact, and the factors that enable or inhibit change.
But, for all their advancement, these two building blocks are insufficient. We need a third leg of the nonprofit measurement stool to achieve more balance: feedback. Distinctly focused on the customer or constituent experience, feedback involves systematically soliciting, listening to, and responding to the experiences of nonprofit (or government direct-service-provider) participants and customers about their perceptions of a service or product. By listening to customers’ experiences, preferences, and ideas, we can gain unique insights that will help improve the quality and effectiveness of social programs.
Innovation, Revelation, and Amplification
Certain organizations are already leading the way in using feedback. Many have embraced customer perspectives as a crucial component of their work to source innovation, to surface hidden problems, or simply to amplify marginalized voices in our typical systems of service delivery. Examples of these three advantages underscore how adopting feedback into the measurement process can benefit both the programs and their respective clients.
Sourcing innovation | Some organizations implement all three legs of the stool, such as Nurse-Family Partnership (NFP), a model evidence-based program that recently used feedback to question its assumptions about what its clients actually wanted. NFP began in 1977 as a research project in Elmira, New York, whose studies determined that when a nurse regularly visits with a first-time mother for two years, providing a range of support and information, the arrangement produces many benefits, such as better birth and early child outcomes, and improved parenting. The program has been externally evaluated for 40 years and expanded into 42 states and six tribal communities. But in 2015, then new CEO Roxane White and new Chief Communications and Marketing Officer Benilda “Benny” Samuels determined that even evidence-based programs needed periodic innovations to reach new mothers and retain participation. So NFP decided to participate in Listen for Good, a systematic feedback tool to ask mothers about their experiences with the program—from whether they would recommend it to other new mothers to what they saw as its strengths and improvable areas.
Some staff were skeptical that these women would want to participate. They worried that the mothers wouldn’t want to use their data plan minutes to respond to the Listen for Good survey via text message. But when staff sent the feedback survey to 10,000 recipients, they received almost 1,000 responses in 20 minutes. The first thing they noticed was just how much the mothers appreciated the invitation; they saw it as a sign of respect. While they provided positive feedback about the program overall, they also had innovative ideas for improvement, such as connecting the participating mothers with one another, not just with the NFP staff; creating an app for NFP’s print materials; and, counter to the staff’s expectation, asking to be able to communicate with the nurses both via text and in person.
These recommendations led to innovations now being tried at NFP, including creating a new feedback team that is not only engaging mothers but also inviting feedback from staff, volunteers, and partner organizations. NFP, the gold-standard, evidence-based program, has integrated feedback as a third leg of its measurement stool to unlock new insights and drive continuous improvement.
Surfacing hidden problems | The Second Harvest Food Bank serves millions of people each year in Silicon Valley but never systematically solicited feedback from its customers until 2016. From its first feedback efforts in 10 locations, Second Harvest learned that customers from different cultural communities were having vastly different experiences with food and service at the food banks—white and Latino clients were markedly more satisfied than Asian clients. This insight led Second Harvest to experiment with more culturally sensitive approaches to its work, including new-volunteer recruitment and training, food choices consistent with traditional Asian diets, and even a new location for food pickup, to better serve the Asian community.
Second Harvest, like many nonprofits, will likely never invest significantly in an expensive third-party evaluation, but rigorous systematic feedback has bolstered its understanding of client experiences and preferences. If the organization can improve clients’ experiences, it will be better positioned to accomplish its mission to reduce hunger in all local communities.
Giving voice to those who are least heard | Epiphany Community Health Outreach Services (ECHOS) is a nonprofit ministry of the Episcopal Diocese of Texas that provides health and social services to the growing population of immigrants and refugees in Houston, Texas. ECHOS provides a range of critical safety-net services, including English language classes and, more recently, Hurricane Harvey relief services. Through its first efforts at customer feedback, ECHOS learned that clients were waiting excessive amounts of time, which consequently made ECHOS’ support difficult to access.
ECHOS’ staff realized that this experience contradicted their intention to treat immigrants and refugees with respect. Consequently, ECHOS is changing its registration process so that clients no longer have to stand in line to be received. It is also instituting expanded hours and making workflow improvements to increase efficiency. In fairly short order, the organization has transformed how it manages the flow of people, in order to promote a more positive and respectful customer experience.
Iterations on Feedback
When I began my career in applied research 30 years ago, I was taught that client satisfaction surveys were useless. They were seen as “lite,” in contrast with “hard” outcomes. Because of the power differential between nonprofits and their clients, evaluators assumed that satisfaction measures would always be positive and therefore not meaningful.
It’s a new day now. Throughout the social sector is a growing recognition of the importance of being human centered—that is, of putting the people we seek to benefit at the center of problem solving. That human-centered design principle should also apply to nonprofit measurement.
Many funders are already interested in connecting more with the communities they aim to serve, as well as looking for new measurement tools. In fact, a recent study by the Center for Effective Philanthropy found that foundation CEOs believe that listening more to the people they hope to help is essential to their success. The Fund for Shared Insight, the philanthropic collaborative that has been the driving force behind Listen for Good and other feedback efforts, has grown rapidly in the past four years, from six participating funders to 78 co-funders and counting.
US-based funders, such as the Plough Foundation in Memphis, Tennessee, have found feedback to be a powerful tool for their grantees. “The organization had never asked [its] populations what they wanted,” says Diane Rudner, the Plough Foundation board chair. She describes the learning from feedback, particularly from an organization serving people with developmental disabilities, as “so valuable… it’s amazing!”
Internationally focused Omidyar Network partnered with the Acumen Fund on “lean data sprints,” in which they gathered feedback from approximately 30,000 customers from 68 Omidyar investees across 18 countries. Like Listen for Good, the Acumen tool uses the Net Promoter system, which involves a calculation of customer experience scores. Based on this first effort to be customer-centric in its measurement, Omidyar and its investee organizations generated actionable insights about each relevant sector—such as independent media, education, and financial inclusion—and each organization’s perceived strengths and weaknesses.
The Omidyar Network still relies on traditional impact evaluation when it can, and on elaborate dashboards to monitor the progress of the organizations and businesses it supports. But now that the company has added a third dimension, customer feedback, to its measurement stool, it’s positioned for customer experience to drive improvement.
Let me be clear: I am not arguing against monitoring or evaluation. They are both important tools. Evaluation helps us to gain a deep understanding of what works and why, and monitoring helps us track our progress and provides useful signs for course correcting. But not every organization can invest equally in each leg. The advantage of feedback, when properly integrated, is that it is both information-rich and affordable. The insights, ideas, and preferences of our ultimate beneficiaries can unlock new possibilities for operational improvements, programmatic innovation, and more respectful engagement.
As a mentor of mine advised, “Let not the abuse of a thing be an argument against its proper use.” It’s time to stop denigrating satisfaction surveys and unleash the power of feedback in new ways. Let’s strengthen our measurement tools to be reliable, comparative, and simple to use, with both quantitative inputs and qualitative comments. And let’s start listening to gain insight, to improve, and to innovate.