Consumer satisfaction is one of the primary ways we assess private sector services. The majority of online consumers read ratings and reviews on websites such as Amazon, TripAdvisor, and eBay before making a purchase. We post product complaints online because we know other consumers are interested in our experience. What’s more, professionals from mechanics to insurance sellers consider customer feedback when assessing their market position and efficacy.
Although there are ongoing conversations about best metrics for judging quality, there is consensus that consumer satisfaction and feedback are useful and drive quality through competition. And there is a generally held belief that in a democracy public opinion matters.
But one area of public life is notably void of consumer feedback—the public and nonprofit sector serving low-income families. (We identify the people who use services as “consumers” instead of as “recipients,” to underscore that their feedback matters and to make programs and services accountable to them.)
A family that participated in the Family Independence Initiative, where Mia is vice president, exemplifies what low-income families experience when engaging with program providers. Three years ago, Jesus and Carmén, parents of three children living in a cramped two-bedroom apartment in San Francisco, learned of a city-run, low-income, first-time homebuyers program. They signed up and were told that before they could learn if they qualified for a loan, they would need to take a financial education class. Arranging childcare for their kids, one of whom is disabled, was not easy. Jesus rearranged his restaurant work schedule and the couple took the two-hour bus ride on weekends. It took almost two months to complete the eight hours of required training. It then took two more months to get a compatible meeting time with the loan specialist. Within five minutes, the specialist told them that their income was just below the threshold required and that they didn’t qualify for a loan.
This story of program wastefulness and disregard for consumers’ time is all too common. With no way to give meaningful feedback about their experience, Jesus and Carmén joined the ranks of voiceless and frustrated consumers. Moreover, their “participation” likely was used as a positive program statistic—evidence to funders that the program has broader financial education impact!
Creating methods for consumer feedback in the public sector is critical on a number of fronts: it provides funders with detailed information about program strengths and shortcomings and creates accountability to consumers; the feedback can be used to make programming more efficient and effective; and consumers will see, as programs are adjusted, that their voice makes a difference. James D. Power IV, former vice president of J.D. Power and Associates, the company that brought customer service research to many industries, sees feedback as essential to improving the quality of nonprofit services. Consumer feedback, for example, changed the automotive industry, leading to increased transparency, competitiveness, and better products. “We are a better society for it,” says Power. “I believe that nonprofits need to adopt a similar perspective, and make sure that continuous improvement is part of their operation.”
This is not the first effort to talk about program recipients as consumers. In the mid-1990s, as policy and social welfare analysts were looking at welfare reform, M. Katherine Kraft from the Robert Wood Johnson Foundation and professor Irene Bush of Rutgers University proposed using consumer reviews as a tool to evaluate and improve social welfare programs. Their argument was similar—that listening to recipients as consumers can improve service standards. But nothing came of the proposal.
Now is the time to fully implement the consumer feedback approach in human services. As the nation tightens its belt and programs and services contend with budget cuts, there is increasing pressure to justify funding with results. This, along with an insistence on evidence-based solutions, is a positive trend. But unless we integrate feedback from consumers, we will not have a complete picture of program impact.
Listening to Recipients
Why is there so little interest in learning from users of human services? Our current system, based on needs assessment and identification of deficits, maintains a stereotype of program recipients as helpless and uninformed. Professionals in the nonprofit sector—often social workers and program managers—are seen as most qualified to set goals and action plans. It’s not uncommon for program directors to think that program recipients “don’t really know what’s good for them.”
A 2006 study by the Annie E. Casey Foundation, “The Culture of Money,” pointed out that the trend of funding financial training has been based on the stereotype of low-income families as unable to manage money. This stereotype allows for social workers in their 20s, who may themselves be relatively financially illiterate, to hold power over families that have navigated wars, refugee camps, or our toughest neighborhoods with few resources. The stereotype of low-income people as incapable and in need of guidance is deeply entrenched in the service sector. Given the dynamic of decision making and power, it’s not a surprise that there is little interest in finding out what low-income program recipients think about the services they are receiving. Their perspective is simply not valued.
It is critical not only to gather and weigh consumer voices in program evaluation, but also to integrate this feedback into accountability systems and make it ongoing. Who will do this? Unlike the private sector, where competition creates consumer choice and value, competition is lacking in the social sector. But there is an underlying power: funders. If funders value consumer voices and seek and act on consumer feedback, then providers will be more consumer conscious. Money is power and the funders hold the key.
Feedback collection by funding agencies has begun to happen, thanks to prestigious social program evaluators such as MDRC and Abt Associates. The US Department of Health and Human Services also has called for consumer feedback in many of its funded areas (although the feedback has been a one-shot deal leading to a onetime change). And foundations increasingly are interested in their users’ feedback. The Bill & Melinda Gates Foundation recently announced the results of their grantee perception report. The feedback was sobering. The foundation received below-average ratings from grantees about grant experiences. As a result, CEO Jeffrey Raikes has pledged to improve practices.
Consumer Voice in Action
Funders currently spend a lot of time doing due diligence on competing program proposals, but they may hesitate to incorporate consumer feedback into their diligence process because they either believe there is no clear system for collecting that feedback or fear that recipients won’t provide constructive feedback. Yet there is plenty of evidence that recipients do have valuable feedback, and the tools exist to collect it. Professional pollsters are able to identify target populations and customize surveys that can garner feedback, establish baseline sentiments, and then track changes over time. The Knight Foundation used Gallup pollsters to learn about resident perspectives for a study on local economic growth, and the California Endowment used political pollsters to gauge the sentiments and impact of a campaign around the Affordable Care Act. Current technology also provides the means to capture user feedback in an ongoing way quickly and at little cost. With 74 percent of adults online, nonprofits can also collect feedback using sites like GreatNonprofits.
GreatNonprofits, where Perla is CEO, has a database of 1.4 million US nonprofits, including most public and private social programs. It allows people to post reviews, photos, and videos about nonprofit programs. The website also enables nonprofits to invite their users and volunteers to post feedback about them. SF Connect is one forward-thinking program that has collected feedback on GreatNonprofits.org from its homeless clients. Dariush Kayhan, the former executive director, explained, “SF Connect uses the reviews as a way to improve the program’s offerings and impact. Reviewers have had a lot of ideas for us, the sorts of services that we ought to bring there; some we could plow right into the organization to improve it.”
In the end, consumer feedback has the greatest value for the consumers themselves. Feedback allows people to distinguish between good and bad services, to avoid wasting time, and to exchange valuable information (such as loan requirements). This exchange is empowering, and technology has made it more accessible.
Over the Family Independence Initiative’s last 10 years working with low-income families, we have seen this in action. Families enrolled in the initiative receive laptop computers to track their monthly progress in six areas, including income/asset building, health/wellness, education/skill building, and networking/helping others. They also rate any program in which they participate on a scale of one to five stars and provide comments. It is a simple system, and the families report that they find the tracking tool extremely useful.
Positive experiences have led to changes in the expectations of entire communities. In the Iu Mien refugee community in Oakland, Calif., for example, families are sharing information about what colleges are best and within financial reach. Information is also shared about scams and scam artists. A predatory lender was exposed and no longer does business in an Oakland-based Salvadoran community he tried to exploit, thanks to the information-sharing network.
Integrating consumer feedback into assessments will make programs more effective and more impactful. As programs listen to clients and shift definitions and measurements of program success, we will see greater competition and innovation that will result from prioritizing consumer voice, just as it has in the private sector.