Using Measurement to Manage Impact

Enterprises like Sekaf, a shea butter producer in Northern Ghana, can benefit from a deeper, bottom-up understanding of the social change they create. (Photo by Martina Castro)

Over the last few years, impact investors have made many efforts to standardize “output metrics”—data about products, services, and business practices—so that they can draw better comparisons across companies (see IRIS and HIPSO). But while these metrics are useful for portfolio monitoring, they have two important shortcomings. First, they are typically effective in capturing the scale of social impact but are less telling of its depth. Outputs can tell us about the nature of products and services, and the number of people they reach, but not about their effect on people’s livelihoods and well-being.

Second, this approach tends to view businesses as data providers, rather than data users. Yet companies also need data as they build their business lines, explore new markets, and target customer segments. By collecting data directly from people, companies can better understand who they want to source from, sell to, and staff up with. They can also learn how customers are using their products and services, and about the material effect of those products on their lives. In this way, measurement not only shows the impact a business has already created, but also allows for impact management—the integration of social and environmental performance at each stage of the investment cycle.

A Deeper Look at Impact

The CDC Impact Fund invests in funds and other intermediated vehicles that deliver high development impact, particularly to underserved populations. In-mid 2016, we began piloting insights studies, which we called “deep dives,” to complement existing data reporting and get a deeper, bottom-up understanding of how people engaging with portfolio companies were experiencing change. In the first year, we undertook deep dives into five portfolio companies in Ghana, Nigeria, and Kenya to cover companies operating in the agriculture, energy, and ICT sectors (see reflections from our first study here). Our North Star was to develop a measurement approach that was light-touch but in-depth, and that could generate value for investors, investees, and end-beneficiaries alike. Here’s a look at the approach we developed and what we learned.

1. Match the research methods to the investment context. We found success in remaining methods-neutral. That is, we didn’t prescribe a particular data collection methodology or choose a single research partner. Instead, we developed a set of five engagement principles:

• Participatory. Align collection with investor and investee data needs, and center on upcoming business decisions.
• Value-added. Help companies access data that provides new insights into their business models and stakeholder groups.
• Right-sized. Use methods proportionate to the scale of impact.
• Confidential. Keep all data secure, and never share it outside of the fund or company.
• Independent. Have a team separate from company and fund management lead the process to ensure impartiality.

Based on these, we built data collection around the type of impact we wanted to investigate and the specific needs of the company in question. This led to a mix of research designs and methods, including observations, focus groups, and semi-structured interviews. Mostly, however, we used surveys. Our research partners, 60 Decibels and the Busara Centre for Behavioural Economics, ultimately talked to more than 2,000 people to understand the perspective of suppliers, distributors, customers, and communities.

2. Anchor around business decision-making needs. Collecting impact data is a means to an end, not an end in itself; what we do with the data is what matters. We therefore designed these studies to feed into businesses’ future decision-making by capturing both descriptive information (such as units purchased or increased income) and predictive information.

For example, with Sekaf, a shea nut processor and trader in Ghana, we not only looked at the value the company had already created in terms of stable livelihoods for existing suppliers, but also asked about suppliers’ intentions to continue working with the company and whether they were likely to recruit friends and family into the supplier network. This helped Sekaf understand its prospects for growth at a time when its main challenge was deciding how to expand raw material procurement.

3. Draw clear boundaries to make findings actionable. Attempting to measure the net impact of a company—its full range of positive and negative impacts on social and environmental issues—can be extremely expensive and lengthy. Instead, we tried to focus on selected parts of a business and a few affected groups, particularly ones that were experiencing the most impact or that we knew less about.

This was easier said than done. It’s tempting to want to know everything about everyone, and companies and investors often want different information. For example, for our first deep dive, which focused on a poultry feed manufacturer in Ghana called Agricare, we investigated the impact the company was having both upstream (on farmers supplying maize for the feed) and downstream (on poultry farmers). But our insights and recommendations were so wide-ranging that the business had trouble determining whether it should prioritize changes to the marketing or procurement arms of the company.

Instead, when investigating multiple groups or sides of a business, we recommend thinking about separating deep dives into a series of bite-sized studies. In a later study of a wireless internet provider in Kenya called Poa, for example, we first studied the impact of the company’s home broadband system on customers, then its community Wi-Fi offer.

4. Make sure you are ready to mobilize quickly. If you can’t collect data within a company’s preferred timeline, it’s best to postpone the exercise altogether. We try to conduct deep dives when companies face strategic and product decisions, and feed information back to them as fast as possible so that they can act. Traditional impact assessment methodologies haven’t been great at providing real-time or quasi-real-time data to investors and companies. Delays reinforce the oft-held perception that impact measurement is too slow, academic, or “unbusiness-like” for purposes other than reporting. Our quickest deep dive took six weeks, while our longest took six months. Having re-approved research partners and enumerators, and using tried-and-tested approaches, can help speed the process and make the exercise more relevant to company operations.

5. It’s never too early to think about impact. We often hear fund managers saying it’s too early to measure impact. But measurement can be just as useful at the beginning of the business life cycle as it can during the growth phase. Collecting customer data as soon as a new product or service launches, or when a company markets it to a new segment or geography, provides a window into likely future impact. It also comes at a time when the company has to make decisions about whether to pivot or persevere in strategies such as marketing tactics and distribution channels.

For example, in our work with SunCulture, an early-stage venture in East Africa that supports farmers, we gathered feedback from early adopters of the company’s new solar-powered water pump. The insights helped SunCulture understand how its product created value for customers by saving them time and money (through the replacement of expensive diesel pumps and inefficient manual methods), and shaped how it structured sales and after-sales processes.

6. Adjust language to the business reality. Some investees in the CDC Impact Fund portfolio self-identify as social enterprises, while others think of themselves as “finance-first” businesses that generate impact through the nature of the products and services they sell. When discussing deep dives with companies, it’s useful to adjust language to match the mission of the company.

In speaking with the CEO of a finance-first company, for example, presenting the exercise as a tool for understanding impact may relegate it to the corporate social responsibility department. It’s more effective to introduce it as a tool that can help the company get better data to understand the wants and needs of its customers, suppliers, or employees. This approach cuts across both impact and business objectives; for companies to succeed and last, they need to consider the interests of a wide range of stakeholders, not just the financial interests of their shareholders.

When we carefully design deep dives so that there is a strong overlap between commercial and social data points, the findings are more likely to generate insights that businesses actually act on.

7. Standard measurement should remain rigorous. Generating rapid insights does not mean throwing good research practice out the window. This is particularly important if investors are using findings to make recommendations to companies about business operations and strategy. With the online Kenyan newspaper Hivisasa, for example, we conducted phone surveys with a representative sample of readers and even ran a randomized control trial to measure behavioral changes.

Accurate data collection and analysis, as well as ethical practices related to voluntary participation, informed consent, and the proper storage and use of data are important to maintaining research credibly.

8. Ensure that findings are actionable. Interpreting sometimes-contradictory customer feedback and data, and translating it into actionable recommendations for investees often took us as long as the data collection process itself. Investors and intermediaries need to play an active role in helping companies think through how they can turn data into actions that drive both commercial success and greater impact, particularly in cases where deep dive findings challenge fundamental assumptions underpinning the business strategy.

The Agricare deep dive, for example, revealed interesting data on the socio-economic profile of farmers buying feed through distribution channels; the company was actually selling to many more small-scale farmers than it thought. By triangulating this information with other qualitative feedback from customers and non-customers, we were able to create actionable recommendations for how the company packages, prices, and promotes its products to meet the needs of different market segments.

The Potential of Impact Management

Deep dives are now an integral part of our approach. They have improved our understanding of the impact of our portfolio, and investee businesses have taken concrete business actions to respond to their findings, including:

• Re-designing products to better meet customer pain points
• Setting up an after-sale call center to continue collecting customer insights
• Exploring ways to provide protective clothing and equipment to suppliers to improve safety and health outcomes, and contributing to better supplier retention
• Improving timeliness of procurement practices to improve supplier satisfaction
• Reviewing product pricing to better respond to different market segments

A bottom-up, people-centric view of impact—to complement top-down, investor-centric metrics—requires a different approach to measurement, one that better matches the welcome trend in impact investing toward proactive impact management vs. passive impact measurement. It means understanding and speaking the languages of both business and social impact, and knowing when and where to use them. Done well, approaches that help move beyond metrics can help shift our idea of impact measurement from an obligation to meet investor and asset owner requirements, to a strategic opportunity to support companies to improve both their business and impact performance.

Read more stories by Martina Castro & Matt Ripley.