(Illustration by iStock/Viktoria kurpas)
In theory, social impact measurement should be a powerful tool to improve the European social economy. It can help individual organizations set realistic objectives; monitor, learn from, and improve their activities; prioritize decisions; and access funding. Collectively, social impact measurement can help organizations working on similar social issues or in similar geographic areas better understand the aggregate impacts of their work and collaborate to achieve greater change.
And at a European level, agreed upon standards, common indicators, and benchmarks can allow policy makers to evaluate the impact of the social economy on society, advocate for more public funding of social economy organizations, and help donors and investors direct their resources to the interventions that have the most impact.
But in practice, social impact measurement has often come up short. Nonprofit organizations and social enterprises, often pressed for money, typically underinvest in impact measurement. As a result, they lack the evidence that is needed to secure funding from government, grantmakers, and impact investors. It is a vicious cycle that breeds cynicism and distrust.
It doesn’t need to be this way. We believe that there is a new path in which social economy organizations develop a learning culture where impact measurement is truly integrated into the workings of the organizations, producing data that matters and that is inclusive of the voices of the people who are impacted, and that is actionable by the organizations doing the impact work and also useful for the policy makers and funders who support that work.
Of course, impact measurement is not without its challenges. Critics have pointed out the risk of channeling resources to interventions that are easy to measure, but potentially with low impact. The long-term effects of interventions involving multiple stakeholders and addressing complex challenges are more difficult to measure and might therefore be overlooked. And bad actors can find ways to game the indicators and cherry-pick service users to help them meet their targets.
Although such arguments are valid, we believe that the risk of not measuring social impact in Europe and aggressively pushing this agenda far outweighs any of these concerns. Complex societal issues such as climate change and COVID-19 need measurement and calculation to incentivize action and create essential changes. Tools, calculations, and data for assigning accountability to actors have the potential to not only serve the social economy but also to shift the entire European economy in a more socially and environmentally positive direction.
Efforts to account for social impact by social economy organizations may alter European financial markets, corporate activity, and public administration from within, transforming our understanding of what is “social” and facilitating the urgent task of responding and building back better in the light of the pandemic. Moreover, the social economy must not lose sight of the fact that what is not measured (and hence not valued), may not have sufficient (or any) value within the system.
Lack of Impact Measurement in Practice
So what do we mean when we refer to social impact measurement? Social impact measurement is the measurement of social change for the targeted population attributed to the activities of an organization during a specific period of time. The change might be positive or negative and there may be unintended consequences of one’s actions to consider. Impact measurement involves several dimensions as defined by the Expert Group on Social Entrepreneurship (GECES), which advised the European Commission (EC) on the roll-out and research on social economy policies. These include the impact measurement process, the indicators used (that can be standardized for specific interventions), as well as the principles for reporting, transparency, and disclosure.
Unfortunately, measuring impact is not a common practice in Europe. In a recent cross-country comparison of impact evaluation in eight European countries (Denmark, Estonia, France, Germany, Italy, Poland, Spain, and the United Kingdom), as many as 45 percent of expert respondents said that impact evaluations are only carried out occasionally. How can we explain this?
Many operating foundations, nonprofit organizations, and social action associations have not fully bought into the logic of why they need to spend precious resources on social impact measurement. Some social economy organizations believe that impact reporting is a top-down bureaucratic burden, and there is also some cultural resistance to the idea of counting ‘what cannot be counted.’
Even when organizations want to measure impact there is often not enough money to spend on monitoring, evaluation, and learning, which can be relatively costly. This is especially true if it comes at the expense of delivering frontline services. In many instances, donors and public sector commissioners are demanding impact measurement but are still unwilling to pay for it.
If organizations do conduct social impact measurement, it is generally at the level of measuring outputs (the number of people attending job training programs, for example) rather than outcomes (the number of people who attended job training who were able to get a well-paying job). Even worse, the number of social sector organizations who do impact assessment ‘by the gut,’ or with anecdotal, qualitative data is still surprisingly high. As a result, impact measurement has been patchy and inconsistent and has generated data of dubious quality which cannot be easily compared.
Social enterprises in Europe have been the loci of much innovation and activity relating to social impact measurement and can be at the cutting edge of practice. But that is becoming less true. Today, many social enterprises are struggling to deliver attractive financial returns and many impact investors are focusing more on tracking financial than on social measures. As a result, many social enterprises are not prioritizing social impact measurement and often only tracking a few operational KPIs (key performance indicators) for impact reporting.
In Europe, there have been many initiatives by impact investors to develop frameworks and standards for measuring social impact, including the European Venture Philanthropy Association’s five-step process and the Impact Management Project. But it can be a challenge to harmonize between different tools. Most impact investors are opting for customized frameworks, which makes it difficult to compare impact data across investors and investees. Although methodologies such as social return on investment and social impact accounting have gained some traction in Europe, there is a growing awareness that a synthetic unit of social impact analysis may be impossible.
Bright Spots in Social Impact Measurement
In spite of all of the impediments to doing impact measurement, there are positive developments that lead one to be hopeful. One of these is the growing interest in shared measurement, where organizations from different sectors agree on an approach for measuring change within an entire system (the education system in a certain territory, for example). Although formalized shared measurement is rare in Europe (with only a few collective impact initiatives present), the COVID-19 pandemic has accelerated the desire by social economy organizations to work together and create common ways of measuring impact.
The increased use of social impact bonds is also having a positive impact, by educating the field about the importance of social impact measurement, creating a growing number of people skilled in measurement, and enabling the development of a high-quality evidence base for certain interventions. Several European countries have large markets for social impact bonds (the United Kingdom has had 69, the Netherlands 15, and Portugal 13). The European Investment Bank recently launched a €10m fund with BNP Paribas to co-invest in social impact bonds.
Social economy organizations are beginning to make better use of data and digital technology for measurement and evaluation. Using big data, for example, can increase sample sizes, increase the probability of including vulnerable groups, and help develop longitudinal datasets. In the United Kingdom, for example, Data Labs helps social economy organizations create quasi-randomized control trials by using large-scale administrative data to find a control group.
Stakeholder-led impact measurement is now emerging in Europe as a crucial approach. Some funders are moving toward a partnership model with grantees and investees, actively integrating beneficiaries and communities’ voices in the social impact measurement process. For example, the ESADE-BBK Community of Practice on Impact Measurement and Management is comprised of 27 European foundations collaborating on impact management. Many members are incorporating a diversity and inclusion lens, assessing power dynamics in their approach and moving toward a more participative, co-creative, partnership with their grantees and investees to determine what might be measured and for whom.
And the growth of sustainable and ESG investing is creating the impetus for clearer, comparable, global, and environmental social impact data. In addition to movements such as B Corporations, there is the potential for cross-pollination between the corporate, financial, and social sectors on how to measure social impact.
A New Approach to Measuring Impact Is Needed
While there are bright spots and excellent practices, much of the social economy is in the grip of a vicious circle where lack of resources and incentives for impact reporting are leading to what we call a proving impact model of impact measurement where data is generated purely for fundraising or compliance purposes. Because organizations are not fully investing in the evaluation process they end up producing low-quality data which is not shared or used for learning and generating evidence, a problem that is affecting the entire sector.
The Vicious Circle of Proving Impact
After working with many funders and social economy organizations on social impact measurement, we have come to believe that their central focus needs to be on learning, not on compliance, and that this mindset shift will help unlock a virtuous circle that leads to improving impact. Social impact measurement must be framed as a holistic, organization-wide process rather than as a siloed, technocratic exercise.
The Virtuous Circle of Improving Impact
A large-scale study of learning in the nonprofit sector found that only 40 percent of nonprofit organizations believe that their existing processes are effective for encouraging learning, and only half had created incentives around learning. Developing a learning culture cannot be done overnight. It requires several steps and considerations, as well as changes in the behavior of funders and commissioners (public sector purchasers). A learning culture needs to be reflected in the organization’s leadership and culture, and also in its structures, governance, and processes. Ideally, social economy organizations have defined roles and responsibilities for capturing, distilling, applying, and sharing knowledge internally and externally, with specific processes that are a part of daily workflows.
One way to create a learning culture is to change the language that is used to describe the work. Rather than talking about “impact measurement,” consider calling it “impact management.” This focuses less attention on the data itself and more attention on the process, underlining the need for continuously learning and improving.
It is essential that the impact management process empowers stakeholders and beneficiaries and is part of a multi-stakeholder, collaborative, partnership approach by funders and commissioners, where social economy organizations accept and are not afraid to account for impact risk and negative impact. As the quality and relevance of impact data, evaluation, and reporting increases among social economy organizations, it will naturally lead to increased collaboration and data-sharing and become a source of pride rather than a burden.
Using Policy to Change Social Impact Measurement
The role of impact measurement goes beyond enabling social economy actors to manage towards greater impact. Europe is at the forefront of developing policies to promote a just and sustainable socio-economic development, and integrating impact measurement and management into these policies and systems will be an important part of achieving them.
In December 2019 the EC announced the European Green Deal through which Europe marked its ambition to become the first climate-neutral continent by 2050. As part of the European Green Deal, in January 2020 the EC presented its first reflections on building a strong socially-just Europe by designing an appropriate transition mechanism. It will provide targeted funding to generate the necessary investment in the most affected regions through fundamental restructuring of the economy, structural changes in business models, and new skill requirements.
The social economy—including its organizations, practices, and tools—will be a pivotal lever in achieving a just transition, not only through the implementation of its activities, but also through its experience of measuring and managing impact. For these policies to channel funding to the social economy, the sector needs to be able to show that the public sector gets its “money’s worth.”
The main barriers that social economy organizations face are systemic and related to a lack of financial and human capital dedicated to impact measurement, which results in a lack of transparency and data sharing. To help these organizations build their impact measurement capabilities we recommend that European policy makers consider the following concrete actions:
- Facilitate the upskilling and capacity-building of the people and organizations doing social impact measurement
- Support the training and certification of social impact professionals. European organizations should establish academies and training institutions and create an official certification as social impact evaluator at the European and EU-member country level.
- Support the capacity building of social economy organizations in implementing impact measurement and management. Grants should be provided to social economy organizations to pay for staff to attend specialized trainings offered by certified educational institutions and to hire trained social impact professionals.
- Identify and promote best practices on impact measurement and management for different types of social economy organizations. The European networks and associations that support social economy organizations (such as foundations, cooperatives, and social enterprises) should lead this effort through research and dissemination.
- Promote the philosophy of learning and learning organizations among social economy organizations. The EU and other bodies should sponsor national-level and EU-wide social impact learning conferences and exchanges.
- Catalyze and enable data collaborations and evidence generation
- Catalyze and support open data initiatives relating to social impact. The EU, corporations, and foundations should sponsor national and EU-wide data sharing and open data initiatives relating to social impact metrics and measurement, and create EU-wide data labs on specific social impact issues. The EU should provide grants to support the involvement of data scientists in the social economy.
- Fund research that provides scientific evidence of social impact. Private and public institutions should fund the creation of European initiatives (similar to J-PAL in the United States) to conduct randomized control trials of European initiatives. The evidence gained from this research should be shared broadly in the public domain.
- Promote social impact frameworks and standardization
- Encourage transparency and reporting on social impact metrics. Building on the success of international multi-stakeholder initiatives such as the Impact Management Project and the SDG Impact Standards, European organizations should create clearer guidelines for social impact reporting. Social economy organizations that share data should be rewarded with greater access to funding and public procurement contracts.
- Use data and evidence to drive public funding. Public sector funders need to make greater use of data and evidence to understand if they are allocating public money in the most efficient manner. Initiatives such as a unit cost database that are being implemented in the United Kingdom and Portugal should be explored by other countries and the EU.
Going Forward
Now is a critical moment in the evolution of the European social economy. Will it be able to take its rightful place at the front and center of European development? Will it be embraced, celebrated, and cultivated by the EU as part of the path forward for the continent to transition to a more socially just and environmentally sustainable society? One factor in whether that happens is the extent to which the social economy comes forward in a more powerful and unified voice about the importance of impact evidence.
Until now, social economy organizations have failed to adequately communicate why impact measurement is important. They have had a narrow focus on impact measurement as a fundraising and compliance tool rather than as a transformative learning opportunity. One of the ways to catalyze a change is to implement the ideas at the EU-level that we outlined earlier. But for change to happen it also needs to be driven from within the social sector by champions and trailblazers.
In our work with foundations, nonprofit organizations, and social enterprises we have seen that many people are taking this challenge seriously. But in their watercooler moments social sector leaders share their doubts with one another about whether that transformation can actually happen, and wonder if social impact measurement will instead remain a painful imposition made by funders which doesn’t help their organizations feed more mouths, house more people, or develop more sustainable solutions.
It’s time for the social sector to have a grown-up discussion and dialogue about this dilemma. We need to lead with real examples of how impact data and measurement can be transformative. The easy part is developing a new standard or framework. The hard part is getting the buy-in of the sector from the bottom-up and including all voices, especially those with the greatest needs who often are the least represented.
Support SSIR’s coverage of cross-sector solutions to global challenges.
Help us further the reach of innovative ideas. Donate today.
Read more stories by Lisa Hehenberger & Leonora Buckland.