(Photo by iStock/Eoneren)
As funders of social impact increasingly shift towards addressing the systemic nature of problems and their root causes within complex networks of cause and effect, making long-term commitments of patient capital requires funders to learn to evaluate their impact more comprehensively.1 However, they face a tricky balancing act: They need to grasp the complexity and ever-shifting nature of the systems they seek to change, but they also need constant feedback loops that signal progress towards impact goals.
No matter how much data gets collected, funders still often struggle to strike this balance. For one thing, there are few generally agreed-upon metrics for measuring the impact of long-term investments, or how to assess whether a portfolio of investments achieves a collective impact that exceeds the contribution of its individual components. The result is that philanthropic funding tends towards short-term projects that follow a straight line of cause and effect and are hence easier to evaluate.
Identifying success factors for lasting change is critical to cultivating a learning organization, one that is capable of continuous reflection and refinement of its approach over time. Investors and partners in developmental work must ask how we know we're really changing systems, and how we can we fit what we’ve learned into our monitoring and evaluation systems. Perhaps the toughest question is what to do with the results, and how to make evaluations actionable for decision-makers and stakeholders. For this reason, it’s crucial to take a step back and think about the assumptions behind our initiatives, how the context of the systems is shifting, and figuring out what steps come next. Whether that means tweaking our approach and experimenting with new solutions—or scaling up solutions, changing investment strategies altogether, and forging new partnerships—we need to make evaluation an ongoing part of the process, regularly collecting and analyzing feedback to help us stay on course, rather than waiting until the end of a project to evaluate.
This process can also guide the flow of patient capital to allow for sustained experimentation and adaptation, ensuring that we are not just reacting to immediate needs but are also building towards long-term systemic change.
Because traditional impact assessment methods often focus narrowly on individual projects or immediate outcomes—thereby limiting funders' ability to grasp broader systemic shifts—Sattva Consulting collaborated with Omidyar Network India (ON India) to develop a tailored approach for philanthropic players engaged in long-term investments. This approach can enable funders to map the evolution of their initiatives, comprehensively assess their contributions to sectoral development, and pinpoint areas of significant progress. Validated across two key ecosystems—Digital Society and Property Inclusivity—where ON India has made over one hundred investments, it can enhance understanding of systemic shifts driving impact at scale and facilitate strategic recalibration by providing real-time assessments of effective strategies and areas needing attention. The approach involves a self-reflection exercise centered on mapping three key areas:
- The System: What is the system that we're trying to change? What specific parts of the system have actually changed, and how substantially?
- Contributions: Did our efforts actually help bring about these changes in the system? What role did we play in making that happen?
- Pathways: Did we choose the right ways to push the system forward at the right times? Were the strategies and actions we took the most effective ones for moving the ecosystem in the right direction? Should we change or refine our investment approach based on the stage of the system?
1. Mapping the system: Is the problem getting better or worse?
Building a comprehensive vocabulary to describe the state of a problem in a system is essential for fostering alignment among stakeholders and charting a collective path forward. This process goes beyond semantics and requires a deep understanding of the material events, ongoing debates, and power dynamics shaping the system's evolution. Meticulously mapping these elements is critical for identifying key inflection points and understanding how they influence the system's trajectory, thereby gaining insights into the underlying drivers of change to tailor interventions accordingly.
There are three key principles for doing so:
- Large-scale systems change is triggered by key developments, enabling factors, events, and existing structures. It is important to map all factors, events, and foundational structures while assessing change in the ecosystem and understanding the readiness of the ecosystem to drive the right type of interventions at the right time to influence change.
- Taking an individual-centric approach. While understanding a problem in a system, it is important to account for an individual's lived reality. The evolution of policies and regulations introduced by institutions, while also remaining cognizant of the interests of vulnerable and lagging populations, can help understand progress made at scale.
- Understanding trade-offs between benefits and risks. The thematic framing and the approach for mapping the evolution of the landscape should include two aspects from the last-mile beneficiary’s point of view: access to the benefits of the service; and safeguards from the risks and harms.
To map a problem in a system we’ve developed a five-stage framework, which every system we’ve observed evolves through.
Click to enlarge
During the Latent stage, the system remains unaware of the challenge's existence, lacking a common vocabulary and viable solutions. There is minimal awareness among the public regarding the issue at hand. Many communities, especially in under-resourced areas, may practice behaviors or follow norms due to a historical lack of access and a limited understanding of the detrimental effects.
In the Nascent stage, awareness about the challenge begins to grow. Initiatives by government agencies, NGOs, and international organizations highlight the risks and social implications associated with the issue. Siloed efforts by various stakeholders to promote solutions and build infrastructure start gaining momentum, albeit with varying degrees of success across different regions.
In the Emerging stage, political will and public initiatives gain traction, bringing the issue into mainstream discussions. Efforts to eliminate the challenge include large-scale programs aimed at promoting behavior change, constructing necessary infrastructure, and improving access to essential facilities. Community-led initiatives also play a crucial role in raising awareness and mobilizing communities to adopt new practices.
The Mainstream stage arrives when all stakeholders share a common vocabulary, there is consensus on solutions, and there are minimal exclusion risks across geography and communities. This stage is characterized by the widespread implementation of programs and policies. Successful initiatives result in significant behavioral change, leading to a substantial reduction in the issue's prevalence. Civil society and philanthropic organizations complement government efforts by providing technical expertise and mobilizing resources at the grassroots level.
Finally, in the Transformed stage, a challenge is considered largely resolved, with demonstrated success of implemented solutions and continuous improvement mechanisms in place to minimize any unintended effects. While some challenges may persist, the overall mission demonstrates the potential for systemic change in addressing complex societal issues.
2. Mapping contribution of your investments: How much did my efforts influence systemic change?
After mapping the evolution of the system, the next step is to assess your contributions to the key events to better understand whether and how your efforts may have influenced these changes in the system. If so, what role did you play in making it happen? This process requires evaluating your impact through the lens of the 3S framework: Seminality, Scale, and Sway.
To begin with, consider the Seminality of your intervention. Were your efforts unique in contributing to significant events within the system? For example, Aadhaar, a 12-digit identification number for Indian residents, serves as proof of identity and grants access to various government schemes for those eligible. ON India's collaboration with IDinsight and Dalberg to assess the State of Aadhaar in India was seminal as it gave voice to the feedback from more than 167,000 Indian residents across the country on what Aadhar and digital identity meant to them, highlighting significant benefits as well as concerns about exclusion. Creating the largest primary dataset on Aadhaar usage and digital ID globally aided the Unique Identification Authority of India (UIDAI) authorities in expanding Aadhaar enrollment drives to include transgender and homeless populations as priority groups.
Next, consider the Scale of your investments. Did your interventions lead to large-scale population-level changes? ON India’s support to Tata Trusts in providing land titles to the Odisha government using drone technology resulted in land rights for 55,000 slum households in 2019. Through a partnership with the state government, ON India and Tata Trusts were able to address the technical challenges and provide the necessary expertise to implement policy more speedily and at a lower cost. This initiative also showcased the potential of technology to address complex issues like land rights and slum redevelopment on a large scale.
Lastly, evaluate the Sway of your work. Did your efforts influence key decision-makers to drive significant events? For example, ON India’s investees (21N78E, Aapti Institute, Mythos Labs, and others) raised awareness about data protection issues in India among citizens, playing a crucial role in advocating for personal data protection laws. Their efforts, along with others in the ecosystem, contributed to the enactment of the Digital Personal Data Protection (DPDP) Act in 2023, the first cross-sectoral law on personal data protection in India. This legislation, which now mandates obtaining consent before processing personal data, marks a significant milestone in India’s data privacy landscape, made possible by swaying decision-makers in the legislative landscape.
3. Tapping into the right pathways at the right time to propel the ecosystem forward: How do I refine my approach to solving the problem continuously?
Based on learnings from our evaluations of the ecosystems that ON India operates in, it's evident that shifting mindsets and paradigms play a crucial role in any system. While individual mindset shifts can happen relatively quicker, challenging societal norms and beliefs require a more nuanced approach. Societies often resist change, especially when it comes to deep-rooted paradigms.
In the Latent Stage, it's essential to establish a shared vocabulary, setting the foundation for productive discussions within the ecosystem.
Moving into the Nascent Stage, the focus shifts to conducting thorough, solution-oriented research. It's about identifying actionable pathways and investing in comprehensive initiatives to lay the groundwork for informed strategies.
Transitioning into the Emerging stage, innovation becomes key: piloting new solutions, continuous experimentation in specific contexts, and producing research relevant to policymaking are crucial steps.
As the system progresses to the Mainstream stage, scaling proven solutions becomes the priority. Long-term investments and tailoring solutions to meet the needs of diverse populations are essential for sustainability.
Finally, in the Transformed stage, a reflective approach is crucial. Evaluating the long-term success of implemented solutions helps minimize unintended consequences and allows for continuous improvement. It's about taking stock of what's been achieved and striving for ongoing progress.
A Call to Reflect, Review, and Refine
The journey towards transformative change transcends strategic efforts; it embodies a collective commitment to impact. Investment portfolios represent more than just diverse assets; they symbolize our collective capacity to drive positive change.
This systematic approach to mapping and evaluating ecosystems not only provides a framework for comprehending systemic shifts but by consistently assessing intervention effectiveness at each stage—whether in establishing foundational vocabularies, conducting research, innovating, scaling initiatives, or reflecting on outcomes—organizations can dynamically adapt their approaches. This can be especially helpful to build the “catalytic” power of philanthropic capital, which can seed work and experimentation even in under-focussed and nascent areas, learning from failures and building on successful experiments. This iterative process of reflection and refinement ensures responsiveness to evolving challenges and opportunities within complex systems, fostering a proactive approach to addressing entrenched societal issues. It cultivates a culture of learning and adaptation across the impact sector, enhancing our ability to catalyze enduring change.
1 In 2016, Rockefeller Philanthropy Advisors launched the “Shifting Systems Initiative,” encouraging philanthropic foundations to invest in transforming systems. Building on this momentum, the Foundation announced in 2023 that it will invest over $1 billion over the next five years to advance the global climate transition, ensuring inclusive participation of all actors. Similarly, Co-Impact's 2018 announcement to invest $500 million in systems change initiatives has galvanized 40 funders from 16 countries, deploying $350 million towards health, education, and economic systems reform in Africa, Asia, and Latin America, while also raising over $450 million with support from over 20 funders for gender equity. Meanwhile, Ashoka, a global network of more than 3,500 social entrepreneurs, continues to champion the link between social entrepreneurship and systems change, demonstrating the profound potential of long-term investment in entrepreneurs driving transformations through experimentation and innovation.
Read more stories by Rathish Balakrishnan, Shilpa Kumar & Palagati Lekhya Reddy.