Perhaps I don’t understand, since this seems to be an excerpt from a book, but “there may be no economic value in a beautiful sunset, an endangered species, or a wonderful work of art” is pretty clearly wrong. Real estate with a nice view is worth more, for example. In the San Francisco Bay area, we say that a view of each bridge is worth an additional $100,000 on the value of a house. Regarding endangered species, I worked on a case involving a gasoline pipeline that had to be re-routed at a cost of $1 million a mile to avoid an endangered salamander under the Endangered Species Act. And in regard to works of art, the newspapers weekly have stories on outrageous prices paid for them.
Thank you for this eloquent discussion of the discrete purposes to which measures of social value can be applied, and for your ideas about the way in which the design of measures should reflect the inherently dynamic and subjective nature of this value. Having worked with investors, businesses, nonprofits and a few governmental entities for ten years on measuring impact, we at SVT Group have come to regard what is happening, and what you speak of, as the advent of a new business discipline, which we’ve begun to refer to as “Impact Management.”
This new discipline is larger than any one organization, metric, standard or even framework. Thousands of players, both inside and outside of academia, are contributing to its rapid emergence and evolution. This discipline reflects many of the phenomena you articulate- in particular, the difference between measures used for marketing, measures used to communicate with potential or actual investors, and those used to inform internal operations- the latter two having to do with understanding and communicating what is actually happening or may happen, and the former having to do with extracting the business value to be gained by the “social value proposition” one offers.
I would offer one critique of what you’ve said. For several years I have been hearing tee-up statements like “everyone is interested in social value” that come with a “but” clause similar to this one: “Alas, they cannot agree on what it is, let alone how to assess it,” followed by a new solution. This framing makes it sound like there is no progress being made and I think does a disservice to the massive, complex and powerful undertaking we’re all a part of.
What I am seeing, and would invite you to recognize more than I feel you have here, is that as more and more nonprofit and for-profit organizations and funder/investors are striving today to understand and manage the impact we are all generating, a market conversation is beginning to take place, and solutions are rising up that meet many of the design criteria you’ve said are missing from the existing landscape.
I think the guidelines you offer are good ones, but would hope that you would simply adjust the temperature of the bath water of certain emerging approaches, rather than throwing it out and the baby with it.
For example, a number of approaches that do account for the dynamic, subjective nature of value, including some that I think you might have dismissed as not doing so, exist and are being used for specific purposes. One example is the Social Evaluator process and online tool, and another is SROI analysis as proposed by the 500-member and growing SROI Network (both of which SVT is involved with and champions among other tools and approaches).
Social Evaluator walks the user through the process of thinking through what one’s impact is, figuring out decent measures for it, and valuing those measures in dialog with stakeholders. This can be done sloppily or well, but all the fundamental steps are there and it’s a useful tool for building basic impact measurement capabilities in the field. Social Evaluator has naturally taken on the role of a bridge between the research community, that studies what interventions work best, and the practitioner community that’s out there doing the work. When a practitioner starts thinking about what to track often the first question is, what are others tracking? That prompts a conversation with those who’ve done the same kind of thing, and those who’ve studied what works, which can be distilled into metrics, and Social Evaluator is helping to underwrite these sorts of collaborations to simultaneously provide a useful tool and advance the state of practice. These metrics can be updated in real time as innovations are discovered in practice or by researchers- and ideally by both.
Social Evaluator doesn’t solve the problem of data collection, and it isn’t an automatic solution to everything, but it facilitates the thinking and consensus and connections that need to be made for this new discipline to really flourish. It does get groups to think more deeply about what impact they have, and to clarify what exactly they do, for whom.
SROI analysis is a different sort of effort that complements a tool like Social Evaluator but has its particular purpose as well. SROI analysis is getting to be a pretty sophisticated thing, kind of like accounting, that takes training to use and to understand. SROI analysis of the type now being advanced by The SROI Network basically asks that the user have a grounding in the SROI method, and investigate the context and process that was used to arrive at the results. An SROI ratio, like any other ratio, is meaningless when taken out of context, and the SROI Network is basically promoting something that requires skill to produce and use properly. Like a car, driving it without training is not a great idea. But like a car, it is quite useful when used skillfully.
I would love to hear you say more about how you see these and other existing efforts playing a constructive, if circumscribed, role, and what you would say their appropriate roles are.
As a business with a social objective our way of thinking about this is that “profit is redefined in human terms rather than pure quantitative analyses that remove human and social concerns in the name of profit”
The statement derives from the core argument from a paper describing a compassionate interpretation of capitalism which concludes that money represented by imagined numbers has trumped real human beings.
In an approach such as the unproven social impact bond, an attempt is made to relate social outcome to financial benefit. For example, what would have been spent by government in dealing with prison re-offenders after social investment has been made to reduce it.
The immediate problen I see with this is that for any social action there are many players, often volunteers, who might legitimately have a claim on such savings. If one organisation is deemed responsible, others will be disinclined to participate. If all are given credit, the country might soon be bankrupt.
For a simpler interpretation of social return, I offer the main outcome of sourcing a development initiative in Siberia which was aimed at providing emergency food relief and local economic development thru microfinance. With all financial investment returned the outcome included 10,000 new businesses raising the number of those generating incomes by a quantifiable margin in a city of 600,000.
Again from work in Ukraine which leveraged government action in doubling the allowances paid to adopters, we see after 4 years a quantifiable increase of 40% on domestic adoption.
I suspect that we may no more be capable of measuring the full impact of actions leading to a family having an income or a child having a family, than that which exists already, for those who benefit from traditional capitalism.
Business like ours which have a social objective aren’t investing their profits to be reimbursed. They are doing it because they are surplus to what we need for our own existence. We should be able to borrow from an ethical investor for seed capital without the need for complex investment instruments.
One comment to add. I still do see a need for a universal metric that all entities strive to use to quantify societal impact, not only to assess the impact of each initiative but also for various stakeholders to compare among initiatives, both within each respective sectors and among sectors.
I personally hope that the Global Reporting Initiative (GRI) be utilized not only for corporations but other entities in the non-profit sectors and etc. I do acknowledge the challenges that this poses in terms of objectivity and accurateness, together with the burden on entities to adopt this reporting initative. However, if all entities report under a universal metric, this would allow stakeholders to engage in intra-sector comparison together with cross-sector comparison of societal impact. And I belive that the ability to compare and trend using a common metric would be the great value for such initative, even when all the negatives associated with going this route is taken into consideration.
I definitely would like to thank Geoff again for the great insight, as the method he proposes gives us a great way for entities funding societal impact initatives a way to better evaluate the options available to fund. And lastly, a big thanks to Stanfordd Social Innnocation Review for allowing us access to valuable insights.
Fascinating and pithy argument - thank you. I wanted to query one element:
“For instance, several European foundations that support undocumented migrants have developed the demand side of this emerging social market by encouraging larger NGOs and public authorities to allocate resources (for example, for housing and health care) to it. On the supply side, these foundations have funded promising projects that are more effective at meeting the needs of this group.”
Isn’t the example you give of how to encourage the demand side for underserved groups still quite similar to traditional approaches of top down organisations providing resources/services into perceived gaps in need? I’d argue that’s really another type of more efficient work from the supply side - efficient because it is supplying to comparatively underserved beneficiaries.
What then would be an example of truly encouraging more effective development of the demand side? How about the wording you used originally to describe this: “help less powerful players have a voice in the market”? That could be giving underserved groups tools to coalesce and find their voice such as access to the public sphere via the media or the development of organisational structures so that those groups can form and advocate a cohesive point of view. The Young Foundations’ own Uprising program model is a great example, albeit focused on selected individuals rather than broader groups. A key aspect of that program that is so impressive is that participants are not directed towards a particular social problem they should be addressing, but rather given the tools to identify a demand and then advocate for it so that supply is provided (hopefully in effective ways).
As other commentators have said, a useful dissection of some of the key issues surrounding social value – and one that deserves debate. I take the central message of the piece as being that “the greatest contribution that funders can make is often not to measure value, but to forge the links between supply and demand that will later generate value”. This seems to me to be a more mature focus than trying measure value well enough for evidence based policy or funding decisions. Nevertheless I have a couple of quibbles and comments.
Firstly I suggest that using an analogy of physical laws and exploration is too obviously inappropriate to be worth pursuing, except to make the most basic point. A much more interesting comparison is with the biological sciences, the field where systems thinking originated. The sort of multi-dimensional influences the Geoff highlights as applying in the social sciences have direct parallels in biology – most obviously perhaps in ecology. “Invest(ing) in effective supply by supporting promising projects and collecting evidence of what works” is close to the scientific model of developing a hypothesis and then testing it. There is an interesting discussion on this (the article and the posts that follow) on Ben Goldacre’s Bad Science site at http://www.badscience.net/2011/05/we-should-so-blatantly-do-more-randomised-trials-on-policy/
Second I’m unclear about the third purpose in the triad of “accounting to external stakeholders, managing internal operations, and assessing societal impact”. In my company, Hall Aitken, we ask our clients to be clear about their priority purpose in evaluation, using three options of: reporting to funders, internal improvement and accounting to their community. The first two are obviously the same as those in the article but I’m not sure who the audience or ‘customers’ are for ‘assessing societal impact, and I’d welcome some clarity on that.
Finally I just want to emphasize that a large number (possibly the majority) of users of SROI and other approaches would agree that with Geoff’s conclusion that “what matters is the quality of the discussion and negotiation, and the depth of the learning”. But at the moment the temptation to focus on the end point (the SROI ratio for example) and not the journey (the thinking about how to get the best impact) deflects from this interest. So any shared frameworks need to include consideration of how outcomes are achieved and not just what they are.
The Purdue Peace Project measures its “social value” (assuming that peace is a social value) as a prime factor in decision-making:
All PPP Projects Are Selected Based on Pre-Determined Criteria – the most important criterion is whether PPP’s work with local citizens can reduce the likelihood of political violence.
· The PPP Director gathers data from multiple sources to determine whether or not that criterion is met before a project begins.
· Before a project begins, the PPP Director also travels multiple times to the communities with whom PPP may work and meets with local actors to build relationships, gather information, and see if local actors would like to collaborate with PPP.
To Monitor Our Work’s Progress…
· PPP’s West Africa Program Manager and our local collaborators monitor our work’s progress regularly by working with local peace committees frequently, through observation of their activities, and by talking with opinion leaders and other community members about their perceptions of our work.
· PPP researchers have weekly Skype calls with our West Africa Program Manager and our local collaborators in order to discuss our work’s progress and impacts.
· PPP researchers also monitor media coverage related to our work and the region we are working in every day.
· PPP researchers discuss our work’s progress during weekly team meetings. These discussions serve as good brainstorming sessions for moving our work forward.
· Jessica Berns, the consultant to Milton Lauenstein (PPP’s benefactor), also serves as a valuable source of feedback in moving our work forward. Jessica and Stacey have weekly phone calls.
To Assess Our Work’s Impact….
· PPP collects and analyzes data at multiple points in time before, during, and after a project.
· These data consist of (a) secondary data such as media coverage and online reports from other organizations (non-governmental organizations, governments, etc.); (b) focus groups, (c) one-on-one interviews, (d) surveys (when culturally appropriate), and (e) observations. PPP researchers record all focus group and interview data (after receiving permission) and transcribe these recordings verbatim. PPP researchers then analyze the data in various ways to measure impact.
· The way PPP assesses impact depends on the nature of the project, but in general we look for (a) changes in perceptions and behaviors among individuals over time, (b) changes to media coverage of the issue and the local peace committee over time, and (c) other kinds of outcome changes (e.g., the Berekum chieftancy dispute being resolved; local peace committees preventing violence in neighboring communities; local peace committee members being asked to serve on regional or national task forces ).
· All of our claims about our work having impact are grounded in empirical data, collected over time.
· We also consider what other international or domestic non-governmental organizations are working in the area when making claims about our work’s impact.
PPP Conducts Post-project Evaluation and Data Collection (3 months, 6 months, 12 months, and 18 months after a project’s completion).
COMMENTS
BY Marc Brenman
ON August 12, 2010 03:42 PM
Perhaps I don’t understand, since this seems to be an excerpt from a book, but “there may be no economic value in a beautiful sunset, an endangered species, or a wonderful work of art” is pretty clearly wrong. Real estate with a nice view is worth more, for example. In the San Francisco Bay area, we say that a view of each bridge is worth an additional $100,000 on the value of a house. Regarding endangered species, I worked on a case involving a gasoline pipeline that had to be re-routed at a cost of $1 million a mile to avoid an endangered salamander under the Endangered Species Act. And in regard to works of art, the newspapers weekly have stories on outrageous prices paid for them.
BY Sara Olsen
ON August 12, 2010 06:40 PM
Thank you for this eloquent discussion of the discrete purposes to which measures of social value can be applied, and for your ideas about the way in which the design of measures should reflect the inherently dynamic and subjective nature of this value. Having worked with investors, businesses, nonprofits and a few governmental entities for ten years on measuring impact, we at SVT Group have come to regard what is happening, and what you speak of, as the advent of a new business discipline, which we’ve begun to refer to as “Impact Management.”
This new discipline is larger than any one organization, metric, standard or even framework. Thousands of players, both inside and outside of academia, are contributing to its rapid emergence and evolution. This discipline reflects many of the phenomena you articulate- in particular, the difference between measures used for marketing, measures used to communicate with potential or actual investors, and those used to inform internal operations- the latter two having to do with understanding and communicating what is actually happening or may happen, and the former having to do with extracting the business value to be gained by the “social value proposition” one offers.
I would offer one critique of what you’ve said. For several years I have been hearing tee-up statements like “everyone is interested in social value” that come with a “but” clause similar to this one: “Alas, they cannot agree on what it is, let alone how to assess it,” followed by a new solution. This framing makes it sound like there is no progress being made and I think does a disservice to the massive, complex and powerful undertaking we’re all a part of.
What I am seeing, and would invite you to recognize more than I feel you have here, is that as more and more nonprofit and for-profit organizations and funder/investors are striving today to understand and manage the impact we are all generating, a market conversation is beginning to take place, and solutions are rising up that meet many of the design criteria you’ve said are missing from the existing landscape.
I think the guidelines you offer are good ones, but would hope that you would simply adjust the temperature of the bath water of certain emerging approaches, rather than throwing it out and the baby with it.
For example, a number of approaches that do account for the dynamic, subjective nature of value, including some that I think you might have dismissed as not doing so, exist and are being used for specific purposes. One example is the Social Evaluator process and online tool, and another is SROI analysis as proposed by the 500-member and growing SROI Network (both of which SVT is involved with and champions among other tools and approaches).
Social Evaluator walks the user through the process of thinking through what one’s impact is, figuring out decent measures for it, and valuing those measures in dialog with stakeholders. This can be done sloppily or well, but all the fundamental steps are there and it’s a useful tool for building basic impact measurement capabilities in the field. Social Evaluator has naturally taken on the role of a bridge between the research community, that studies what interventions work best, and the practitioner community that’s out there doing the work. When a practitioner starts thinking about what to track often the first question is, what are others tracking? That prompts a conversation with those who’ve done the same kind of thing, and those who’ve studied what works, which can be distilled into metrics, and Social Evaluator is helping to underwrite these sorts of collaborations to simultaneously provide a useful tool and advance the state of practice. These metrics can be updated in real time as innovations are discovered in practice or by researchers- and ideally by both.
Social Evaluator doesn’t solve the problem of data collection, and it isn’t an automatic solution to everything, but it facilitates the thinking and consensus and connections that need to be made for this new discipline to really flourish. It does get groups to think more deeply about what impact they have, and to clarify what exactly they do, for whom.
SROI analysis is a different sort of effort that complements a tool like Social Evaluator but has its particular purpose as well. SROI analysis is getting to be a pretty sophisticated thing, kind of like accounting, that takes training to use and to understand. SROI analysis of the type now being advanced by The SROI Network basically asks that the user have a grounding in the SROI method, and investigate the context and process that was used to arrive at the results. An SROI ratio, like any other ratio, is meaningless when taken out of context, and the SROI Network is basically promoting something that requires skill to produce and use properly. Like a car, driving it without training is not a great idea. But like a car, it is quite useful when used skillfully.
I would love to hear you say more about how you see these and other existing efforts playing a constructive, if circumscribed, role, and what you would say their appropriate roles are.
Thank you for your thoughts and leadership!
BY Jeff Mowatt
ON August 13, 2010 10:20 AM
As a business with a social objective our way of thinking about this is that “profit is redefined in human terms rather than pure quantitative analyses that remove human and social concerns in the name of profit”
The statement derives from the core argument from a paper describing a compassionate interpretation of capitalism which concludes that money represented by imagined numbers has trumped real human beings.
http://www.p-ced.com/1/about/background/
In an approach such as the unproven social impact bond, an attempt is made to relate social outcome to financial benefit. For example, what would have been spent by government in dealing with prison re-offenders after social investment has been made to reduce it.
The immediate problen I see with this is that for any social action there are many players, often volunteers, who might legitimately have a claim on such savings. If one organisation is deemed responsible, others will be disinclined to participate. If all are given credit, the country might soon be bankrupt.
For a simpler interpretation of social return, I offer the main outcome of sourcing a development initiative in Siberia which was aimed at providing emergency food relief and local economic development thru microfinance. With all financial investment returned the outcome included 10,000 new businesses raising the number of those generating incomes by a quantifiable margin in a city of 600,000.
Again from work in Ukraine which leveraged government action in doubling the allowances paid to adopters, we see after 4 years a quantifiable increase of 40% on domestic adoption.
I suspect that we may no more be capable of measuring the full impact of actions leading to a family having an income or a child having a family, than that which exists already, for those who benefit from traditional capitalism.
Business like ours which have a social objective aren’t investing their profits to be reimbursed. They are doing it because they are surplus to what we need for our own existence. We should be able to borrow from an ethical investor for seed capital without the need for complex investment instruments.
BY Dae-In Cha
ON August 19, 2010 12:10 AM
Thank you very much for the insightful review.
One comment to add. I still do see a need for a universal metric that all entities strive to use to quantify societal impact, not only to assess the impact of each initiative but also for various stakeholders to compare among initiatives, both within each respective sectors and among sectors.
I personally hope that the Global Reporting Initiative (GRI) be utilized not only for corporations but other entities in the non-profit sectors and etc. I do acknowledge the challenges that this poses in terms of objectivity and accurateness, together with the burden on entities to adopt this reporting initative. However, if all entities report under a universal metric, this would allow stakeholders to engage in intra-sector comparison together with cross-sector comparison of societal impact. And I belive that the ability to compare and trend using a common metric would be the great value for such initative, even when all the negatives associated with going this route is taken into consideration.
I definitely would like to thank Geoff again for the great insight, as the method he proposes gives us a great way for entities funding societal impact initatives a way to better evaluate the options available to fund. And lastly, a big thanks to Stanfordd Social Innnocation Review for allowing us access to valuable insights.
BY Michael Bosse
ON October 20, 2010 08:11 PM
Fascinating and pithy argument - thank you. I wanted to query one element:
“For instance, several European foundations that support undocumented migrants have developed the demand side of this emerging social market by encouraging larger NGOs and public authorities to allocate resources (for example, for housing and health care) to it. On the supply side, these foundations have funded promising projects that are more effective at meeting the needs of this group.”
Isn’t the example you give of how to encourage the demand side for underserved groups still quite similar to traditional approaches of top down organisations providing resources/services into perceived gaps in need? I’d argue that’s really another type of more efficient work from the supply side - efficient because it is supplying to comparatively underserved beneficiaries.
What then would be an example of truly encouraging more effective development of the demand side? How about the wording you used originally to describe this: “help less powerful players have a voice in the market”? That could be giving underserved groups tools to coalesce and find their voice such as access to the public sphere via the media or the development of organisational structures so that those groups can form and advocate a cohesive point of view. The Young Foundations’ own Uprising program model is a great example, albeit focused on selected individuals rather than broader groups. A key aspect of that program that is so impressive is that participants are not directed towards a particular social problem they should be addressing, but rather given the tools to identify a demand and then advocate for it so that supply is provided (hopefully in effective ways).
BY Jeremy Wyatt
ON October 17, 2011 08:27 AM
As other commentators have said, a useful dissection of some of the key issues surrounding social value – and one that deserves debate. I take the central message of the piece as being that “the greatest contribution that funders can make is often not to measure value, but to forge the links between supply and demand that will later generate value”. This seems to me to be a more mature focus than trying measure value well enough for evidence based policy or funding decisions. Nevertheless I have a couple of quibbles and comments.
Firstly I suggest that using an analogy of physical laws and exploration is too obviously inappropriate to be worth pursuing, except to make the most basic point. A much more interesting comparison is with the biological sciences, the field where systems thinking originated. The sort of multi-dimensional influences the Geoff highlights as applying in the social sciences have direct parallels in biology – most obviously perhaps in ecology. “Invest(ing) in effective supply by supporting promising projects and collecting evidence of what works” is close to the scientific model of developing a hypothesis and then testing it. There is an interesting discussion on this (the article and the posts that follow) on Ben Goldacre’s Bad Science site at http://www.badscience.net/2011/05/we-should-so-blatantly-do-more-randomised-trials-on-policy/
Second I’m unclear about the third purpose in the triad of “accounting to external stakeholders, managing internal operations, and assessing societal impact”. In my company, Hall Aitken, we ask our clients to be clear about their priority purpose in evaluation, using three options of: reporting to funders, internal improvement and accounting to their community. The first two are obviously the same as those in the article but I’m not sure who the audience or ‘customers’ are for ‘assessing societal impact, and I’d welcome some clarity on that.
Finally I just want to emphasize that a large number (possibly the majority) of users of SROI and other approaches would agree that with Geoff’s conclusion that “what matters is the quality of the discussion and negotiation, and the depth of the learning”. But at the moment the temptation to focus on the end point (the SROI ratio for example) and not the journey (the thinking about how to get the best impact) deflects from this interest. So any shared frameworks need to include consideration of how outcomes are achieved and not just what they are.
BY Milt Lauenstein
ON June 22, 2015 02:26 PM
The Purdue Peace Project measures its “social value” (assuming that peace is a social value) as a prime factor in decision-making:
All PPP Projects Are Selected Based on Pre-Determined Criteria – the most important criterion is whether PPP’s work with local citizens can reduce the likelihood of political violence.
· The PPP Director gathers data from multiple sources to determine whether or not that criterion is met before a project begins.
· Before a project begins, the PPP Director also travels multiple times to the communities with whom PPP may work and meets with local actors to build relationships, gather information, and see if local actors would like to collaborate with PPP.
To Monitor Our Work’s Progress…
· PPP’s West Africa Program Manager and our local collaborators monitor our work’s progress regularly by working with local peace committees frequently, through observation of their activities, and by talking with opinion leaders and other community members about their perceptions of our work.
· PPP researchers have weekly Skype calls with our West Africa Program Manager and our local collaborators in order to discuss our work’s progress and impacts.
· PPP researchers also monitor media coverage related to our work and the region we are working in every day.
· PPP researchers discuss our work’s progress during weekly team meetings. These discussions serve as good brainstorming sessions for moving our work forward.
· Jessica Berns, the consultant to Milton Lauenstein (PPP’s benefactor), also serves as a valuable source of feedback in moving our work forward. Jessica and Stacey have weekly phone calls.
To Assess Our Work’s Impact….
· PPP collects and analyzes data at multiple points in time before, during, and after a project.
· These data consist of (a) secondary data such as media coverage and online reports from other organizations (non-governmental organizations, governments, etc.); (b) focus groups, (c) one-on-one interviews, (d) surveys (when culturally appropriate), and (e) observations. PPP researchers record all focus group and interview data (after receiving permission) and transcribe these recordings verbatim. PPP researchers then analyze the data in various ways to measure impact.
· The way PPP assesses impact depends on the nature of the project, but in general we look for (a) changes in perceptions and behaviors among individuals over time, (b) changes to media coverage of the issue and the local peace committee over time, and (c) other kinds of outcome changes (e.g., the Berekum chieftancy dispute being resolved; local peace committees preventing violence in neighboring communities; local peace committee members being asked to serve on regional or national task forces ).
· All of our claims about our work having impact are grounded in empirical data, collected over time.
· We also consider what other international or domestic non-governmental organizations are working in the area when making claims about our work’s impact.
PPP Conducts Post-project Evaluation and Data Collection (3 months, 6 months, 12 months, and 18 months after a project’s completion).