November 14, 2013
Stanford University Frances C. Arrillaga Alumni Center (Click for a map)
McCaw Hall
326 Galvez St.
Stanford, CA 94305
Access video, presentation slides, and other resources from the conference at FSG's conference archive page.
Next Generation Evaluation: Embracing Complexity, Connectivity, and Change
Program Overview
To be more effective in their efforts to solve chronic social problems, social sector organizations need to identify evaluation solutions that can meet the sector’s fast-growing need to learn from data in an increasingly complex, interconnected, and rapidly changing world.
On November 14, FSG and Stanford Social Innovation Review will convene nonprofit leaders, grantmakers, evaluators, and other interested organizations and individuals to explore the possibilities for learning and evaluation in the social sector. Topics will include:
- Key trends: What are the fundamental forces driving the need for social sector evaluation to evolve?
- Game-changing approaches: What are the most promising methods and examples of how social sector players are practicing evaluation in light of these new trends?
- Future of the sector: What are the implications for the social sector and individual organizations moving forward?
Three game-changing approaches will be discussed in detail: Developmental Evaluation, Shared Measurement, and Big Data. These approaches have the potential to change how we view evaluation in a significant way.
- Developmental Evaluation: Developmental evaluation, which emerged as a new evaluation approach a few years ago, offers nonprofit organizations, evaluators, and funders a very different way of engaging in evaluation practice. Instead of operating against a fixed evaluation plan with a linear and predictive logic model, developmental evaluation offers a much more real-time, learning-oriented, feedback-based, and insight-driven approach to program design and early implementation of a program or initiative.
- Shared Measurement: When groups of organizations come together to co-determine outcomes and indicators that matter, share their data, and learn from each other, they engage in shared measurement. Coupled with a technology platform, shared measurement systems have the potential to build common understandings, reduce redundancies, identify gaps, and increase transparency, accountability, and alignment.
- Big Data: The explosion of social media and the development of an increasingly robust digital infrastructure have created a staggering amount of data. To express this new phenomenon, experts have coined the term big data, which includes sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, cell phone use, and more. Increasingly, big data is characterized by the “3 Vs”—sheer volume, greater variety, and increased velocity.
The day will include interactive exercises that allow participants to understand and interpret the key trends, presentations from experts and case study discussions on each of the three approaches, plus panel sessions as well as participant reflections that explore the “so what” and “now what” for the future of organizations and the sector as a whole.
To learn more about Next Generation Evaluation approaches and ideas, please click here to download a learning brief.
Sessions
Opening Keynote
Introducing Next Generation Evaluation
Hallie Preskill, Managing Director, FSG
The opening keynote will explain why we’ve come together to discuss the next generation of evaluation approaches in the social sector. The keynoter will review the trends that have emerged from working in complex, connected, and changing environments, and the ways in which evaluation needs to evolve to stay credible and useful. Then, she will discuss three new approaches to evaluation—developmental evaluation, shared measurement, and big data—and argue why these approaches are particularly worthy of our attention.
Morning Overview Presentations
Developmental Evaluation: An Approach to Evaluating Complex Social Change Initiatives
Kathy Brennan, Research and Evaluation Advisor, AARP
This session will introduce developmental evaluation—what it is and is not—and the core underpinnings of this approach to evaluating complex social change initiatives. This session will draw on real world experience and insights gained from recent developmental evaluation work at Living Cities and highlight the key learnings that came from using the approach.
Shared Measurement: A Catalyst to Drive Collective Learning and Action
Patricia Bowie, Consultant, Magnolia Place Community Initiative
Using a shared measurement approach to learning enables organizations to move quickly from an idea for change to actually testing and implementing that change. This session will introduce this approach by exploring how the Magnolia Community Initiative has used shared measurement and a learning system to provide the structure and functions to support collective actions.
Using Data for Good: The Potential and Peril of Big Data
Lucy Bernholz, Visiting Scholar, Center on Philanthropy and Civil Society, Stanford University
Big data is the talk of the town. It holds the promise of making previously invisible correlations visible and informing smarter decision making. But what are the costs associated with big data? What are the ethical dimensions of working with large data sets or passive data? And how should the sector think about these issues? This session will introduce the potential and peril of big data in the social sector.
Afternoon Case Study Sessions (Running Concurrently)
Frontiers of Innovation: A Case Study in Using Developmental Evaluation to Improve Outcomes for Vulnerable Children
Jamie Radner, Assistant Professor, School of Public Policy and Governance, University of Toronto
Based at the Harvard Center on the Developing Child, the Frontiers of Innovation project is a multi-stakeholder initiative to stimulate innovation nationwide to achieve better outcomes for vulnerable young children. The developmental evaluation team from the True Point Center for Higher Ambition works closely with the initiative’s decision-makers to apply multiple evaluative techniques to help continually improve the initiative. This session explores how developmental evaluation has facilitated real-time learning to support the initiative’s design and success.
Project SAM: A Case Study in Using Shared Performance Measurement for Community Impact
Sally Clifford, Program Director, Experience Matters; and Tony Banegas, Philanthropy Advisor, Arizona Community Foundation
The over-arching goal of Project SAM (Shared Asset Measurement) was to create a shared performance measurement system for organizations providing financial education and empowerment services to economically distressed communities in Arizona. A cohort of 15 grantees worked together for one and one-half years to develop common logic models, outcomes, indicators, and data collection tools. This session explores the journey and the lessons learned, as well as their plans for the next phase.
UN Global Pulse: A Case Study in Leveraging Big Data for Global Development?
Robert Kirkpatrick, Director, UN Global Pulse
Global Pulse is an innovation initiative launched by the Executive Office of the United Nations Secretary-General, in response to the need for more timely information to track and monitor the human impacts of global and local socio-economic crises. This session will explore how Global Pulse functions as a laboratory through which the UN and its partners can explore how new sources of digital data and real-time analytics can help decision-makers understand changes in human well-being, and monitor the impacts of policy and program interventions.
Afternoon General Sessions
Panel: Implications for the Social Sector (“So What?”)
Lisbeth Schorr, Senior Fellow, Center for the Study of Social Policy; Fay Twersky, Director, Effective Philanthropy Group, William and Flora Hewlett Foundation; and Alicia Grunow, Senior Managing Partner, Design, Development, and Improvement Research, Carnegie Foundation for the Advancement of Teaching. Moderator: Srik Gopalakrishnan, Director, FSG
This panel will explore the implications of Next Generation Evaluation for the broad social sector. The panelists will react to the three approaches that have been discussed during the course of the day, share how these approaches are shaping their own work, and identify additional emerging approaches for consideration. They will also respond to additional questions from the moderator and the audience.
Small Group Discussion: Implications for Individuals and Organizations (“Now What?”)
These small group discussions will allow audience members to explore the implications of the three approaches to evaluation for their own organizations. Among the questions they might explore are, What ideas can audience members take home and implement right away? and What ideas need further dialogue and consideration? To enable more fruitful discussions, the small groups will be comprised of people who play similar roles at their organizations.
Closing Keynote
Embracing Complexity, Connectivity, and Change
Brenda Zimmerman, Associate Professor of Policy, Schulich School of Business, York University
When faced with complex challenges or interventions, one can choose to either reduce the complexity or embrace it. Traditionally, complexity reduction is seen as a prudent and appropriate approach because it creates clarity, focus, and practical ideas. Ironically, the opposite impact is often achieved because the very complexity is what needs to be understood. In this session we will explore how to embrace complexity in our evaluation practice.
TOP OF PAGE
Schedule
Check-in and breakfast are available 7:30 - 8:30am.
All sessions will be held in McCaw Hall except the Case Study Sessions at 12:30pm.
|
Time |
Session |
Speakers |
|
8:30 – 8:50am |
Welcome and Conference Overview |
Eric Nee, John Kania |
|
8:50 – 9:30am |
Opening Keynote: Introducing Next Generation Evaluation |
Hallie Preskill |
|
9:30 – 11:30am |
Overview Presentations |
|
|
Developmental Evaluation: An Approach to Evaluating Complex Social Change Initiatives |
Kathy Brennan |
|
|
Shared Measurement: A Catalyst to Drive Collective Learning and Action |
Patricia Bowie |
|
|
Using Data for Good: The Potential and Peril of Big Data |
Lucy Bernholz |
|
|
Lunch - Vegetarian and non-vegetarian box lunches available 11:30am - 12:30pm |
||
|
12:30 – 1:30pm |
Case Study Sessions (held concurrently) |
|
|
McCaw Hall - Frontiers of Innovation: A Case Study in Using Developmental Evaluation to Improve Outcomes for Vulnerable Children |
Jamie Radner |
|
|
Fisher LLL - Project SAM: A Case Study in Shared Performance Measurement For Community Impact |
Sally Clifford, Tony Banegas |
|
|
Fisher BMC - UN Global Pulse: A Case Study in Leveraging Big Data for Global Development |
Robert Kirkpatrick |
|
|
1:45 – 2:45pm |
Afternoon General Sessions (McCaw Hall) |
|
|
Panel: Implications for the Social Sector (“So What?”) |
Lisbeth Schorr, Fay Twersky, Alicia Grunow Moderator: Srik Gopalakrishnan |
|
|
2:45 – 3:30pm |
Small Group Discussion: Implications for Individuals and Organizations (“Now What?”) |
Table Discussions |
|
3:30 – 4:15pm |
Closing Keynote: Embracing Complexity, Connectivity and Change |
Brenda Zimmerman |
|
4:15 – 5:00pm |
Refreshments and biscotti |
|
Speakers
Tony Banegas, Philanthropy Advisor, Arizona Community Foundation
Tony Banegas is a philanthropic advisor with the Arizona Community Foundation. He joined the foundation in 2006 and is responsible for grantmaking, donor engagement, and special initiatives such as Latinos Unidos, Latina Giving Circle, the Arizona Family Economic Success initiative, and Project SAM (Shared Asset Measurement). Prior to joining the foundation Banegas spent 18 years in management positions overseeing a variety of projects concerning child development, health, behavioral health, prevention, adult education, child welfare, community development, and employment and training. In 2006 the President of Honduras appointed him as Honorary Consul of Honduras in the state of Arizona with jurisdiction over Utah, Colorado, and Nebraska.
Lucy Bernholz, Visiting Scholar, Center on Philanthropy and Civil Society, Stanford University
Lucy Bernholz is a visiting scholar at the Stanford Center on Philanthropy and Civil Society and a fellow with the Hybrid Reality Institute. She writes extensively on philanthropy, technology, information, and policy on her award winning blog, philanthropy2173.com, and is a frequent conference speaker and an oft quoted media source for NPR, The New York Times, and The Wall Street Journal. Bernholz serves on the board of the Craigslist Foundation, the NeXii Industry Standards and Advisory Board, and is an advisor to the Center for Digital Information. She is the author of numerous articles and books about the business of giving, including the Blueprint Series of Annual Industry Forecasts on Philanthropy and Social Investing, Disrupting Philanthropy, and Creating Philanthropic Capital Markets: The Deliberate Evolution.
Patricia Bowie, Consultant, Magnolia Place Community Initiative
Patricia Bowie is an independent consultant, who for more than 10 years has helped Los Angeles-based organizations, community-based initiatives, and community groups develop new ways to improve their work while sustaining their viability and commitment to their missions. One of her long-term endeavors is the Magnolia Community Initiative. Bowie’s expertise is in helping organizations develop cross-sector relationships to effectively and efficiently work together to achieve common goals. She believes that shared vision, shared measurement, and mutually reinforcing actions within communities and between organizations are the cornerstone to developing sustainable improvement over time.
Kathy Brennan, Research and Evaluation Advisor, AARP
Kathy Brennan is the research and evaluation advisor for the AARP’s Experience Corps program. She has more than 15 years of experience managing and evaluating government, nonprofit, and foundation social welfare initiatives. Brennan began her career in the nonprofit sector at the Urban Institute where she evaluated the Welfare-to-Work grants program. She then moved to Goodwill Industries International to manage workforce development efforts aimed at helping disabled and disadvantaged Americans find living wage jobs. Brennan has worked as an evaluation consultant for Innovation Network and independently, running mixed methods evaluations of domestic initiatives aimed at helping vulnerable populations. Before joining AARP she led evaluation efforts at Living Cities, a funding collaborative.
Sally Clifford, Program Director, Experience Matters
Sally Clifford is program director at Experience Matters, a nonprofit organization that connects individuals wanting social purpose engagement with organizations that could benefit from the time and talents of experienced adults. She has more than 10 years of experience serving Arizona’s nonprofit community in building their capacity to deliver on their missions. Before joining Experience Matters, Clifford served as executive vice-president of the Alliance of Arizona Nonprofits, a statewide membership organization where, among her many responsibilities, she managed Project SAM in partnership with the Arizona Community Foundation. Clifford also spent 17 years with American Express in a wide variety of roles within their Travel Related Services division.
Srik Gopalakrishnan, Director, FSG
Srik Gopalakrishnan is a director at FSG, supporting the firm’s Strategic Learning and Evaluation practice. In this role, he has worked with clients on a variety of projects, including developmental evaluation and shared measurement. Gopalakrishnan joined FSG in 2012 following more than twelve years of experience in the social and business sectors. Most recently, in his role as chief impact and learning officer at New Teacher Center, he created frameworks for impact measurement as well as systems and processes for data-driven learning and improvement. Prior to that, Gopalakrishnan was director of evaluation at the Ball Foundation. Over the course of his career he has led several rigorous formative, developmental, and summative evaluations, as well as projects around organizational change, strategic planning, and leadership development.
Alicia Grunow, Senior Managing Partner, Design, Development, and Improvement Research, Carnegie Foundation for the Advancement of Teaching
Alicia Grunow is the senior managing partner for design, development, and improvement research at the Carnegie Foundation for the Advancement of Teaching. She oversees the core capacities that support all Networked Improvement Communities: analytics, developmental evaluation, design and development, program technologies, and improvement research. Grunow completed the Improvement Advisor program at the Institute for Healthcare Improvement and currently leads the foundation’s efforts to adapt these methodologies for the field of education. She teaches improvement workshops and coaches improvement teams inside and outside of the organization. Before joining Carnegie, Grunow was an instructor in Stanford University’s Teacher Education Program (STEP), where she taught practices to support the academic achievement of English language learners. Previously, she taught for seven years in transitional bilingual and dual language elementary school programs.
John Kania, Managing Director, FSG
John Kania is a managing director at FSG, where he leads the firm’s consulting staff. His recent work at FSG includes strategy development and evaluation for a range of philanthropic clients, including the John S. and James L. Knight Foundation, the McCormick Foundation, Robert Wood Johnson Foundation, and numerous other private and community foundations. Prior to joining FSG, Kania was a partner at Mercer Management Consulting and Corporate Decisions, Inc., where he worked with clients in education, health care, and media. He began his career at Leo Burnett Company, a global advertising agency. He has been published in Stanford Social Innovation Review, The Wall Street Journal, and The Journal of Business Strategy and is a featured author of Learning from the Future, a leading contemporary text on scenario planning.
Robert Kirkpatrick, Director, UN Global Pulse
Robert Kirkpatrick is director of UN Global Pulse, an innovation initiative of the Secretary-General harnessing big data and real-time analytics for global development and crisis resilience. He was the founding chief technology officer of the Silicon Valley global health and disaster technology NGO InSTEDD, and the co-founder of Microsoft Humanitarian Systems. Kirkpatrick has spent more than 15 years developing solutions for public and private sector organizations, with a focus on organizational change. He has done fieldwork in Iraq, Afghanistan, Kashmir, Uganda, Indonesia, Cambodia, and post-Katrina New Orleans.
Eric Nee, Managing Editor, Stanford Social Innovation Review
Eric Nee is the managing editor of Stanford Social Innovation Review, published by the Center of Philanthropy and Civil Society at Stanford University, and co-host of the Social Innovation Conversations podcast channel. He has more than 30 years experience in the publishing industry, most of it covering the high tech industry. Before joining Stanford, Nee was a senior writer for Fortune magazine in the Palo Alto, Calif., bureau. He also helped Time Inc. launch eCompany Now (where he was executive editor), which later merged with Business 2.0. Before joining Fortune, Nee launched Forbes magazine’s Silicon Valley bureau, where he was bureau manager. He also served as editor-in-chief of Upside magazine for close to five years.
Hallie Preskill, Managing Director, FSG
Hallie Preskill is a managing director at FSG. She joined the firm in 2009 and leads its Strategic Learning and Evaluation practice, working on a wide variety of monitoring, evaluation, and learning projects for clients such as MasterCard Foundation, the Knight Foundation, Robert Wood Johnson Foundation, and the Kellogg Foundation. Prior to joining FSG, she spent more than 20 years in academia, including the University of Illinois. Preskill’s research has focused on evaluation capacity building, transfer of learning and training, evaluation use, and evaluation as a catalyst for individual, team, and organizational learning. She is the past president of the American Evaluation Association, and an editorial board member of the American Journal of Evaluation, the Canadian Journal of Program Evaluation, and Foundation Review.
James Radner, Professor, University of Toronto
James Radner is assistant professor at the School of Public Policy and Governance at the University of Toronto; a senior fellow at the Center on the Developing Child at Harvard University; executive director and co-founder of the Boreal Institute for Civil Society; and a senior fellow at the Munk School of Global Affairs at the University of Toronto. His teaching and research focus on domestic and international social and economic development, and the applications of quantitative and qualitative tools to improve the effectiveness of social initiatives. Radner is partnering with the TruePoint Center for Higher Ambition Leadership, the Center on the Developing Child at Harvard University, and Grand Challenges Canada in a series of projects focused on applying science to improve the life prospects of vulnerable young children.
Lisbeth Schorr, Senior Fellow, Center for the Study of Social Policy
Lisbeth (Lee) Schorr is a senior fellow of the Center for the Study of Social Policy, where she works to broaden the understanding of evidence as applied to the design and spread of complex initiatives. She is a national authority on how to improve the future of disadvantaged children, families, and neighborhoods. Schorr has held leadership positions at City Year, the National Academy of Science's Board on Children and Families, the Foundation for Child Development, and the National Selection Committee for Innovations in American Government. She is a member of the Institute of Medicine, the board of directors of the SEED Foundation, and the author of two books, Within Our Reach: Breaking the Cycle of Disadvantage; and Common Purpose: Strengthening Families and Neighborhoods to Rebuild America.
Fay Twersky, Director, Effective Philanthropy Group, The William and Flora Hewlett Foundation
Fay Twersky is director of the Effective Philanthropy Group at the William and Flora Hewlett Foundation, and a member of the board of directors for the UBS Optimus Foundation. Before joining the Hewlett Foundation in 2011, she spent a year advising the Rothschild Family Foundation on issues of strategy, organization, and measurement. Previously, Twersky served four years as a director and member of the leadership team of the Bill & Melinda Gates Foundation, designing and developing its Impact Planning & Improvement division. She was also a founding principal of BTW Informing Change, a strategic consulting firm. Twersky has authored numerous articles and report, including coauthoring “Listening to Those Who Matter Most, the Beneficiaries,” in the spring 2013 issue of Stanford Social Innovation Review.
Brenda Zimmerman, Professor, York University
Brenda Zimmerman is an associate professor of policy at the Schulich School of Business at York University, and the founder and director of the Health Industry Management Program for MBA students. Zimmerman is also an associate faculty member for the Social Innovation Generation at the University of Waterloo. Her primary research applies complexity science to management and leadership issues in organizations, especially health care and nonprofit organizations, experiencing high levels of uncertainty and turbulence. She has written articles, book chapters, and a co-authored book on the topic of complexity and management in health care, Edgeware: Complexity resources for Healthcare Leaders. Her latest co-authored book, Getting to Maybe: How the World is Changed, is a Canadian best seller that has also been published in Japanese and Korean.
Videos
Introducing Next Generation Evaluation
Hallie Preskill, Managing Director, FSG
Using Data for Good: The Potential and Peril of Big Data
Lucy Bernholz, Visiting Scholar, Center on Philanthropy and Civil Society, Stanford University
Contact Information
Carrie Pogorelc
Stanford Social Innovation Review
Email: [email protected]
Phone: (650) 724-3309
Cosponsors
FSG is an international nonprofit consulting and research organization dedicated to discovering better ways to solve social problems. FSG helps foundations, nonprofits, corporations—individually and collectively—achieve social impact. FSG employs 120 staff members at offices in Boston, Geneva, Seattle, San Francisco, Seattle, and Washington, DC. Our approach combines the analytical rigor and data-driven methodology of world-class strategy consulting with expertise in the management, organizational, and evaluation issues unique to the social sector. fsg.org
Stanford Social Innovation Review (SSIR) is an award-winning magazine and website that covers cross-sector solutions to global problems. SSIR is written for and by social change leaders in the nonprofit, business, and government sectors who view collaboration as key to solving environmental, social, and economic justice issues. Published at the Stanford Center on Philanthropy and Civil Society, SSIR bridges academic theory and practice with ideas about achieving social change. SSIR covers a wide range of subjects, from microfinance and green businesses to social networks and human rights. Its aim is both to inform and to inspire. ssireview.org
The Stanford Center on Philanthropy and Civil Society (Stanford PACS) develops and shares knowledge to improve philanthropy, strengthen civil society and effect social change. Stanford PACS is a research center for students, scholars and practitioners to explore and share ideas that create social change. Its primary participants are Stanford faculty, visiting scholars, postdoctoral scholars, graduate and undergraduate students, and nonprofit and foundation practitioners. As publisher of SSIR, Stanford PACS informs policy and social innovation, philanthropic investment, and nonprofit practice. pacscenter.stanford.edu