Signaling and Confirming

More people than ever before are taking action online to address important issues—whether it’s by sharing a video about rescue workers in Syria or joining a virtual climate march. Technology allows social change organizations to increase their reach in a cost-effective way, leading to the conversion of more people into online and offline supporters.

Despite longstanding criticism of “low-barrier” actions constituting “slacktivism,” it’s become clear that these new forms of participation—backbones of movements such as the Arab Spring or the fight for marriage equality—are making a difference. Now foundations and universities are investing in research like The Media Impact Project to understand not if but how to measure civic participation.

Traditional metrics, of course, help organizations focus efforts on their missions, learn from what works, and optimize future results. Participation metrics amplify these benefits by allowing organizations to gauge the willingness and power of participants to create change. Think of measuring participation as understanding the “health” of current and potential communities an organization may call upon to support its mission.

A New Framework for Measuring Participation

It’s best to think about measurement as a holistic process, where participation measurement is an important component of a streamlined set of metrics rather than an add-on to an exhaustive list. To do this, we suggest working through these four steps:

Step 1. Define a small set of measurable goals.

When defining goals, stay focused on your organization’s most important work and develop a handful of goals that will drive toward impact.

In one case, a foundation focused on addressing climate change was struggling to narrow down its overarching goal of building political will to support climate action. By stepping back to answer, “What is the concrete change we want to see in one year, three years, five years?” the team was able to define a sequence of measurable goals and then set a timeframe for each goal to ensure accountability. The foundation, for example, decided that if US policymakers in influential states had not taken action to reduce emissions within two years, they would adjust their strategy.

Step 2. Detail “signaling” and “confirming” metrics.

Organizations should track two important metrics for each goal: those that indicate progress toward achieving goals (“signaling metrics,” such as Facebook “Likes”) and those that provide demonstrable proof that you have achieved the goals (“confirming metrics,” such as statements by influential decision makers).

Many organizations measure only signaling metrics, because they are generally quicker and easier to track. But only by tracking both signaling and confirming metrics, and drawing the relationship between them, can you tell whether your work is making a difference.

The nonpartisan, grassroots movement Everytown for Gun Safety launched a national campaign to encourage people to support laws to reduce gun violence, and—from the outset—determined both signaling metrics (including “Retweets” of its Twitter messages) and confirming metrics (such as the election of political leaders endorsed by the campaign).

Everytown for Gun Safety used this framework to define its participatory measures of success in the months leading up to the midterm elections.

Gathering signaling metrics as the campaign was underway allowed Everytown to make strategic adjustments in real time and provided continuous information to share with supporters. Letting stakeholders know when the campaign had collected one million “Gun Sense Voter” pledges, for instance, helped keep stakeholders engaged during the lead-up to elections and provided a clear signal of what voters would do at the polls. Gathering confirming metrics ensured that Everytown stayed focused on its ultimate goals—passing Initiative 594, for instance, which won with 60 percent of the vote in Washington state.

Step 3. Set and share aspirational but achievable benchmarks.

Next, attach specific values to each signaling and confirming metric—and if you can’t put a number on the metric, think of a new metric.

These benchmarks should be aspirational so that organizations can communicate them to supporters and inspire greater action, but they should also be achievable and not exceed the effort required to meet the goals. While five million petition signatures might be a bragging point, for example, getting there will just waste organizational resources if you only need 500 signatures to change the policy.

The Institute for New Economic Thinking, a think tank focused on building a global community of economic leaders, has set aspirational but achievable benchmarks for a new initiative to engage college undergraduates in a more pluralist approach to economic thinking. As a signaling metric benchmark, it aims to subscribe 1 percent of the US target population to its initiative by year two. As a confirming metric benchmark, it aims to increase students’ professed interest in the pluralism of economic instruction by 10 percent—across four target countries within three years.

Step 4. Gather data and learn from the metrics you’ve defined.

Organizations can use free online measurement tools (such as Google Analytics, Facebook Insights, and SurveyMonkey), as well as offline measurement tools (such as focus groups and one-on-one conversations). Choose the simplest possible tools to collect data and begin learning from day one.

The Milton Hershey School, a residential school for low-income youth, is using this framework to measure the impact of a new initiative that aims to integrate social and emotional learning into students’ home-life curriculum. Easy implementation is critical for this ambitious effort, so the team is using SmartSheet surveys and Excel to collect data and learn from day one, rather than waiting to build an advanced and expensive data system.

What Does This Mean for You?

We can measure the real impact of participation, and the more organizations that do so, the more we can gather a shared repository on what metrics and actions make a difference.

As you experiment with this framework (download a blank version here) consider these questions:

• How can you use this measurement process to take more risks?
• Which of your signaling metrics do you think are most likely to indicate eventual impact?
• Which metrics are members of your community likely to take the most pride in?

Read more stories by Emma Jane Bloomfield & Erin Morgan Gore.