Civic Engagement

Elusive Effectiveness

A look at how to determine the effectiveness of community advocacy and representation.

In 1999, Los Angeles reformed its city charter and created a system of neighborhood councils—groups designed to promote citizen participation in government and to make government more responsive to local needs. Now, 14 years later, there are 90 certified neighborhood councils, each representing an average of 38,000 residents.

The program is meant to give residents a voice, and the councils meet periodically with city officials to air their concerns. But like many other civic associations’ intentions, its goals of community advocacy and representation are somewhat intangible and hard to measure. For a long time, researchers have struggled to determine how to assess the effectiveness of such groups.

Kyu-Nahm Jun of Wayne State University and Ellen Shiau of the University of Southern California decided to analyze the effectiveness of the L.A. neighborhood councils by comparing multiple constituencies. They surveyed three groups of stakeholders—neighborhood council volunteer board members, project coordinators from the city’s Department of Neighborhood Empowerment, and city council staff—to find out how they rated effectiveness on several measures.

Jun and Shiau found that these groups had very different responses. The stakeholders in the study each view effectiveness from a different perspective and simply don’t share the same objectives, Jun and Shiau say. For example, local elected officials have to think about the needs of their entire district, not just those of one neighborhood council. On the other hand, the main goal of the project coordinators, who serve as “street-level administrators,” is the successful development of the neighborhood council system as a whole as outlined by the city charter.

Ray Dart, director of the business administration program at Trent University in Peterborough, Canada, has been examining the elusiveness of nonprofit effectiveness through small case studies. Jun and Shiau’s data-rich paper “really shows that the idea of effectiveness is a will-o’-the-wisp,” Dart says. “It is contested, it is socially constructed, and its meanings are interpreted through different kinds of political and sociological lenses. And it just seems to be a basic fact that there is no [single] thing [that is] effectiveness. It’s always going to be a plural.”

Projects to assess the effectiveness of civic associations face difficult tradeoffs, says Kenneth Andrews, a sociologist at the University of North Carolina who led a study in 2010 focusing on the Sierra Club. Studies that look at a specific organization necessarily ask questions that are specific to the goals of that group. But that brings up additional questions about whether the findings apply to other types of civic associations.

Nevertheless, “the effort to locate these organizations and their key stakeholders is a really powerful way to look at these questions,” he says.

Kyu-Nahm Jun and Ellen Shiau, “How Are We Doing? A Multiple Constituency Approach to Civic Effectiveness,” Nonprofit and Voluntary Sector Quarterly, 41, 2012.

Tracker Pixel for Entry


  • BY Daniel Bassill, D.H.L.

    ON April 17, 2013 08:05 AM

    Working with a PhD student from the University of Kansas my organization created an on-line documentation system in 1999-2000 to collect actions from members that were intended to achieve goals of the organization. They had build a version of this for a project in Kansas City where many different stakeholders were involved and they were trying to understand the impact of the group over time. My system is called OHATS (Organizational History and Tracking System) More than 1700 actions have been documented since 2000. You can learn about this at

    The challenges are huge. First, while we were able to get a small grant to build the platform, we could not get follow up money to keep updating the platform or to create summary reports. Thus, the first version of the platform was hard to use, fell victim to spammers, and thus was not used much between 2003 and 2006. In 2007 a volunteer built a new, interactive version, which is what’s now, but I still have not had the funds to educate people to use it, build summary reports, etc. Second, it’s extremely difficult to motivate people to use this. Even the PhD student who built it has not been an active recorder in using it. I’ve been the primary user because I see this as an organizational history that can be looked at in the future, only if the information is available.

    I have quite a bit of information on the OHATS site that can help people understand it, including a discussion forum with each metric chart where people can ask questions and offer their own help.

Leave a Comment


Please enter the word you see in the image below:


SSIR reserves the right to remove comments it deems offensive or inappropriate.