Elusive Effectiveness

In 1999, Los Angeles reformed its city charter and created a system of neighborhood councils—groups designed to promote citizen participation in government and to make government more responsive to local needs. Now, 14 years later, there are 90 certified neighborhood councils, each representing an average of 38,000 residents.

The program is meant to give residents a voice, and the councils meet periodically with city officials to air their concerns. But like many other civic associations’ intentions, its goals of community advocacy and representation are somewhat intangible and hard to measure. For a long time, researchers have struggled to determine how to assess the effectiveness of such groups.

Kyu-Nahm Jun of Wayne State University and Ellen Shiau of the University of Southern California decided to analyze the effectiveness of the L.A. neighborhood councils by comparing multiple constituencies. They surveyed three groups of stakeholders—neighborhood council volunteer board members, project coordinators from the city’s Department of Neighborhood Empowerment, and city council staff—to find out how they rated effectiveness on several measures.

Jun and Shiau found that these groups had very different responses. The stakeholders in the study each view effectiveness from a different perspective and simply don’t share the same objectives, Jun and Shiau say. For example, local elected officials have to think about the needs of their entire district, not just those of one neighborhood council. On the other hand, the main goal of the project coordinators, who serve as “street-level administrators,” is the successful development of the neighborhood council system as a whole as outlined by the city charter.

Ray Dart, director of the business administration program at Trent University in Peterborough, Canada, has been examining the elusiveness of nonprofit effectiveness through small case studies. Jun and Shiau’s data-rich paper “really shows that the idea of effectiveness is a will-o’-the-wisp,” Dart says. “It is contested, it is socially constructed, and its meanings are interpreted through different kinds of political and sociological lenses. And it just seems to be a basic fact that there is no [single] thing [that is] effectiveness. It’s always going to be a plural.”

Projects to assess the effectiveness of civic associations face difficult tradeoffs, says Kenneth Andrews, a sociologist at the University of North Carolina who led a study in 2010 focusing on the Sierra Club. Studies that look at a specific organization necessarily ask questions that are specific to the goals of that group. But that brings up additional questions about whether the findings apply to other types of civic associations.

Nevertheless, “the effort to locate these organizations and their key stakeholders is a really powerful way to look at these questions,” he says.

Kyu-Nahm Jun and Ellen Shiau, “How Are We Doing? A Multiple Constituency Approach to Civic Effectiveness,” Nonprofit and Voluntary Sector Quarterly, 41, 2012.

Read more stories by Corinna Wu.