The Seductions of Quantification
Sally Engle Merry
243 pages, University of Chicago Press, 2016
Last year, during a trip to Zambia and Kenya, I learned that 75 percent of women in those countries were victims of gender-based violence. The sources for that statistic seemed credible: a Minister of Health in one country and the leader of a highly respected non-governmental organization in the other. Although I had seen many inspiring things during my trip, this sobering statistic stuck with me. When I mentioned it to a colleague, she said, “Well, it all depends on how it is measured.” And that, essentially, is the topic of Sally Engle Merry’s book, The Seductions of Quantification: Measuring Human Rights, Gender Violence, and Sex Trafficking.
Merry, a professor of anthropology at New York University, offers a thorough critique of what she describes as the “indicator culture” that shapes policy around the globe. Indicators are all systematic ways of organizing, comparing, and presenting information about various social phenomena, but they come in many forms. Adding to well-known economic indicators like gross domestic product, the international aid community is increasingly focused on designing new indicators to inform humanitarian and development work. It is with these that Merry takes issue.
As its title suggests, Merry’s book encourages us not to take these indicators as unvarnished facts: Don’t be seduced by quantification. Instead, Merry wants us to remember that indicators draw on subjective information about social phenomena, quantify it, package it, and then represent it as objective and reliable. “Quantification has a great deal to contribute to global knowledge,” she writes, “but it is important to resist its seductive claim to truth.” Using examples from three domains—violence against women, human trafficking, and human rights—she argues that the largely Western experts who develop such indicators simplify complex on-the-ground situations in ways that often fail to account for crucial cultural context. The quest for technical expertise shuts out participation from the Global South, and the people whose circumstances the indicators measure typically lack a voice in their construction. Merry argues this is risky because what gets measured shapes our sense of what needs to change. For a more nuanced understanding, she suggests we supplement quantitative measurements with qualitative, ethnographic methods.
To show how subjective choices shape seemingly objective indicators, Merry describes at length the development of dueling frameworks for measuring violence against women, by various UN organizations. One framework focuses on first-hand information about women’s experience of specific violent activities, with questions such as, “In the last 12 months, did your husband punch you, slap you, try to choke or burn you?” A second framework includes questions about emotional abuse and fear. And third focuses on specific injuries. Each brings a different approach to conceptualizing violence against women, quantifying it, and suggesting solutions. Even within a given framework, rubrics may vary to account for differences in context. The creators of the list of specific violent activities, for instance, modified it for Bangladesh to include “Burn/acid throwing, dropping from higher place, smash the finger or hand, kick in the abdomen,” as those are common forms of violence in that country. As Merry shows, the decision to favor standardization or customization is often arbitrary.
All of Merry’s topics are tough reads. Partly, this is due to the issues she tackles: violence, human trafficking, and the lack of basic human rights. But partly, it’s because the writing is dense, full of lists, acronyms, long descriptions of proceedings, and repetition. And although Merry’s overall points are mostly persuasive, she never provides a clear alternative. She calls for narrative accounts to complement quantification and argues that a wider range of people—including those most affected—should be able to contribute to the development of indicators. I generally believe in the importance of this kind of input, but Merry fails to make a persuasive case for the material difference it would make, and I found myself frustrated by her lack of specificity on how a better system might work.
In my experience, no measure is perfect. But regardless of cultural norms, all of the frameworks Merry describes point to significant problems with gender-based violence, human rights, and trafficking. Even admittedly insufficient indicators demonstrate a need to address such abuse—and in so doing, can push governments and the international community to take action. In fact, the indicator reports that Merry credits with having the most impact, the United States’ Trafficking in Persons (or TIP) Reports, do not live up to her standards for high-quality indicators. These reports, which rank countries on a variety of trafficking measures, are created solely by the US government and offer one of the simpler representations of the issues. And yet Merry notes that they bring significant attention to the problem of trafficking and drive needed resources toward solutions.
If you can get through the book, you will undoubtedly emerge a more skeptical consumer of the various indicators that get tossed around as truth. But as imperfect as they are, these indicators still provide general insights about dire violence and human rights abuses. I remain concerned about the high level of gender-based violence—as measured by any framework—in the countries I visited last year. But next time someone cites a statistic about it, I will be sure to ask, “Do you know what is included in that measure?”