Big data—the massive data sets that we collect and analyze to help understand complex systems, and that we mine to reveal trends in new ways and at a scale and speed often impossible even a decade ago—is all the rage. It has transformed how we conduct science, business, public health, and humanitarian efforts.
So can big data help local communities, cities, and nations cope with the disruptions and inevitable surprises from climate change? This is the question I explored while participating in the Bellagio/Poptech Fellows retreat sponsored by the Rockefeller Foundation. Big data has transformed scientists’ ability to understand the climate system and develop plausible scenarios of the future. However, while those roles are important, ultimately society’s ability to cope with climate change will depend less on the accuracy of these projections and more on the level of “resilience derived from bottom-up community efforts.”
The answer to whether big data can help communities build resilience to climate change is yes—there are huge opportunities, but there are also risks.
Feedback: Strong negative feedback is core to resilience. A simple example is our body’s response to heat stress—sweating, which is a natural feedback to cool down our body. In social systems, feedbacks are also critical for maintaining functions under stress. For example, communication by affected communities after a hurricane provides feedback for how and where organizations and individuals can provide help. While this kind of feedback used to rely completely on traditional communication channels, now crowdsourcing and data mining projects, such as Ushahidi and Twitter Earthquake detector, enable faster and more-targeted relief.
Diversity: Big data is enhancing diversity in a number of ways. Consider public health systems. Health officials are increasingly relying on digital detection methods, such as Google Flu Trends or Flu Near You, to augment and diversify traditional disease surveillance.
- Self-Organization: A central characteristic of resilient communities is the ability to self-organize. This characteristic must exist within a community (see the National Research Council Resilience Report), not something you can impose on it. However, social media and related data-mining tools (InfoAmazonia, Healthmap) can enhance situational awareness and facilitate collective action by helping people identify others with common interests, communicate with them, and coordinate efforts.
Eroding trust: Trust is well established as a core feature of community resilience. Yet the NSA PRISM escapade made it clear that big data projects are raising privacy concerns and possibly eroding trust. And it is not just an issue in government. For example, Target analyzes shopping patterns and can fairly accurately guess if someone in your family is pregnant (which is awkward if they know your daughter is pregnant before you do). When our trust in government, business, and communities weakens, it can decrease a society’s resilience to climate stress.
Mistaking correlation for causation: Data mining seeks meaning in patterns that are completely independent of theory (suggesting to some that theory is dead). This approach can lead to erroneous conclusions when correlation is mistakenly taken for causation. For example, one study demonstrated that data mining techniques could show a strong (however spurious) correlation between the changes in the S&P 500 stock index and butter production in Bangladesh. While interesting, a decision support system based on this correlation would likely prove misleading.
- Failing to see the big picture: One of the biggest challenges with big data mining for building climate resilience is its overemphasis on the hyper-local and hyper-now. While this hyper-local, hyper-now information may be critical for business decisions, without a broader understanding of the longer-term and more-systemic dynamism of social and biophysical systems, big data provides no ability to understand future trends or anticipate vulnerabilities. We must not let our obsession with the here and now divert us from slower-changing variables such as declining groundwater, loss of biodiversity, and melting ice caps—all of which may silently define our future. A related challenge is the fact that big data mining tends to overlook the most vulnerable populations. We must not let the lure of the big data microscope on the “well-to-do” populations of the world make us blind to the less well of populations within cities and communities that have more limited access to smart phones and the Internet.
The big data revolution is upon us. How this will contribute to the resilience of human and natural systems remains to be seen. Ultimately, it will depend on what trade-offs we are willing to make. For example, are we willing to compromise some individual privacy for increased community resilience, or the ecological systems on which they depend?—If so, how much, and under what circumstances?
From Appropriate Technology to Appropriate Big Data
The opportunities and risks around big data for climate resilience reminds me of the dormant “Appropriate Technology Movement,” brought to prominence with Schumacher’s influential book Small Is Beautiful: Economics as if People Mattered. In the face of rapid technological growth, the appropriate technology movement promoted small-scale, decentralized, locally controlled and people centered technologies. Is it time for an “appropriate big data movement”—one that considers the needs of communities, captures the broader context in which they exist, and pushes society to confront the trade-offs in the decisions (or non-decisions) we are making?