Free the Knowledge

Somewhat unwittingly, I have become regularly involved in discussions over the use, benefit, and quality of effectiveness research in the social sector. In general I’m a defender of the type of rigorous evaluation of social programs that has recently emerged from academic circles (see my three-part series on randomized control trials at the Financial Access Initiative blog).

But there are deep, deep flaws in the academic research model, if not in the methods employed—flaws that severely limit the knowledge we gain. The social sector continues to outsource knowledge creation to the academic sector, which is operating according to the rules of academe, not in the public interest. It’s time to free the knowledge, and social sector actors have a key role to play.

What are the problems with the academic research model?

First, the primary purpose of academic research is status within academic communities. To advance—and attain any job security—academics have to publish research in academic journals. Thus, academics are biased toward research that will be publishable in academic journals.

The academic journals, in turn, are biased toward novel research. This means they are generally not interested in research that confirms earlier findings, or that adds nuance and practicality—the kind of knowledge that social sector actors need—to earlier research. And that means that research doesn’t get done. The journals also have strong biases for positive results, which yields suspect research. Recently, studies have suggested that up to 80 percent of published medical research can’t be replicated (and therefore the results are not trustworthy). There is no reason to believe that social science research is significantly better.

Finally, there are strong incentives to lock up knowledge and data—both for journal editors and for researchers. The best research is based on a great deal of data collection, data that can turn into lots of published papers if the researcher keeps the data to themselves. Meanwhile, journal publishers have incentives to make subscriptions very expensive and private.

As a result of these dynamics, useful knowledge for the social sector coming from academic researchers is severely limited. Happily, the academic publishing model is likely in its death throes. More and more people are questioning the value and the business model of walled-garden, peer-reviewed academic journals. Recently Princeton University adopted a policy that prevents Princeton academics from granting copyright on their research to closed journals. Princeton’s move could be the start of one of the most socially important changes of our time by generating a torrent of open-access academic research.

But there is more to be done. Here’s what the social sector should do now to free the knowledge:

1) Don’t wait for academics to do quality research on issues you care about. The toolkits for designing studies and surveys are easily available. Start building knowledge on effectiveness of your programs now.

2) If you do work with an academic, require that any resulting papers be published in open-access journals or be made publically available.

3) Post the results of research you sponsor or are involved in and submit it to knowledge repositories like IssueLab.

Collectively, we can free the knowledge from the shackles of academia and make our whole sector more effective. 

Read more stories by Timothy Ogden.