Penn Press public policy and international relations editor Bill Finan comments on the scope of peer review in light of current events in the Middle East.
Questioning the Limits of Peer Review in the Social Sciences
Peer review is a defining element of what it means to be a university press. The process of subjecting a book manuscript to outside evaluation by experts in the field is central to the integrity and purpose of the university press. For a trade press, the first question is Will it sell? For a university press, Is it a significant contribution to the scholarship? There are pitfalls in the peer review process, however, especially in the social sciences. The eruption of the Arab Spring and the upsetting of an entire field of study in political science is a case in point.
Peer reviewers serve two purposes: they provide quality control and also guidance on how to think about the content of the book manuscript the editor has acquired. The first ensures that facts, methodologies, and relevant citations to the discipline’s literature are, respectively, accurate, sound, and encompassing.
The second role—how to think about the content—is especially important both to the acquiring editor and the press: acquiring editors generally are not able to judge each manuscript in the same way an expert in the field can, since the expert is conversant with the most recent research and also has a specialist’s knowledge of the material that an editor, who is generally responsible for more than one area, or list, does not. The editor and the press rely on the peer reviewers much like a physician relies on a battery of tests to determine what illness a patient might be suffering from or as a scientist draws on the results of tests to determine the validity of a hypothesis.
This, of course, is how peer review should ideally work. But what happens when the body of work on which the peer reviewers have contributed to and rely on to measure and evaluate the judgments and work of others turns out to be wrong? That question is especially pertinent at this moment in the field of Middle Eastern studies.
The Arab Spring, the shorthand appellation that has come to be used to describe the revolts that have spread throughout the Middle East and North Africa the past 15 months, is the event that prompts this question (see http://anarabcitizen.blogspot.com/2012/01/origin-of-term-arab-spring.html for a concise historical genealogy of the term). The Arab Spring began in Tunisia in late 2010 with the self-immolation of a fruit and vegetable vendor. This sparked a popular uprising that saw the removal of the country’s long-time authoritarian leader. It then spread rapidly through the rest of North Africa and the Middle East, leading to the removal of Hosni Mubarak in Egypt and Muammar Qaddafi in Libya. And it continues today, as Syria’s Bashar Assad bloodily attempts to crush attempts to remove him from office.
The revolts in the region have caught nearly an entire field of study by surprise. A similar cascade of events between 1989 and 1992 in Eastern Europe and the Soviet Union caught another field of study—Sovietology—and left it surprised and bewildered. Even today what have become Russian studies specialists continue to debate just what exactly happened. Middle Eastern studies is now in the process of trying to understand just what has happened; one respected and long-time specialist said to me last year that “we all got it wrong” in terms of the scholarship on the Middle East.
What did they get wrong? The conventional wisdom—one of the baselines of scholarship used to evaluate the work of any specialist on the region by their peers—held that the leaders in the Middle East and North Africa would remain in power for the near term. Although the scholarship was aware of currents of discontent among the citizenry in the region, the prevailing view held that the political leadership would remain securely in place because of a “robust authoritarianism.”
One Middle Eastern studies scholar, F. Gregory Gause at the University of Vermont, has written directly on how the community got it wrong. His essay in Foreign Affairs last summer, “Why Middle East Studies Missed the Arab Spring,” was an honest attempt to explore how the field—including him—missed the signs of authoritarian instability in the Middle East and North Africa. As Gause wrote:
The vast majority of academic specialists on the Arab world were as surprised as everyone else by the upheavals that toppled two Arab leaders last winter and that now threaten several others. It was clear that Arab regimes were deeply unpopular and faced serious demographic, economic, and political problems. Yet many academics focused on explaining what they saw as the most interesting and anomalous aspect of Arab politics: the persistence of undemocratic rulers.
The field missed the revolution because it was focused on understanding why revolution would not occur. The tactics and strategies authoritarian rulers put into place to ensure that they would remain in power were the main variables on which the field generally placed emphasis and tried to explain. Now the debate begins, as it has with Gause, on where the specialists went wrong.
Did peer review fail? There is no simple yes or no answer. The political scientist Eva Bellin has written an important and considered paper on what was missed but also what needs to be saved in the earlier scholarship in “Reconsidering the Robustness of Authoritarianism in the Middle East: Lessons form the Arab Spring,” (Comparative Politics, January 2012). She notes that we cannot just dismiss the body of work that developed on why authoritarians remained in power so long since the work was not intended to predict the revolts that have occurred.
That is true. But we are still left with the quandary John Maynard Keynes lamented on the limits of theory, whether political or economic, when it comes to stasis and crisis: "Long run is a misleading guide to current affairs. In the long run we are all dead. Economists set themselves too easy, too useless a task if in tempestuous seasons they can only tell us that when the storm is long past the ocean is flat again." We need only change “economists” to “political scientists” to appreciate that scholarship, especially when attempting to define current history, has its limits—as does peer review.
Chechnya: From Nationalism to Jihad, by James Hughes at the London School of Economics and Political Science, was cited in "Boston Marathon Bombing: What Do Chechens Have Against the U.S.?," a recent post on the Foreign Policy in Focus blog. The post argues that Hughes' work "sheds some light on possible reasons that Chechen separatists might attack the United States."
In this wry and insightful memoir, distinguished American diplomat John Paton Davies, Jr. describes his upbringing and wartime adventures in Asia, encounters with key twentieth-century figures from Mahatma Gandhi to Joseph Stalin, and how he carried on after his Foreign Service career was cut short by McCarthyism. 
