By Aidan Donovan
November 17, 2020
Washington Post and Foreign Policy have published findings from TRIP surveys about presidential elections, impeachment, and American foreign policy strategy. Consensus support for a particular position among international relations (IR) scholars might be a strong rationale for policy action. We know that policymakers demand and utilize academic work under certain conditions. On the supply side, there is a push within the academy to increase the rewards for producing policy-relevant research. It matters what scholars think about international issues. The challenge is that the reasoning behind consensus positions within the academy is often masked by political differences and the format of traditional survey research.
Scholarly research is useful to policymakers when the underlying logic is coherent and when policy prescriptions are clear. Policy relevance is stronger when many scholars have similar scholarly suggestions. In areas where there is a consensus among scholars, we should attempt to understand the underlying reason(s) for scholarly agreement. Understanding the logic of their positions, and whether this is consistent among scholars, is just as important as the conclusion.
For example, 83 percent of scholars opposed President Trump’s decision to leave the Intermediate-Range Nuclear Forces Treaty (INF) with Russia. Some scholars likely believe that the treaty is an effective security tool, while others may view the treaty as ineffective (given alleged non-compliance) but view the departure as an unnecessary provocation. Scholars’ consensus opposition is clear, but the survey does not provide information on how each of the scholars reached this conclusion. Policymakers can also learn from scholars on topics where there is not a consensus, such as the effectiveness of economic sanctions. It would be worthwhile to know if scholars consider the same or different factors in evaluating the effectiveness of economic sanctions.
It is hard to learn from IR scholars without understanding why they think the way they do.
Learning from scholars can be challenging because scholars are overwhelmingly liberal, compared to a more divided electorate and policy community. In our 2017 Snap Poll, only 11 percent of scholars identified as conservative on economic or social issues. Almost two-thirds (62 percent) of international relations scholars self-identify as both economically and socially liberal. Survey questions and interpretations should reflect the complex relationship between ideology and expertise.
Scholarly credibility to speak on important issues needs to be accompanied by a coherent rationale for their consensus. On many issues, it is hard to know if the IR consensus is rooted in expert evaluation or political preferences. We asked scholars in 2017 whether President Trump’s unpredictable behavior has been an effective negotiation tactic, as he suggested in the campaign. Just 8 percent of liberal scholars say this has been effective, yet 45 percent of conservative scholars say it has been an effective negotiation tactic. There is both some theoretical support for projecting unpredictability or irrationality and a lot of skepticism from prominent scholars such as Stephen Walt. However, we are unable to determine whether theory and evidence, or politics, explains conservative scholars’ more favorable perspective on President Trump’s international negotiations.
“As a bargaining technique, the madman theory has a certain logical coherence to it … but the key lesson is that there is little or no evidence that the madman theory of diplomacy actually works” — Stephen Walt
The effect of politics on scholarly attitudes is exemplified by questions on the U.S. foreign aid budget. We asked in Snap Poll X (2017) whether the U.S. is spending too little, too much, or the right amount on foreign aid. The question did not mention any politicians by name, so respondents are more likely to report their genuine views on U.S. foreign aid. IR scholars overwhelmingly support greater spending on Aid. Overall, 82 percent of scholars thought the U.S. was spending too little and just 4 percent thought the U.S. was spending too much. Just 2 percent of liberal scholars and 16 percent of conservative scholars thought the U.S. should spend less on foreign aid. IR scholars supported increasing aid and opposed reducing aid regardless of political ideology.
Do you think the U.S. is now spending too little, about the right amount, or too much on foreign aid? (October/November 2017)
When we ask scholars to evaluate Trump’s performance specifically, just 3 percent of IR scholars report that he has done as well or better than Obama on “development and foreign aid” even though the U.S. aid budget as a whole has been consistent across administrations. The president’s rhetoric on aid and development may be somewhat significant in aid effectiveness, but the single biggest difference is that we are now asking about Trump (an especially unpopular figure among scholars) specifically. The average aid budget during Obama’s second term was $49.5 billion. The average aid budget from Trump’s first two years in office was just three percent lower at $48 billion. Aggregate aid has essentially been steady despite President Trump’s isolationist rhetoric. We asked in September 2020 whether Trump has done as well as or better than Obama on a list of issues, including “development and foreign aid.” Just three percent of scholars reported that Trump has even been equally as strong as Obama on this issue. Not a single self-reported Democrat answered affirmatively, and just 6 percent of Independents and 16 percent of Republicans did as well. A partisan-proof result is expected since Republican scholars are nearly evenly split between supporting Biden and Trump in the upcoming general election. The most likely explanation for these diverging results is that scholars considered their negative opinion of President Trump in evaluating his performance on foreign aid. If we asked the general 2017 question on the US foreign aid budget, we would likely find that substantive attitudes on aid have not changed much in three years.
Can we encourage evidence-driven guidance from scholars?
Providing value-neutral questions and response options can encourage more substantive, and less political responses. In Snap Poll XI (2018), we asked scholars if the powers of the U.S. president have increased under President Trump, compared to the three previous presidents. This is potentially value neutral because a strong executive is not necessarily good, bad, liberal, or conservative. Liberal scholars were just slightly more likely than conservative scholars to answer that the president’s powers had increased under President Trump. Among conservative scholars, 5 percent say presidential power increased a lot and 15 percent say it increased somewhat. Among liberal scholars, 7 percent say presidential power increased a lot and 26 percent say it increased somewhat.
Have the powers of the U.S. president under President Trump increased a lot / somewhat, remained the same, or decreased a lot / somewhat compared to the previous three administrations?
The poll also asked scholars if they thought President Trump overstepped the foreign policy powers of the Office of the President. While liberal scholars were only slightly more likely than conservative scholars to say that President Trump increased the power of the president, liberal scholars were overwhelmingly more likely to say that he overstepped his foreign policy powers. Just 13 percent of conservative scholars said President Trump overstepped his foreign policy powers, compared to 47 percent of liberal scholars. Scholars seem to be capable of putting aside their political values at times, but the question asked makes their political perspective more or less relevant to their responses.
It may be reasonable to assume that substantive knowledge and experience does, on average, make scholars more liberal. However, liberal substantially outnumber conservatives in a variety of fields. At least 80 percent of academics in english, history, and psychology are liberal, suggesting that self-selection may explain IR scholars’ liberalism as much as their subject-matter expertise. Conservative scholars will therefore be in the minority among academics on most political topics. Scholars as a whole could benefit from investigating whether observed divides are substantive or political when conservative scholars are in clear disagreement with the rest of the IR academy.
TRIP surveys of scholars, journalists, and think-tankers strive to gather expert opinions rather than politically-driven responses. Therefore, our survey questions must be designed in a way that intentionally attacks the issue of ambiguity in survey interpretation. Questions must be carefully worded to cue political beliefs and reactions only when intended. In our most recent snap poll, we asked scholars the following question: “Regardless of the substance of their foreign policy agendas, how effectively you believe each candidate would be in achieving his respective foreign policy goals over the next four years.” The phrase “regardless of the substance of their foreign policy agendas” reminds respondents to take each candidate’s goals as a given, which should produce more substantive responses regarding the candidate’s foreign policy capabilities.
Additionally, multiple-choice questions could be supplemented with open-ended questions requiring explanations allowing researchers to understand the underlying reasoning to their scholarly advice. Scholars are roughly split between thinking the threat of economic sanctions would decrease or have no effect on the probability of election interference from other countries. We know that scholars are also split on the effectiveness of sanctions overall. Asking for an explanation could elucidate if those the overall wariness toward sanctions, or something specific to election interference, is driving skepticism of the efficacy of using sanctions to discourage election interference. The University of Chicago Booth School of Business Economic Experts Panel is a useful guide. The survey provides on-the-record respondents the opportunity to explain their answers and identify their confidence level and the school tracks responses over time to allow for track-record evaluation. This methodology incentivizes substantive and thoughtful response and discourages and identifies unsound beliefs.
Expert surveys are a useful way to synthesize the research and impact within peer-review academic literature and to highlight analysis that may be useful to policymakers and voters. The success of survey responses in fulfilling this role is directly related to the proportion of each response that is a product of subject matter expertise and experience (instead of personal beliefs and ideology). TRIP surveys recognize the importance of careful wording and interpretation and the majority of IR scholars undoubtedly seek to provide impartial responses. Nonetheless, the academy could benefit from surveys that provide stronger incentives for careful reflection and substantive response. Policymakers need to be careful in using the results of expert surveys as justification for action in cases where the underlying logic is not clear. If scholars come to similar conclusions based on different reasoning, policymakers need to understand the differences and how each might apply to the present (real-world) situation. Surveys like TRIP can encourage policy relevance in the academy by incentivizing substantive and sufficiently-detailed responses that are directly useful to policymakers and politically-active citizens.
Aidan Donovan is a senior at the College of William and Mary, majoring in Economics and Government. He has worked as a Research Assistant for TRIP since February of 2019. His interests include law and economic policy, and he is particularly interested in understanding how scholars think and communicate with policymakers and the public.