Ben Goldsmith is an associate professor of political science at the University of Sydney who studies international conflict, international public opinion, and U.S. foreign policy. In 2010, Ben began work on forecasting genocide and politicide. I first met him that year, in New Orleans at the annual meeting of the International Studies Association. We talked over lunch about statistical forecasting of rare and calamitous political events and have stayed in touch ever since. It's been great to see his project develop, and I'm glad to be able to spotlight it here.
You've been working for the past several years on a project to forecast onsets of genocide and politicide. Can you tell us more about that work?
We began our work in mid-2010 after receiving a grant from the Australian Responsibility to Protect Fund, an initiative of the Australian government. From the outset, one focus was to integrate machine-learning approaches into the efforts at forecasting genocide and politicide and other types of mass atrocities. So the grant helped support collaboration between political science, my field, and computer science, that of my co-investigator, Prof. Arcot Sowmya of the University of New South Wales, also in Sydney. Since the funding was mainly from the government (with some extra support from the University of Sydney), the focus was on creating tools that could be useful in policy applications. Specifically, we wanted to contribute to efforts to create reasonably accurate early warning systems for atrocities that could be useful in efforts at prevention.
The work was challenging, but I think we have made a good contribution. The cross-disciplinary collaboration has, I think, been very useful. From the computer science perspective, we have found that among all the tools at my collaborators’ disposal, an approach called generalized additive models (GAMs) appears to consistently produce the most accurate forecasts of genocide and politicide based on out-of-sample testing. From the political science perspective, I think our emphasis on predictors that tend to have high variation across time has been another contribution. We explicitly took this on as a potential way to improve on structural approaches and improve the accuracy of year-to-year forecasts with temporally dynamic predictors. Our models include things like political assassinations, the use of guerrilla tactics in internal conflicts, and changes in the number of soldiers in the military.
How has the work been received so far? Do you get the sense that any governments or advocacy groups are trying to use your forecasts to make better decisions?
I would like to think that the work has been received fairly well. There is certainly awareness about it among governments and advocacy groups. Part of the mandate of the grant was that we directly communicate our findings to these groups through in-person presentations and reports oriented towards policy makers. I cannot say whether or in what ways our work is being used to inform particular decisions. However, we have had considerable interest from the news media (including the New York Times but also some specialized U.S. publications that I think policy makers pay attention to, and good exposure in the Australian news media as well), and our website I believe has a decent number of hits. I think there is some awareness of what we have done, and so our forecasts and approach might be considered when risks are assessed.
In addition to two reports for policy makers and analysts that we have produced, with forecasts for the period 2011-2015, we have also published some academic papers. In particular, the most important I think is our 2013 article in the Journal of Peace Research, which publishes some of the best research using quantitative methods in conflict studies and international relations. This paper I hope can contribute to thinking about forecasting mass atrocities and encourage others to also submit their work to journals that provide transparency and rigorous peer review. In the paper we explain why we use a two-stage approach to forecasting genocide, and attempt to provide a comparison of the forecasting performance of our approach with that of Barbara Harff, who is a pioneer in the field and produced the model that I believe is most widely known and used among policy makers.
What's next for your project? Do you all plan to routinely update those forecasts?
We do plan to regularly update the forecasts. We also plan to continue our efforts to improve them. We have hopes for securing funding supporting work towards both of these goals in the near future. The University of Sydney has already provided some seed funding to begin some work towards re-thinking the definition and coding of “genocide,” which is something I think is important to undertake. We post news about publications, funding, and other activities on our website.
When and how did you become interested in working on this topic?
I became interested in this work in a rather indirect way. As a newly minted Ph.D. I had read an article by Gary King and Langche Zeng in the journal World Politics in 2001 titled “Improving Forecasts of State Failure.” While my own interests have been in the area of international relations and conflict between countries, I was really intrigued by the possibilities of both forecasting and also using the machine-learning type approaches discussed in this important paper by very highly regarded political scientists. I thought I might be able to take some of their advice about how to improve forecasts and apply it to a specific area. It was only somewhat later that it occurred to me that a focus on genocide and politicide would be both a potentially useful project, and also a somewhat limited and manageable one. After trying to read some of the literature on neural networks and support vector machines and other types of machine learning approaches, I knew I needed to collaborate with people who knew what they were doing with these tools. So, I began to seek out computer scientists after moving to an Australian university from Singapore in 2005. There are many excellent computer scientists here. I was lucky to find Arcot Sowmya at UNSW, because she and her then-Ph.D. student Dimitri Semenovich combined not only high-level expertise but also a genuine interest in the substance of what we were doing.