Visit the Museum


Learn about the Holocaust


Academic Research

Remember Survivors and Victims

Genocide Prevention

Antisemitism and Holocaust Denial

Other Museum Websites

< Preventing Genocide Blog

Comparison Survey Launch: Which countries are most likely to experience new mass killing in 2019?

Stay Connected

To help the Simon-Skjodt Center’s Early Warning Project forecast atrocity risk in 2019, please participate in our annual pairwise comparison survey, an innovative opinion aggregation method, which presents countries head-to-head and simply asks respondents to choose which is more likely to experience a new mass killing in the new year. The survey will run for one month, until December 31, 2018.

Many thanks to all who participated last year. More than 10,000 votes were cast, helping us define our priority countries to track in 2018. (See the 2018 report.)

How can I participate?

Go to our survey on Once you vote on one pair of countries, you will be presented with another, and you can repeat this process as many times as you like—even just a few minutes of your time will improve our results. 

What is a pairwise comparison survey?

A pairwise comparison survey involves a single question with many possible answers on which participants vote, one pair at a time. The response options presented to participants—in this case every country with a population greater than 500,000—are randomly generated at the beginning of the survey period. However, as the survey progresses and more people record responses, the survey mechanism “learns” from previous responses and presents increasingly difficult pairs. 

(Note that we previously called this methodology a “wikisurvey;” because the AllOurIdeas platform we use was designed to allow  participants to propose new voting options. In this case, the options are set to be every country with a population over 500,000. Since we are using the pairwise methodology without this “wiki” component, we call it simply a “pairwise comparison survey.”)

How does the Early Warning Project use the results?

The results are visible as the survey progresses, and will be published when it is finished (see our reports forecasting 2014, 2015, 2016, 2017, and 2018). The annual comparison survey is one of three quantitative methods used by the Early Warning Project to assess atrocity risk worldwide. Along with the statistical risk assessment and  public opinion pool, we present the comparison survey results to policymakers in the US and abroad, NGOs, and researchers to help them prioritize prevention efforts and further develop the crowd forecasting field.

How do you define mass killing?

The Early Warning Project’s Statistical Risk Assessment estimates the likelihood of a “mass killing” episode. We consider a mass killing episode to have occurred when the deliberate actions of armed groups, including but not limited to state security forces, rebel armies, and other militias, result in the deaths of at least 1,000 noncombatant civilians targeted as part of a specific group over a period of one year or less.

In countries where at least one episode of mass killing is already occurring—most notably Burma/Myanmar, Nigeria, North Korea, Sudan, South Sudan, and Syria (see the full list of ongoing mass killings)—we are not asking you to assess the chances that the ongoing episode will continue or intensify. Instead, we are interested in the risk that a new episode with a different target group and/or different perpetrator will begin in 2019. For example, there are currently two ongoing mass killings in Sudan, in Darfur and South Kordofan/Blue Nile, so there would only be a new onset if the government of Sudan began targeting another group (i.e. opposition party supporters) or a non-state armed group in Sudan perpetrated a mass killing.

  • A noncombatant civilian is any person who is not a current member of a formal or irregular military organization (e.g., militia, terrorist group) and who does not pose an immediate threat to the life, physical safety, or property of other people.
  • The reference to deliberate actions distinguishes mass killing from deaths caused by natural disasters or infectious diseases, the accidental killing of civilians during war, or the unanticipated consequences of other government policies. Generally, fatalities are considered deliberate if they result from actions designed to compel or coerce civilian populations to change their behavior against their will, and if the perpetrators could have reasonably expected that these actions would result in widespread death among the affected populations. Examples of such actions include, but are not limited to, forced mass starvation, the intentional confiscation or destruction of healthcare supplies, forced relocation, and forced labor.
  • To distinguish “mass killing” from large numbers of unrelated civilian fatalities, the victims of mass killing must appear to be perceived by the perpetrators as belonging to a discrete group. That group may be defined communally (e.g., ethnicity or religion), politically (e.g., partisan affiliation or ideology), socio-economically (e.g., class or profession), or geographically (e.g., residents of specific villages or regions). Unrelated executions by police or other state agents would not qualify as mass killing, but capital punishment directed against members of a specific unarmed political or communal group would.
View All Blog Posts