I, Coder - Examining the Impact of Artificial Intelligence on Event Data Coding


TitleI, Coder - Examining the Impact of Artificial Intelligence on Event Data Coding
LocationSan Diego
SubjectConflict Event Data, Machine Coding, Space-Time-Comparability, Data Quality
Appeared InPaper Prepared for the 53rd International Studies Association Annual Convention

Abstract: The internet promises ad hoc availability of any kind of information. Conflict researchers are seemingly only bound by the effort needed to find and extract the information from international news sources, which have become available at a fingertip. This begs the question whether the sheer number of accessible news sources and the speed of the news cycle dictate an automated coding approach in order to keep up? Will the initial costs of implementing such a system outweigh the possible loss of information? We answer these questions for the "Event Data on Conflict and Security" project (EDACS)1 and carry out both human and machine assisted coding to generate temporal and spatial disaggregated event data for armed conflicts. In this pilot, we compare both approaches in a quantitative analysis and qualitatively by using spatial-temporal comparability measures. While the quality of human-coding exceeds a pure automated approach, a compromise between efficiency and quality results in a supervised semi-automated machine learning approach. We conclude by critically reflecting on the possible discrepancies in the analysis of these resulting datasets.

download pdf