Inlay

Political Scientist at the @UZH. IR good stuff & quant text analysis. Former Berliner, adopted Zürcher. Website: https://johannesscherzinger.com/

PS: the oft-overlooked Appendix and Replication Material offers clear guidance on how to set up LLMs and how to validate such an approach using human hand-coding, interviews, salience measurements, simple keyness measures, locally integrated models, and various other robustness tests!

Perfect day for a Lehrstuhlausflug!

🚨 New(ish) Publication Alert! Delighted to see my article Credit Claiming in the EU @thejop.bsky.social I explore a classic question for accountability in Europe: when and why do govs claim credit for the work of the EU and shift blame onto Brussels to avoid responsibility? doi.org/10.1086/732970

Finding 2: There exists great heterogeneity within the Global South, but still a majority is more supportive than the inst. average.

We conclude the article with four simple and transparent guidelines when using LLMs for IO/IR research. We hope that this will be part of a bigger conversation on how to move forward as a subfield with relatively scarce data (which AI could alleviate to some extent).

Beyond these findings, the stance dataset we provide lets users aggregate positions by individual member state, by UNSC member, and on by agenda item. Including two sentence summaries of their argument for or against sanctions.

Finding #1: The Global South is not as anti UN-sanctions as conventional wisdom suggests. We find that G77 states (taken as GS proxy) have been consistently supportive of UN sanctions for decades (see West P3 and Western Elected Council Members, plus China and Russia for comparison).

We start by showing under which conditions LLMs can be useful in the measurement of rev. preferences (e.g. when voting is heavily skewed, or severe select. bias exist). We maintain, however, that nothing beats subject-matter expertise and manual human hand-coding. We find the following below.

My friend @antonpeez.bsky.social and I wrote a new paper in @the-peio.bsky.social on how to use LLMs to measure revealed preferences in diplo. speeches. We use UN sanctions as a test case to illustrate the approach and offer a template for future IO/IR research. Thread below doi.org/10.1007/s115...

UN sanctions are an important tool for maintaining international peace and have shaped the global security order since the end of the Cold War. However, given contemporary UN Security Council (UNSC) dynamics, new sanctions regimes have become rare, and policymakers are pessimistic about the future of the tool. We evaluate this pessimism by offering a long-term perspective, analyzing all UNSC members’ positions on UN-mandated and unilateral sanctions from 1992 to 2023, introducing the UNSC Sanctions Stance Dataset. We examine 5,984 sanctions-related Council speeches using a large language model for measurement, and validate this procedure with hand coding, descriptive analyses, and expert interviews. Our description of the construction process and guidance on using LLMs for measurement serve as a template for creating LLM-coded and manually and substantively verified datasets from IO speech data. The analysis focuses on Russia, China, and the Global South, and traces their positions in detail. We find that Russia turned from limited support of UN sanctions in the 2000s to total opposition in the mid-2010s, while China has persistently been opposed, offering at most begrudging tolerance. The P3 (France, UK, US) have consistently been supportive. Meanwhile, states from the Global South hold a distinct intermediate position, opposing unilateral sanctions while cautiously but consistently supporting UN-mandated sanctions. This finding cuts against an assumption of Global South opposition to sanctions in general.

Incumbents in multilevel systems are assumed to exploit uncertainty of responsibility by claiming credit and shifting blame, yet little is known about when and how they engage in these rhetorical stra...