3L

Course Content

How can we estimate preferences, traits, and other difficult-to-measure concepts?

How can we make survey data comparable across survey respondents, or even across separate surveys?

How can we account for the bias and uncertainty in our measurement?

How can we reduce large and complex behavioral data to its underlying dimensions?

This course focuses on methods to discover, understand and visualize latent patterns in data and is especially suited to students with projects using survey data and other forms of relational data used in political science, sociology, economics, business, marketing, and psychology. The course introduces students to measurement theory and methods of scaling techniques, integrating Multidimensional Scaling, Item Response Theory, and Ideal Point Estimation. The first part of the course will provide an overview of the foundations of these techniques and introduce students to the most common methods for scaling and “spatial” analysis and the visualization of latent patterns in survey and behavior data. The course will demonstrate how to interpret, measure, and visualize latent dimensions of data via a variety of scaling methods using the open-source programming language R. The course will also discuss a range of applications these methods to social science studies of relational and perception data derived from elite behaviour and surveys, especially for identifying latent preferences of political, economic and social actors. The course concludes with discussions of the most recent advances in the field, including applications for text analysis, and practical advice for those seeking to use such methods in social science research, relevant to the students enrolled.

The course first covers how to analyse data from scales found in surveys (such as Likert-type scales), focusing on surveys that ask respondents to place themselves and / or stimuli on issue or attribute scales. The course begins with approaches to scaling to generate bias-adjusted and latent spatial data from survey responses, such as the Aldrich-McKelvey scaling and ‘Basic Space’ scaling with Anchoring Vignettes as methods for addressing perceptual bias in the form of “Differential Item Functioning.”  The course next examines similarities and dissimilarities data and covers multidimensional scaling (MDS) with a focus on the SMACOF optimization method implemented in R as well as Bayesian applications to Metric Multidimensional Scaling. Next, the course covers unfolding analysis of rating scale data from surveys such as favorability scales for stimuli such as politicians or social groups. Finally, the course provides an extensive overview of IRT and ideal point estimation, generally focused on binary choice data, which includes those used in ‘roll call voting’ analysis of elite behavior in parliaments and courts. Here we will cover Poole and Rosenthal’s W-NOMINATE and Poole’s Optimal Classification unfolding method, as well as a variety of Bayesian analysis techniques for binary and ordinal choice data using Item Response Theory (IRT). An extensive range of Bayesian techniques is discussed, including Bayesian Aldrich-McKelvey Scaling, Ordinal and Dynamic Item Response Theory (IRT), Bayesian Multidimensional Scaling (MDS), and Bayesian Unfolding. The final section will discuss recent methods for scaling to a variety of different data types, including social media and text data, and the latest computation innovations to apply scaling methods to ‘big data’.

This course will enable students to derive latent spatial preference information and/or a dimensional structure from various types of survey and behavior data, which is applicable to a wide range of social science applications, academic and non-academic alike. Consumers of research based on these methods will also benefit from a deeper understanding of this type of methodology, its potential and its limitations.

Data types covered for Scaling, IRT and ideal point applications:

• Survey Response Data
• Likert Scales
• Voting data (e.g. legislative roll calls, votes on courts)
• Perceptual and Preferential Data
• Similarity data
• Text Data

Text(s)
Analyzing spatial models of choice and judgment, 2nd Edition 2020
Author(s): Armstrong, D. A., Bakker, R., Carroll, R., Hare, C., Poole, K. T., & Rosenthal, H.;  CRC Press. ISBN-13: 9781466517158
This book and other materials will be provided by the Summer School as part of your course material.

Course Objectives
Students will learn to use various computational methods to generate measures of ideology and preferences and understand the latent dimensional properties of social science data, including surveys and legislative data. Students will understand the theories behind these methods and the relationships between Item Response Theory, Ideal Point Estimation and other scaling methods. As these techniques are fundamental parts of much recent work in social science, students will be able to both understand and produce this research based on measuring concepts in this way.

Course Prerequisites
The course is designed to be accessible to social science graduate students of all backgrounds. However, students familiar with the R programming environment will find it easier to adapt to course content and assignments, so it is recommended to familiarize oneself with the basic structure of R/Rstudio, such as via the 1-day introduction to R offered the Sunday before the first day of class. In addition, the course assumes basic familiarity with general statistics (OLS and MLE).

Background knowledge required
Statistics
OLS = elementary
Maximum Likelihood = elementary

Computer Background
R = elementary

Introduction: Overview of Scaling Methods, IRT, and Ideal Point Estimation Core reading Ch 1 of Armstrong et al. Discussion of scaling methods in general and the general theory behind Ideal point estimation and IRT models. 

Scaling for Single-item scales Core reading Ch 2 of Armstrong et al.
Supplementary reading Zakharova, M. and Warwick, P.V., 2014. The Sources of Valence Judgments: The Role of Policy Distance and the Structure of the Left–Right Spectrum. Comparative Political Studies, 47(14), pp.2000-2025.

Scaling Issue Multiple Issue Scales Core Reading Ch 2 of Armstrong et al.
Supplementary reading Bakker, R., Jolly, S., Polk, J. and Poole, K., 2014. The European common space: Extending the use of anchoring vignettes. The Journal of Politics, 76(4), pp.1089-1101.

Similarities data and Agreement scores Core reading Ch 3 of Armstrong et al.
Supplementary reading Gross, D.A. and Sigelman, L., 1984. Comparing party systems: A multidimensional approach. Comparative politics, 16(4), pp.463-479.
Bornschier, S., 2010. The new cultural divide and the two-dimensional political space in Western Europe. West European Politics, 33(3), pp.419-444

Rating scale data Core reading Ch 4  of Armstrong et al.

Binary data and NOMINATE Ideal Point Estimation Core reading Ch 5 of Armstrong et al.
Supplementary reading Hix, Simon, Abdul Noury, and Gérard Roland. “Voting patterns and alliance formation in the European Parliament.” Philosophical Transactions of the Royal Society of London B: Biological Sciences 364.1518 (2009): 821-831.

Item Response Theory Core reading Ch 5 of Armstrong et al. Clinton, J., Jackman, S. and Rivers, D., 2004. The statistical analysis of roll call data. American Political Science Review, 98(2), pp.355-370.

Non-parametric Unfolding Methods: Optimal Classification Core reading Ch 6 of Armstrong et al.
Supplementary reading Rosenthal, Howard, and Erik Voeten. “Analyzing roll calls with perfect spatial voting: France 1946–1958.” American Journal of Political Science 48.3 (2004): 620-632.

Additional topics: Dynamic and Ordinal IRT Core reading Ch 7 of Armstrong et al

Additional topics: Expectations Maximization IRT Core reading Ch 7 of Armstrong et al  and Imai, K., Lo, J. and Olmsted, J., 2016. Fast estimation of ideal points with massive data. American Political Science Review, 110(4), pp.631-656.

Supplementary reading Barberá, Pablo. “Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data.” Political Analysis 23.1 (2014): 76-91. Martin, Andrew D., and Kevin M. Quinn. “Dynamic ideal point estimation via Markov chain Monte Carlo for the US Supreme Court, 1953–1999.” Political Analysis 10.2 (2002): 134-153. Slapin, Jonathan B., and Sven‐Oliver Proksch. “A scaling model for estimating time‐series party positions from texts.” American Journal of Political Science 52.3 (2008): 705-722.