Application: please email Melanie mels@essex.ac.uk to apply to attend this course.


Dominik Duell is Lecturer for Applied Quantitative Methods in the Department of Government at the University of Essex and received his PhD in Politics from New York University in 2014. He studies voting behavior, political competition, and electoral accountability in representative democracies with a special focus on how social identity relationships influence citizens’ choices. His work lies at the intersection of Political Economy, Political Behavior, Experimental Methods, and Political Psychology. In his research, Dominik builds on strategic and behavioral theories of voting and implements laboratory, field, and survey experiments to analyze how individuals evaluate their politicians’ performance, when they discriminate in favor of their social group, how they make redistributive allocation decisions, or how they coordinate their choices with their peers when forming electoral coalitions. His work is published in the American Journal of Political Science and Comparative Political Studies and has been supported by a number of different academic and government institutions, including the U.S. National Science Foundation and the French National Research Agency (ANR).

Course description:
Experimental research often generates only a few observations for data analysis: Researchers conducting a pilot study need to infer from little data whether fielding the main experiment is worthwhile or whether adjustments are necessary; ethically challenging research sometimes is not allowed to broaden data collection beyond a handful of subjects; or, budget constraints severely limit the number of observations per treatment or the number of treatments itself.

We often work with sample sizes for which standard parametric methods building on particular distributional assumptions and asymptotic theory are not appropriate. When researchers are unable to increase their sample size but are not willing to walk away from research questions on rare phenomena, it is essential to understand whether and how small-N studies run into problems with respect to statistical inference. In short, researchers need to know about statistical methods appropriate to learn from small samples to be more confident in their findings.

This course introduces methods for data exploration and statistical analysis of small samples. It combines non-parametric methods with re-sampling and simulation tools. For an estimate of a treatment effect, non-parametric methods require fewer assumptions about the underlying populations. Those methods can easily be paired with re-sampling methods to obtain the distribution of statistics in situations where small samples do not meet distributional assumptions or where the theory needed to support parametric methods is intractable. Simulation methods paired with appropriate visualizations allow us to meaningfully explore the raw data itself, to investigate the small sample properties of estimators, to draw descriptive inference, or to gain intuitions about which inferences may generalize to the target population.

In particular, the course starts out with covering theoretical properties of standard estimators when samples are small, demonstrating estimation biases in exemplifying small data sets and through simulations, linking the Rubin causal model and estimation of treatment effects to implication of randomisation techniques on the number of observations, introducing the concept of statistical power and evaluation of its dependency on sample size, and providing exercises on power analysis, simulations, and re-sampling in Stata and R.

Then, the course continues with ways to generate more robust estimates and more meaningful visualizations of treatment effects recovered in small samples. We look at non-parametric methods, in particular Binomial test, Sign test, Wilcoxon sign rank test, Wilcoxon/Mann-Whitney rank sum test, Kolmogorov-Smirnov test of equality of distributions, Tukey confidence interval, Fisher exact test, Spearman R, Kendall Tau, Somers’ D, Theil statistic and Theil-Sen estimator, as well as Kolmogorov confidence interval; and, compare performance of those non-parametric tests and estimators to their parametric equivalent. The course also includes some guidelines on how to use averaging and smoothing techniques for visualizations. Finally, non-parametric methods are then paired with simulation and re-sampling tools. Specifically, the course goes through some of the theory behind simulations and bootstrapping method, shows how to explore small sample properties of parametric and non-parametric methods by way of simulation, evaluates small sample behavior of bootstrapping, and illustrates how to augment experimental data through simulations and re-sampling.

Course Fee

The fee to attend this course is £400 unless booked with the Experiments Course based here at EssexLab then there is a discounted rate of £200.