2P AI-Enhanced Research Design

Charles Crabtree is an assistant professor in the Department of Government at Dartmouth College, director of the Fundamental Needs Lab, co-founder and past co-director of the Baltic LEAP foreign study program, and current co-director of the Department of Government’s Honors Program.

Course Content

This five-day intensive course offers social science researchers a comprehensive introduction to the practical applications of Large Language Models (LLMs) like GPT in their research workflows. Through hands-on activities, ethical discussions, and real-world examples, participants will learn how to leverage LLMs to streamline literature reviews, enhance observational and experimental research designs, and automate or expedite data collection. The course emphasizes practical takeaways, ensuring that participants leave with actionable skills to responsibly integrate AI into various stages of the research process. By the end of the course, participants will be equipped to critically and effectively use LLMs while maintaining research integrity and transparency.

Day 1: Introduction to LLMs and Their Capabilities

Objectives:

• Understand the fundamentals of LLMs.
• Explore the strengths and limitations of LLMs in social science research.

Session Breakdown:

1. What are LLMs?
   1. Overview of how models like Claude and GPT work.
2. Concepts: Transformers, training data, fine-tuning.
   1. Applications in Social Science
3. Examples of LLMs in political science, sociology, and economics.
   1. Hands-On: Prompt Engineering Basics
4. Crafting effective prompts.
   1. Ethical Considerations
5. Bias, misinformation, and responsible AI use in research.

Day 2: Using LLMs for Literature Review Assistance and Idea Generation

Objectives:

• Use LLMs to streamline the literature review process.
• Understand the strengths and caveats of AI-assisted research design.

Session Breakdown:

1. Automating Literature Reviews
   1. Using LLMs to summarize articles, group research, and generate annotated bibliographies.
2. Research Question Generation
   1. Crafting and refining research questions with LLMs.
3. Hands-On: Building a Research Design Outline
   1. Using LLMs to brainstorm hypotheses and research frameworks.
4. Discussion: Quality Control and Validation
   1. Verifying AI-generated content for accuracy and relevance.

Day 3: Data Collection and Analysis with LLMs

Objectives:

• Learn how LLMs can aid in survey design and analysis.

Session Breakdown:

1. AI-Assisted Survey Design
   1. Designing effective survey questions and pre-testing with LLMs.
2. Developing and Analyzing New Outcome Measures
   1. Sentiment analysis, topic modeling, and summarization with LLMs.
3. Hands-On: Developing Experimental Stimuli with LLMs.
   1. Using LLMs to create experimental treatments.
4. Discussion: Validity and Bias in AI-Driven Analysis

Day 4: Writing and Drafting Research Outputs with LLMs

Objectives:

• Leverage LLMs to improve the drafting and editing of research papers.
• Understand ethical boundaries when using AI in academic writing.

Session Breakdown:

1. Drafting Introductions and Abstracts
   1. Using LLMs to generate concise research summaries.
2. Editing and Enhancing Writing
   1. Tools and techniques for refining academic prose.
3. Hands-On: Improving Your Writing
   1. Use LLMs to improve a section of your current research project.
4. Ethics of AI-Assisted Writing
   1. Plagiarism, authorship, and transparency.

Day 5: Evaluating, Presenting, and Future Trends

Objectives:

• Understand best practices for presenting AI-assisted research.
• Discuss emerging trends and future directions for LLMs in social science.

Session Breakdown:

1. Evaluating AI-Generated Research Outputs
   1. Criteria for assessing quality and credibility.
2. Presenting Your Research
   1. How to communicate AI-assisted methods transparently.
3. Hands-On: Presenting a Mini-Project
   1. Students present their AI-assisted research design or analysis.
4. Future of AI in Social Science
   1. Trends, challenges, and opportunities.

Background Knowledge:

Maths

Linear Regression – Elementary

Statistics

OLS – Elementary
Maximum likelihood – Elementary

Software

R – Elementary

Students should have some familiarity with popular large language models, such as chatGPT, Claude, or Gemini. Software used for teaching: R and LLMs