Week 3: Fundamentals of questionnaire design

CHC9012 | National Taiwan Normal University

Overview

This week, we will cover the basics of questionnaire (or experimental) design. We will learn the technical terms to describe the methodology employed in a questionnaire or experiment, which will be essential for you when reading academic papers, attending conferences, and writing your own final term papers.

Experimental design variables & structure

We will discuss the following core concepts through several studies involving language:

  • Variables: Independent variable(s) and dependent variable(s).
  • Types of design: Within-participants design, between-participants design, and mixed designs.
  • Factorial design: What “factors” and “levels” actually mean.
  • Counterbalancing: Why there is a need to counterbalance, and how Latin square designs work to achieve it.

Discussion material for this week

Our discussion of these concepts will be based on the following two readings:

  • Abbuhl et al. (2014) (from the book “Research Methods in Linguistics”)
  • Rogers & Révész (2020) (from the book “The Routledge Handbook of Research Methods in Applied Linguistics”)


📊 Graphic Summary

Here is a visual breakdown to help you synthesize the experimental design concepts we are discussing this week:

📖 Concept definitions & examples

Concept Definition Example application
Independent variable The variable that is manipulated by the researcher, also known as the variable of influence. It is expected to bring about variation or change in the dependent variable. Corrective feedback
Dependent variable The variable that is measured or influenced. Grammatical development
Between-participants design A design where each level of the independent variable is comprised of different participants. Typically developing (TD) children and children with specific language impairment (SLI)
Within-participants design A design where the same individuals are assigned to all levels of the independent variable. A class studying half of target items under a spaced-short condition and the other half under a spaced-long condition
Mixed design A design that combines both within-participants and between-participants factors. Examining L2 performance using task complexity (within-participants) and modality (between-participants)
Factorial design A research design that includes more than one independent variable to examine separate and combined effects. Investigating the effects of wh-dependency and dependency length on sentence processing using a 2 x 2 design
Counterbalancing Having different, randomly assigned participants complete the different levels of the independent variable in different orders to address order or carryover effects. A Latin-square design is frequently used for this purpose to control against test- and task-order effects. Counterbalancing four different tasks across four groups in a Latin-square design so each group completes tasks in a different order
📝 Assignment for next week

Please complete the following tasks before our next meeting:

  1. Brainstorm: Review the concepts we’ve been through today. Start thinking of your own research question and how to translate it using the variables and design structures we’ve learned this week.
  2. Read: Prepare the materials for Week 4:
    • Schütze & Sprouse (2014)
    • Kaiser (2014)