Week 12 - Arranging the data

CHC9012 | National Taiwan Normal University

Overview

This week, we will learn how to arrange the dataset such that it is ready for further exploration.

Prepare your computer:

  • Arrangements: How to prepare your data for further analyses?
  • Checks: How to check that all the data are in order?

A brief summary of the essential elements:

  • load(...): Imports a dataset or workspace previously saved in .RData format.
  • read.csv(...): Imports external data, such as your stimulus material list, from a .csv file.
  • <- NULL: Used to remove unnecessary columns from the dataframe.
  • names(...): Used to view or rename the column headers of the dataset.
  • filter(...): Keeps only the rows that meet specific conditions (e.g., keeping only “Experiment” trials).
  • ifelse(...): Creates a new value based on a condition.
  • spread(...): Reshapes data from long to wide format.
  • left_join(...): Merges two datasets together based on common identifier columns (like ItemNo).
  • as.factor(...) / as.numeric(...): Transforms a variable into a category (factor) or a mathematical number (numeric).
  • ezDesign(...): Generates a heatmap to visually verify that the experimental design is balanced and no data is missing.
  • save(...): Exports your final, prepared dataset into an .RData file for future analysis.




Please complete the following tasks before our next meeting:


💭 Part 1: Plan what you need to do

✍️ Preparatory activity

We should never forget that coding is a tool to achieve a task. For our case, the main task is to prepare the dataset for further analyses, which means:

  • Checking that it is complete and that nothing is missing,
  • Deleting the columns/rows that are not necessary, and
  • Transforming (but not altering) the data

The best way is to look at the data we just imported into RStudio, and take notes about what needs to be done. I personally take physical notes on flying papers for this step.

Show the code to import and show the dataset (same as last week)
read.pcibex <- function(filepath, auto.colnames=TRUE, fun.col=function(col,cols){cols[cols==col]<-paste(col,"Ibex",sep=".");return(cols)}) {
  n.cols <- max(count.fields(filepath,sep=",",quote=NULL),na.rm=TRUE)
  if (auto.colnames){
    cols <- c()
    con <- file(filepath, "r")
    while ( TRUE ) {
      line <- readLines(con, n = 1, warn=FALSE)
      if ( length(line) == 0) {
        break
      }
      m <- regmatches(line,regexec("^# (\\d+)\\. (.+)\\.$",line))[[1]]
      if (length(m) == 3) {
        index <- as.numeric(m[2])
        value <- m[3]
        if (is.function(fun.col)){
         cols <- fun.col(value,cols)
        }
        cols[index] <- value
        if (index == n.cols){
          break
        }
      }
    }
    close(con)
    return(read.csv(filepath, comment.char="#", header=FALSE, col.names=cols))
  }
  else{
    return(read.csv(filepath, comment.char="#", header=FALSE, col.names=seq(1:n.cols)))
  }
}

data <- read.pcibex("../course-materials/Files/results_FAKE_FOR_CHC9012.csv")


Alternatively, just load the R dataset corresponding to the data we have already imported
load("CHC9012_DataImportedFromPCIbex.RData")

This is what the results table looks like. Let’s have a look at its columns, 18 in this file. There are two types: (a) PCIbex’s default columns, and (b) the columns that we asked PCIbex to log.

  • Results.reception.time: Time in the PCIbex server when the results were received. This can be used as the Participant ID, assuming that there are no two participants who submitted the results at the exact same time.
  • MD5.hash.of.participant.s.IP.address: Scripted IP address of the machine used by the participant.
  • Controller.name: Which controller was used to code the experiment.
  • Order.number.of.item: The order of presentation of the whole questionnaire, starting from the very first page.
  • Inner.element.number: Whether specific numbers were assigned to the element.
  • Label: The name given to the webpage in PCIbex (for example, Welcome page, Instructions, Exeriment, etc.).
  • Latin.Square.Group
  • PennElementType: The PCIbex element used for the action (TextInput, Scale, Dropdown, etc.).
  • Parameter: Breakdown of the elements used on a same page.
  • Value: What the participant wrote or selectedm or when an element starts or ends.
  • EventTime: Time point at which an action occured (the time corresponds to the PCIbex server)
  • Comments
  • ItemNo: The ID of the item (here, sentence)
  • Condition: The experimental condition
  • Group
  • UserAge
  • UserDegree
✍️ Preparatory activity, again

Look closely to the results file. What needs to be done?


💻 Part 2: Arrange the dataset with RStudio, according to the plan

You can find below how to use RStudio to perform the transformations listed above.


💾 Part 3: Check and save the data

💭 Checking process

We want to make sure that we did not corrupt our data when arranging them for further analyses. Otherwise, we can run further analyses and have results, but they will have no value!

We will mostly use the ezDesign() function from the ez package.

library(ez)
# Same number of item per condition for each participant? (should be '15')
ezDesign(data3, ID_1_TimeReceived, Condition)

# Which participants share the same IP?
# (hint that this could be the same participant filling the questionnaire twice)
ezDesign(data3, ID_1_TimeReceived, ID_2_IP)

# Number of times an item appeared by participant (should be '1')
ezDesign(data3, ID_1_TimeReceived, ItemNo)

# Were the items well dispatched in the two lists?
ezDesign(data3, ItemNo, Group)

# Were the participants well dispatched in the two lists? (should be '30')
ezDesign(data3, ID_1_TimeReceived, Group)

save(data3,
     file = "CHC9012_PCIbexDataPrepared.RData")

📝 Assignment for next week

Please complete the following tasks before our next meeting: