Module 2:9 - Factorial Repeated Measures
Youtube Playlist Associated Files
Contents
Two Repeated Factors Linear Model[edit]
Model Effects:
- Main Effects - A, B
- α_{j}
- β_{k}
- 2-Way Interactions - A x B
- (αβ)_{jk}
- ε_{ijk} = The error term for the 2-way interaction
- Subject Effects - S, S x A, S x B
- ρ_{i} = Overall offset of the subject (difference of the subject to the grand mean)
- (ρα)_{ij} & (ρβ)_{ik} = Degree to which individual subjects respond to factor A and B respectively (error term in our one factor repeated measures model)
Ratings of Wine Example[edit]
There will be 15 judges. Each judge will rate a total of 8 wines. There will be 4 different types of wine but each wine will be labeled as Cheap or Expensive.
Setup for the JMP Repeated Measures Add-In for Factorial Repeated Measures[edit]
For performing Factorial Repeated Measures, a useful tool within JMP would be the Full Factorial Repeated Measures ANOVA Add-In. This Add-In allows one to generate linear models of mixed effects for full factorial designs analyzing a continuous variable.
Download link: https://community.jmp.com/docs/DOC-6993
Repeated Factorial designs require a cross of all factors where researcher must the subject responses on all possible conditions (within-sujects factors) or one condition(between-subjects factors).
To demonstrate the usage of this Add-In, we'll use the dataset from WineRatings(4x2within), which only involves within-subjects factors for predicting wine ratings.
How to Use the Add-In[edit]
Before doing any analysis, make sure to perform the necessary quality check on the dataset, ensuring each subject has exactly one response for each condition.
After you have downloaded the Add-in, go to Add-Ins -> Repeated Measures -> Full Factorial(Mixed Model)
Parts of the Add-In
- Y, Response: The continuous variable we are trying to predict
- Within-Subject Factors: Independent variable that is manipulated to be tested at every level for each subject. Each subject should only have one response for each level tested.
- Between-Subject Factors: Independent variable where different groups are subjects are used to test each level of that variable
- Subject ID: Identification of the subject. Usually, this add-in will do automatically, but it's always good practice to manually identify each subject.
Wine Ratings Factors:
- Y, Response: Rating
- Within-Subject Factors: Wine, Label
- Between-Subject Factors: None (For analyzing this particular design, all the judges were tested at each condition)
- Subject ID: Judge
After clicking OK, a Fit Model output will pop up(right). One area you should look first is the Fixed Effect Test, which provides a brief overview of which factors and interactions are significant (significant factors and interactions are usually highlighted). For wine ratings, we can see that only Label has a significant main effect on Ratings due to its low p-value.
To generate plots and pairwise tests of individual factors and interactions, expand on "Effects Details." If you want to look at plots, click on the red triangle to "LS Means Plot." For our example, a visual observation on the Wine type plot shows us that there is some change going on, but isn't significant enough to claim a main effect on the ratings. Meanwhile, the Label plots shows a huge change beween Cheap and Expensive wines, where Expensive wines are rated significantly higher than Cheap wines.
Looking at the interaction plot between Wine*Label, we can see that all four types are nearly parallel to each other. Regardless of their label, the cheap wines were still rated lower than the expensive wines. These effects appear very stable and with a high p-value as indicated by the Fixed Effect Test, the interaction has no evidence of being significant.
Repeated Measures with Between-Subjects Factors[edit]
If an experimental design plans to test different subjects at different levels, then that design has a between-subjects factor involved. As a reminder, a factorial design is a type of experimental design where two or more factors, usually with discrete levels, takes on all possible combinations of those levels across factors. Therefore, all factors are completely crossed with each other. For factorial designs involving between-subject factors, each subject rates only one level of a specific factor rather than rating for every possible combination (within-subjects factors). In order to account for between-subjects factors, however, a nested design is required. In a nested design, certain levels of first factor will only occur within one level of the second factor and/or vice versa.
Teaching Efficacy Study Example
For our example, we'll measure teaching effectiveness across schools in different cities(Factor A) and instructor(Factor B). In a typical factorial design, we would measure every possible combination of instructors and schools for our data (bottom left). In real life, however, collecting this data would be very impractical since we would have to fly each instructor to each location, thus dropping schools for a new location. Instead, each instructor would cross with only one school in that specific city(bottom right). Once there is a betweens-subject factor involved, subjects are automatically considered nested within this factor (for our sample, the instructors factor is nested within Schools). To reiterate the related learning module: "When Subjects are treated as factors, and Factor A is a betweens-subject factor, Subjects are said to be nested inside A."
Advertising Example
In our next example, we'll look at how experts and novices differ when rating ads. Expertise will be our betweens-subject factor because each subject can only have one level of it. Another factor, the endorser, will be our within-subjects factors where the endorser is either a celebrity or a non-celebrity. All subjects, regardless of expertise, will have to rate both levels of the endorser factor. Therefore, each subject must should have two ratings total: one for the celebrity ad and one from the non-celebrity ad.
Mixed-Factor Repeated Measures - JMP Add-In[edit]
Setting Up the Model[edit]
To analyse a dataset with both between and within subject factors, proceed through the following steps:
(1) Ensure data quality using Analyze -> Distribution
(2) Identify within and between subjects factors
(3) Set up model using Full-Factorial Design (Mixed-Model) Add-In
Identifying Within and Between Subjects Factors[edit]
One way to determine if a factor is between or within subjects is by exploring the data using JMP's dynamic linking feature.
Here, one level under the Expertise factor has been selected. We can see that only half of the subjects provided a response for the selected level of this factor which implies that Expertise is a between subjects factor.
Here, one level under the Endorser factor has been selected. Looking at the Subject Number graph, we can see that every subject provided a response for this level of Endorser. This tells us that Endorser is a within subjects factor.
Modeling the Full-Factorial Mixed Model[edit]
To set up the analysis in JMP, use the Add-In for Mixed-Factor Repeated Measures as discussed above. Place the factors you have identified as between-subjects in the box labeled "Between-Subject Factors" and those you have identified at within-subjects factors in the box labeled "Within-Subject Factors".
JMP will produce an output like the one shown below.
Interpreting the Mixed Factor Repeated Measures Output[edit]
Effects[edit]
We start, as we always do, by looking at the main effects and the interaction. These outputs are found in the Fixed Effects Tests section.
In the above screenshot, we see that there is a main effect of Endorser and an interaction between Endorser and Expertise. Just from this information, we know that whatever effect there is of Endorser depends on the Expertise of individuals.
To follow up on these effects, we should go to the Effect Details section and pull up the Least Squares Means Plots.
For information on interpreting LS Means Plots, refer here [[1]] or here [[2]]
Follow-Up: The Interaction[edit]
Sometimes the interaction of two factors qualifies what we can say about the main effects of those factors. Consider the following interaction:
Notice that Endorser is plotted on the x-axis with Expertise as the overlay. We can immediately see that expert raters don't change much between the levels of Endorser while novice raters show a preference for ads with celebrity endorsers over those ads with unknown endorsers.
We may wish to follow up on these qualitative observations with some statistical tests. Here, we can use LS Means Contrasts to only consider the specific tests that we are interested in.
The following graphic shows the setup for the LS Means Contrast between Celebrity vs Unknown for Experts and the resultant test statistic.
This test confirmed that we have no evidence that expert raters change between the levels of Endorser.