Homework 5 Solutions
By Xinyue Liu (Adapted from Previous Solutions)
1. Why might researchers want to use paired design and analysis rather than
... [Show More] just
independent samples? (2 points completion only)
Using paired design allows you to greatly reduce within-group variation, by controlling for individual
effects on treatment outcomes/effectiveness. For example, one individual might inherently have a
better immune system than others (which would contribute to within-group variation in an
independent sample), but a similar reduction in infection duration compared to other subjects (e.g.,
difference or ratio of descriptive parameter) that wouldn't contribute to across-group variation. Thus,
like ANOVA, paired design and analysis partitions across-group and within-group variation and
increases the likelihood of detecting a significant effect (which we'll call power soon). This means
researchers are more likely to calculate a p-value less than their pre-selected alpha if the effect
actually exists.
2. We always use Two-Box sampling without re-centering for computing confidence
intervals of effect sizes, such as the difference between medians. Why can’t we use
Big Box sampling for confidence intervals as we sometimes do for NHST? (2 points
completion only)
When we calculate confidence intervals, we are examining the difference between two groups if
those groups were resampled 10,000 times in an effort to learn about the population differences. In
order to measure the difference between two groups, the two groups need to be kept separate.
NHST, on the other hand, is a measurement of the probability of getting our result given that the null
hypothesis is true, rather than a measurement of difference between two groups, so we had to
pretend the null hypothesis and either combine the two groups' data into one Big Box OR re-center
both groups' data at 0.
3. When do we use a multiple-testing correction? Why is it important? (2 points
completion + 2 points correctness)
When we do NHST for potential F-like values and we find that that there is a statistically significant
difference between groups, then we have to run pairwise tests for each combination of 2 groups in
our data. By doing this, we run multiple tests and have the inherent risk of obtaining false positives.
For example, in running tests on 20 different colors of jelly beans, we found one false positive for
green jelly beans causing acne. Since we want to be sure that this doesn't happen, we have to
reduce our cut off. This allows us to reduce the risk of finding false positives. [Show Less]