Power is the probability that a test will correctly reject a false null hypothesis (i.e., the probability of not committing a Type II error). In other words, power is the ability of a statistical test to detect a true effect if it exists.
- Type I error (α): Rejecting the null hypothesis when it is true (false positive).
- Type II error (β): Failing to reject the null hypothesis when it is false (false negative).
- Power: 1−β, the probability of correctly rejecting the null hypothesis when it is false.
Key Components for Determining Sample Size
To calculate the required sample size for a study, several factors need to be considered:
- Significance Level (α): This is the probability of making a Type I error (commonly set at 0.05). It defines the level of risk you’re willing to take in rejecting a true null hypothesis.
- Effect Size: The magnitude of the difference you’re testing for, often represented as Cohen’s d for continuous data. A small effect size requires a larger sample size to detect.
- Power (1 – β): The desired probability of detecting an effect if one exists. A typical power value is 0.80 (or 80%), meaning you want an 80% chance of correctly rejecting the null hypothesis when it is false.
- Sample Size (n): The number of observations required to detect the desired effect size with the specified significance level and power.
Power analysis is typically used:
- Before conducting a study (a priori power analysis): To determine the sample size required to detect an effect of a given size with a certain degree of confidence.
- After collecting data (post hoc power analysis): To assess the power of a study that has already been conducted, though this is less recommended.
The Best Value of Power
In most industries, a power value of 0.80 (80%) is considered standard, meaning there is an 80% chance of detecting a true effect. However, in certain fields like medicine and pharmaceuticals, higher power (e.g., 0.90 or 0.95) may be desirable to ensure that important effects are not missed, especially in life-critical decisions.
Power and Sample Size Calculation in R
Let’s assume we are conducting a clinical trial to compare the effectiveness of two drugs. The response variable is continuous, and we expect a moderate effect size (Cohen’s d = 0.5). We want to ensure that the power of our test is 80%, and we are using a two-sided t-test at a 5% significance level.
Power Calculation to Determine Sample Size
library(pwr)
effect <- 0.5 # Moderate effect size (Cohen’s d)
alpha <- 0.05 # Significance level
power <- 0.80 # Desired power (80%)
Perform power analysis for a two-sample t-test
sample_size <- pwr.t.test(d = effect_size, sig.level = alpha, power = power, type = “two.sample”)
print(sample_size)
The output will show the sample size required for each group to achieve 80% power, given an effect size of 0.5 and a significance level of 0.05.
Conversely, if you already know the sample size, you can calculate the power using:
n <- 30
power_calculation <- pwr.t.test(d = effect_size, n = n, sig.level = alpha, type = “two.sample”)
print(power_calculation)