Hypothesis testing is a statistical method used to make inferences or draw conclusions about a population based on a sample of data. It is a critical tool in research and data analysis, helping us determine whether there is enough evidence to support a specific hypothesis about a population parameter.
In hypothesis testing, critical value and alpha value are related. A critical value is a point on the distribution of the test statistic that defines the values that cause the null hypothesis to be rejected. The critical value is calculated based on the alpha value and the probability distribution of the idealized model. The critical value divides the probability distribution curve into rejection regions and non-rejection regions.
The significance level, or threshold, set before collecting data to determine if the null hypothesis should be rejected. The alpha value is the probability of making a Type I error, and is typically set to 0.01, 0.05, or 0.10 based on criticality of the industry.
We compare the test statistic to the critical value. If the test statistic is more extreme in the direction of the alternative than the critical value, reject the null hypothesis in favour of the alternative hypothesis. If the test statistic is less extreme than the critical value, do not reject the null hypothesis.
The μ represents the true mean of the entire population. In hypothesis testing, particularly in the case of a one-sample t-test, your null and alternative hypotheses involve making claims about this population mean.
The null hypothesis H0 often asserts that the population mean is a specific value (e.g., μ=120 mmHg), while the alternative hypothesis H1 might claim that μ is either greater than, less than, or not equal to this value, depending on the type of test (one-tailed or two-tailed). The hypotheses are statements about the population mean μ. The sample mean (or x̄) is used to estimate μ and compute the test statistic, but it doesn’t appear directly in the hypotheses themselves.
Key concepts in Hypothesis testing
In hypothesis testing, the null hypothesis (H₀) represents the status quo or a statement of no effect or no difference. The goal of testing is not to prove the null hypothesis but to test whether there is enough evidence to reject it in favor of the alternative hypothesis (H₁), which reflects what you want to demonstrate or prove.
In hypothesis, we test for deviations e.g., in case of normality testing we wanted to test for the deviations from normality and the assumption (status quo) becomes the null. Likewise, in case of a drug test scenario, we are looking for deviations from the assumption (status quo) that the drug is ineffective. If the p-value is small, we reject the null and conclude the drug has an effect.
The null statement must always contain some form of equality (=, ≤ or ≥). Always write the alternative hypothesis, typically denoted with H a or H 1, using less than, greater than, or not equals symbols, i.e., (≠, >, or <).
Type I & Type II errors
Type I Error (False Positive:
Type I error occurs when you reject a null hypothesis that is actually true. Denoted by α (significance level). The best example would be convicting an innocent person. The probability of a Type I error is P(Reject H0∣H0 is true) = α.
Type II (False Negative):
Type II error occurs when you fail to reject a null hypothesis that is false. Denoted by β. The best example would be letting a guilty person go free.
The probability of a Type II error is P(Fail to reject H0∣H0 is false) = β.
The total error is sum of Type I and Type II errors in hypothesis testing.
By setting the null hypothesis to “no effect,” you’re protecting against falsely claiming that the drug works (Type I error) when it doesn’t.
If your data leads to a rejection of the null hypothesis, you can say there’s statistical evidence supporting the effect of the drug.
If you fail to reject the null hypothesis, it doesn’t mean the drug doesn’t work – it just means there isn’t enough evidence to prove that it does (this could happen if the sample size is small or the effect is subtle).