Now the hard work is done. You have written your hypotheses, checked your conditions, calculated your statistics, but now what? What does this mean? 🤷
Our basic conclusion hinges on one of two outcomes: you are either going to reject your null or fail to reject your null. These are based off of the probability of obtaining our test statistic.
Recall that we collected data and used a test statistic to determine the probability of obtaining the observed results, or something more extreme, under the assumption that the null hypothesis is true. If this probability is below a predetermined threshold, called the alpha level (α), you can reject the null hypothesis in favor of the alternative.
If the p-value ≤ α (e.g. 0.05), reject the null hypothesis.
If the p-value > α (e.g. 0.05), fail to reject the null hypothesis.
❇️ Important Note: Never phrase the latter as "ACCEPTING the Ho or Ha"! ALWAYS "REJECT/FAIL TO REJECT"! It's important to note that failing to reject the null does not mean you accept it as true. It simply means that you don't have enough evidence to reject it.
The first, and most common way to conclude our significance test is using our p-value that is generated by our calculator. Remember, our p-value is the probability of obtaining our sample if we have a normal sampling distribution with the null value as our center. We conclude by comparing our p-value to our significance level (which is usually 0.05 unless otherwise noted). If our p-value is lower than our significance level (or our 𝞪), this means that it is unlikely to occur by random chance. Therefore, we have reason to reject our Ho. 😠
If our p-value is not lower than our significance level (or alpha level), then we fail to reject our Ho (not accept). This means that we do not have evidence to reject our Ho in favor of our Ha, but we also don't have evidence to completely accept our Ho as fact. 😵💫
Let's reason this out in terms of probabilities with regards to our hypotheses.
A small p-value indicates that the observed results are unlikely to have occurred by chance if the null hypothesis is true. The traditional cutoff for a small p-value is 0.05, which means that there is only a 5% chance of obtaining the observed results, or something more extreme, if the null hypothesis is true. If the p-value is below this threshold, it is considered statistically significant and you can reject the null hypothesis. If the p-value is above 0.05, it is not considered statistically significant and you fail to reject the null hypothesis. It's important to keep in mind that the p-value is only a measure of the evidence against the null hypothesis and does not measure the probability that the alternative hypothesis is true.
Small p-values indicate that the observed value of the test statistic would be unusual if the null hypothesis and probability model were true, and so provide evidence for the alternative. The lower the p-value, the more convincing the statistical evidence for the alternative hypothesis.
In contrast, p-values that are not small indicate that the observed value of the test statistic would not be unusual if the null hypothesis and probability model were true, so do not provide convincing statistical evidence for the alternative hypothesis nor do they provide evidence that the null hypothesis is true.
While a p-value is the most common way to conclude a test, we can also use our z-score to conclude a test.
Remember that a z-score is how many standard deviations we are above/below the mean. Therefore, if we have a z-score higher than 2, it is pretty unlikely to occur by natural chance (since we have checked our normal condition and know that we are dealing with a normal sampling distribution). This comes from the Empirical Rule that states that 95% of our data in a normal distribution falls within 2 standard deviations. 💯
Therefore, a z-score higher than 2 (or lower than -2) signifies that the probability of it occurring is likely less than 0.05. So we can conclude the same way as we did above with a p-value. It is especially easy to make a reject Ho decision when our z-score is really large (like 4+ or -4). If our z-score is in the range of -2 to 2, it is really hard to reject our Ho, so we will likely fail to reject without further information.
Here is the template you can follow when concluding a one proportion z test for population proportion (or a 1-Prop Z Test):
"Since (p value) </> (alpha level), we reject/fail to reject our Ho. We have/do not have significant evidence of ________ (our Ha in context)." 📝
The big three things you need to have in your conclusion to maximize our credit are: ❗
Compare p-value to significance level
Make a decision (reject or fail to reject)
Include context with inference to TRUE POPULATION PROPORTION.
image courtesy of: imgflip.com
A survey is conducted to determine whether a new advertising campaign is effective at increasing the number of people who are aware of a particular brand. The null hypothesis is that the advertising campaign has no effect on brand awareness, while the alternative hypothesis is that the campaign increases brand awareness. 📺
A sample of n = 500 people is selected, and 250 are shown the advertising campaign while the other 250 are not. The sample is then asked whether they are aware of the brand. The proportion of people in the campaign group who are aware of the brand is p̂ = 0.7, while the proportion of people in the non-campaign group who are aware of the brand is p̂ = 0.5.
The hypothesis test is conducted at a significance level of α = 0.05. The test statistic is calculated to be z = 2.8.
What is the p-value for this hypothesis test?
What is your conclusion about the null hypothesis?
The p-value for this hypothesis test is 0.0026. This means that there is a 0.26% chance of obtaining a test statistic as extreme as 2.8 if the null hypothesis is true. Since the p-value is less than the significance level of α = 0.05, we reject the null hypothesis. This suggests that the advertising campaign is effective at increasing brand awareness! ⭐