Statistical significance is the probability that the result of a given study could have occurred purely by chance. It reflects the degree to which observed results are true. Hypothesis tests aim to determine if the observed difference is statistically significant.
A hypothesis test evaluates statistical significance, whereas practical significance evaluates the significance of results considering all practical conditions. It is an inclusive decision for the process owner.
Sometimes, a hypothesis test can find a claim to be statistically significant. However, a claim may not be worth the effort or expense to implement. Therefore, the organization should always consider practical significance along with statistical significance in a decision-making process. Analysts need to combine engineering judgment with statistical analysis.