Alpha Risk measures the risk of analyzing the results of a statistical test, such as comparison between one process method and another, and determining that any statistical differences are significant, when in fact they are not significant. Understanding the concept of the Alpha Risk – and its opposite, Beta Risk – is especially important for business leaders and project teams who make decisions based on statistical test results.
Alpha Risk measures the potential of taking a statistical finding leading to a decision (such as an F-test, a t-test, a chi-square test, etc.) and finding it significant when it is not. For example, this can occur when a team looks at data from a current operational process and the possible outcome if they changed an aspect of the process. An analysis of the two might show a difference in outcomes, such as less product defects, and the team decides that this change is the one that will provide significantly different outcomes.
Six months later, they discover there really was no difference between method A and method B. That’s a common mistake. It’s known as a Type I error.
The risk of making a Type I error is called the Alpha Risk. Typical Alpha Risks are 0.10, 0.05, and 0.01 that correspond to a 90%, 95%, and 99% level of confidence, respectively.
The main determinant of Alpha Risk is the sample size used in the statistical test or comparison. The smaller the sample size, the higher the Alpha Risk becomes. Conversely, larger sample sizes lower the Alpha Risk.
Examples of Alpha Risk
Small sample sizes plague statistical analysis in everything from sports to investing. For example, fans may overvalue a baseball hitter who goes on a hot streak, deeming the statistical difference between his stats and those of his fellow players as significant. He’s made a starter, but a month later, he’s back on the bench after his hitting numbers have fallen.
The same thing can happen to novice market investors who make stock buys based on short-term stock movement, then watch their portfolio value drop along with the stock price.
Alpha Risk also presents itself in medical research. If researchers test a new medicine on patients in a controlled study and their condition improves, they might conclude that the medicine made the difference. But in just one lab test, other variables could have caused the patient’s improvement. That’s why multiple tests are needed to avoid making a Type I mistake in medical research.
The best way to avoid Alpha Risk is to increase the size of the statistical sample. Teams then have a better chance of capturing a more representative sample of process outcomes.
