Why do statistical based testing?

Edited by John Healy

There is a lot of probability, statistics and data analysis involved with reliability engineering. Why is that? Have you considered why our field of endeavor includes the use of these tools?

Let’s say there were no statistical tools. We would not be able to accurately infer a conclusion based on an observation of a few samples. We might react to everything that we observe – constantly spinning our wheels on minor issues. We might make decisions based on factors that did not include the random variation of the items. We might track failure rates, yet not really know how to determine if the few failures we observe were an indication of a major issue or not.

Sure, we would still have engineering judgment. If a failure was a design error and would impact every item produced, then we could sort that out. Yet, if a failure only occurred in a small subset of items, we would not have the tools to make sense of our observations.

Without statistical tools, we lose a lens into the workings of variation. We need to see the patterns in the data to gain an understanding.

Purpose of testing

Generally, we want to learn something. We want to know about the performance of the product, system or service. Basically, will the product, system or service work.

Even starting up an early prototype is a test. Our belief is that the design and assembly will work well enough to allow us to make some measurements and check our design assumptions.

We may run tests to determine if the product will operate in the expected use conditions. Will it work when connected to other devices? Will it work after the stresses of shipping and installation? Will it work long enough? These are all common reason to do testing.

Specific to reliability we often want to know if the product is robust enough to work long enough. We may want to learn what will fail and approximately when it will fail. We may want to learn how to optimize performance and minimize degradation of performance over time.

The purpose of testing is to learn so our decisions lead us to our goal.

Adding statistical thinking

If we only make one item, then variation among items is not an issue. We would craft a working product and that would be the entire population. We don’t do this very often even with very expensive products. Most of our endeavors involve making many products. Then variation does happen.

Statistical thinking allows us to make good decisions even when there is a lot of variability present. Using statistics, we know how good our inferences are. We can confidently predict future performance.

With statistical tests, we can evaluate the various uncertainties of a design or assembly process and we can estimate the variability in a set of products. Using multiple suppliers, using a range of configurations, all increase the ways a product experiences variation and all of these can be accounted for with various statistical procedures.

Statistics is the language of a decision making. Statistics provides us with the tools to make decisions that lead us to our goals and objectives. It allows us to understand variation as well as average response.

It is statistical thinking and the use of tools such as sample size determination, hypothesis testing, ANOVA and DOE that allow us to learn what we need to know even when there are ample sources of variation. We use statistical tools to make better decisions based on our testing.

The intent of testing is to reduce uncertainty. The addition of statistics allows us to peer past the variation caused clouds and see meaningful results. We need to make decisions, the best we can, and the use of statistical based testing allows us to do so.