AQL decision

Recently I received a question related to setting an Acceptable Quality Level (AQL) for a sampling of fielded electricity meters. The question was on how to select the right AQL for use with the sampling plan. I was not sure from the question if the sample would determine if the population would be replaced or not (expensive), or simply an experiement to determine how the meters are doing after 15 years of service (information only).

A little research found the original sampling plans and most standardized plans today have a few built in assumptions related to AQL:

The question I received was only about the AQL and I responded to that question as stated below, and have left out the fuller consideration of the consumer’s risk and the impact on sampling when both producer and consumer risk’s are considered.

So, in response to how to set the AQL value when using a standard sampling plan:

One of the least liked phrases uttered by statisticians is “it depends”. Unfortunately, in response to the question the selection of the AQL depends on a number of factors and considerations. If one didn’t have to sample from a population to make a decision, meaning we could perform 100% inspection accurately and economically, we wouldn’t need to set an AQL. Likewise, if we were not able to test any units from the population at all, we wouldn’t need the AQL. It’s the sampling and associated uncertainty that it provides that requires some thought in setting an AQL value.

As you will have notices the lower the AQL the more samples are required. Think of it as reflecting the size of a needle. A very large needle (size of a telephone pole) is very easy to find in a haystack. Whereas, the normal size needles is proverbially impossible to find. If you desire to determine if all the units are faulty or not (100% would fail the testing if the hypothesis is true), that would be a large needle and only one sample would be necessary. If on the other hand you wanted to find if only one unit our of the entire population is faulty, that would be a relatively small needle and 100% sampling may be required, as the testing has the possibility of finding all are good except for the very last unit tested in the population.

AQL is not the needle or in your case the proportion of faulty fielded units. It is the average quality level which is related to the proportion of bad units. The AQL is fixed by the probability of a random sample being drawn from a population with an unknown actual failure rate of the AQL (say 0.5%) creating a sample that has a sample failure rate of 0.5% or less. We set the probability of acceptance relatively high, often 95%. Which means if the population is actually mostly as good or better than our AQL we have a 95% chance of pulling a sample that will result in our accepting the batch as being good.

The probability of acceptance is built into the sampling plan. Drafting an operating characteristic curve of your sampling plan is helpful in understanding the relationship between AQL, probability of acceptance, and other sampling related values.

Now back to the comment ‘it depends’. The AQL is the statement that basically says the population is good enough – an acceptably low failure rate. For an electrical meter, the number of out of specification may be defined by contract or agreement with the utility or regulatory body. As a end customer, I would enjoy a meter that under reports my electricity use as I would pay for less than I received. The utility would not enjoy this situation as it provides their service at a discount. And, you can imagine the reverse situation and consequences. Some calculations and assumptions would permit you to determine the cost to the consumers or to the utility for various proportions of units out of specification, either over or under reporting. Balancing the cost of testing to the cost to meter errors and you can find a reasonable sampling plan.

Besides the regulatory or contract requirements for acceptable percent defective, or the balance between costs, you should also consider the legal and publicity ramifications. If you accept 0.5% as the AQL, and there are one million end customers, that is 5,000 customers with possibly faulty meters. What is the cost of bad publicity or legal action? While not likely if the total number of faulty units is small, there does exist the possibility of a very expensive consequence.

Another consideration is the measurement error of the testing of the sampled units. If the measurement is not perfect, which is an reasonable assumption in most cases, then the results of the testing may have some finite possibly to not represent the actual performance of the units. If the testing itself has repeatability and reproducibility issues than setting a lower AQL may help to provide a margin to guard from this uncertainty. A good test (accurate, repeatable, reproducible, etc.) should have less of an affect on the AQL setting.

In summary, if the decision based on the sample results is important (major expensive recall, safety or loss of account, for example) then use a relatively lower AQL. If the test result is for an information gathering purpose which is not used for any major decisions, then setting a relatively higher AQL is fine.

If my meter is in the population under consideration I am not sure if I want my meter evaluated. There are one of three outcomes. First, the meter is fine and in specification, which is to be expected and nothing changes. Second, the meter is overcharging me and is replaced with a new meter and my utility bill is reduced going forward. I may then pursue the return of past overcharging if the amount is worth the effort. Third, the meter is undercharging me, in which case I wouldn’t want the meter changed nor the back charging bill from the utility (which I doubt they would do unless they found evidence of tampering). As an engineer and good customer though, I would want to be sure my meter is accurate, of course.