Accendo Reliability

Your Reliability Engineering Professional Development Site

  • Home
  • About
    • Contributors
    • About Us
    • Colophon
    • Survey
  • Reliability.fm
  • Articles
    • CRE Preparation Notes
    • NoMTBF
    • on Leadership & Career
      • Advanced Engineering Culture
      • ASQR&R
      • Engineering Leadership
      • Managing in the 2000s
      • Product Development and Process Improvement
    • on Maintenance Reliability
      • Aasan Asset Management
      • AI & Predictive Maintenance
      • Asset Management in the Mining Industry
      • CMMS and Maintenance Management
      • CMMS and Reliability
      • Conscious Asset
      • EAM & CMMS
      • Everyday RCM
      • History of Maintenance Management
      • Life Cycle Asset Management
      • Maintenance and Reliability
      • Maintenance Management
      • Plant Maintenance
      • Process Plant Reliability Engineering
      • RCM Blitz®
      • ReliabilityXperience
      • Rob’s Reliability Project
      • The Intelligent Transformer Blog
      • The People Side of Maintenance
      • The Reliability Mindset
    • on Product Reliability
      • Accelerated Reliability
      • Achieving the Benefits of Reliability
      • Apex Ridge
      • Field Reliability Data Analysis
      • Metals Engineering and Product Reliability
      • Musings on Reliability and Maintenance Topics
      • Product Validation
      • Reliability by Design
      • Reliability Competence
      • Reliability Engineering Insights
      • Reliability in Emerging Technology
      • Reliability Knowledge
    • on Risk & Safety
      • CERM® Risk Insights
      • Equipment Risk and Reliability in Downhole Applications
      • Operational Risk Process Safety
    • on Systems Thinking
      • Communicating with FINESSE
      • The RCA
    • on Tools & Techniques
      • Big Data & Analytics
      • Experimental Design for NPD
      • Innovative Thinking in Reliability and Durability
      • Inside and Beyond HALT
      • Inside FMEA
      • Institute of Quality & Reliability
      • Integral Concepts
      • Learning from Failures
      • Progress in Field Reliability?
      • R for Engineering
      • Reliability Engineering Using Python
      • Reliability Reflections
      • Statistical Methods for Failure-Time Data
      • Testing 1 2 3
      • The Manufacturing Academy
  • eBooks
  • Resources
    • Accendo Authors
    • FMEA Resources
    • Glossary
    • Feed Forward Publications
    • Openings
    • Books
    • Webinar Sources
    • Podcasts
  • Courses
    • Your Courses
    • Live Courses
      • Introduction to Reliability Engineering & Accelerated Testings Course Landing Page
      • Advanced Accelerated Testing Course Landing Page
    • Integral Concepts Courses
      • Reliability Analysis Methods Course Landing Page
      • Applied Reliability Analysis Course Landing Page
      • Statistics, Hypothesis Testing, & Regression Modeling Course Landing Page
      • Measurement System Assessment Course Landing Page
      • SPC & Process Capability Course Landing Page
      • Design of Experiments Course Landing Page
    • The Manufacturing Academy Courses
      • An Introduction to Reliability Engineering
      • Reliability Engineering Statistics
      • An Introduction to Quality Engineering
      • Quality Engineering Statistics
      • FMEA in Practice
      • Process Capability Analysis course
      • Root Cause Analysis and the 8D Corrective Action Process course
      • Return on Investment online course
    • Industrial Metallurgist Courses
    • FMEA courses Powered by The Luminous Group
    • Foundations of RCM online course
    • Reliability Engineering for Heavy Industry
    • How to be an Online Student
    • Quondam Courses
  • Calendar
    • Call for Papers Listing
    • Upcoming Webinars
    • Webinar Calendar
  • Login
    • Member Home
  • Barringer Process Reliability Introduction Course Landing Page
  • Upcoming Live Events
You are here: Home / Articles / Credible Reliability Test Planning

by Larry George Leave a Comment

Credible Reliability Test Planning

Credible Reliability Test Planning

How to allocate subsystems’ MTBF requirements with testing? Name-withheld-to-protect-the- guilty proposed “Top-Down” reduction in subsystem MTBF requirements; the more subsystems (in series) that you test, the lower the subsystem required MTBF! “The correct formula is

1/MTBF(subsystem requirement) = 1/MTBF(system requirement) –
((# of subsystems in series – # of subsystems tested)/MTBF(subsystem).”

This “Top-Down…” method is uncited and not found in Internet search. 

Azmat Siddiqi asked me if I could explain the Top-Down formula. It shows how subassembly tests reduce their MTBF requirements. The formula leads to the conclusion that, If you test enough subassemblies, then the subassembly MTBF requirement is less than a series system MTBF requirement! 

Table 1 implements the formula to show how top-down chamber MTBF requirement for a wafer processing machine could be less than the system MTBF requirement, depending on the number of chambers tested. 

Table 0. Alternative test plans: test 1 or 3 chambers in each phase. The two-chamber (2 Ch) series configuration has chamber MTBF less than system goal MTBF!

PhaseGoal MTBF#Test Ch2 Ch Config3 Ch Config 
Ph3 MTBF50186113
Ph4  MTBF1501240300
Ph5 MTBF50018571125
Ph3 MTBF5033550
Ph4  MTBF1503109150
Ph5 MTBF5003353500

If you think that subsystem MTBF requirements must be greater than system MTBF requirement, you’re right, except for a subsystem whose failure increases the reliability of the surviving system. Picky statisticians use “s-coherence” to describe a system that does NOT include subsystems whose failure increases the reliability of the surviving system. [Collet] 

Are there legitimate ways to allocate subsystem MTBFs? Yes, but they don’t account for subsystem reliability tests or field reliability data. 

  • Standard formula for simple MTBF allocation: “3 Ways to do Reliability Allocation #5”, https://fred-schenkelberg-project.prev01.rmkr.net/3-ways-reliability-allocation-5/
  • “Confidence Intervals for MTBF- Accendo Reliability,” by Fred Schenkelberg, edited by John Healy, formerly of BellCore TR-332 and former head of FCC
  • Haire, Maltese, and Sohmer propose a “Top-Down” system availability “apportionment” method
  • Allocation of parallel-subsystem parts in a series system of parallel subsystems subject to a cost constraint [Coit and Smith, George 2004]. Inputs are parts’ reliabilities, their costs per part, and the total cost constraint   

Credible Reliability Test Planning?   

The objective is credible reliability predictions, by testing AND by using: field reliability of comparable parts or products, proportional hazards system modeling, new product design, bills-of-materials, and Bayes statistics. In insurance, credibility “is an actuarial term describing the degree of accuracy in forecasting future events based on statistical reporting of past events.” [https://www.irmi.com/term/insurance-definitions/credibility]

“Reliability is the probability of successful function [according to customers] under specified conditions [field conditions, not test in some lab], to specified ages” [warranty? Useful life? PM? Replacement age?, etc.] Patrick D. T. O’Connor [author’s comments in brackets] Testing is a reliability activity, especially for new products. Why not use observed field reliability of comparable older products and their service parts to help plan new-product tests?

Reliability is not MTBF! Reliability is an age-specific probability function. If you want to test a reliability specification for a single age such as mission time, Bayes’ RnnCmm tests (Reliability is at least nn% with confidence mm%) randomize a single parameter: exponential MTBF, Weibull characteristic life, Gamma distributed constant failure rate, or binomial failure probability [NIST]. Credible Reliability Test Planning uses a scaled “Credible Reliability Prediction” function as a nonparametric Bayes’ “prior” failure rate function, not just a single parameter. It produces nonparametric “posterior” reliability function estimates, depending on  test plan, to help evaluate test alternatives. 

Figure 1. Outline of Credible (Bayes) Reliability Test Planning information flow, with feedback. 

Should you do Nonparametric Reliability tests?

Mathias Jesussek, DATAtab, wrote on LInkedIn, “If you are sure about the distribution assumptions…” (for a parametric reliability test) If not sure:

1. Test the parametric distribution assumption… 

Ho: Sample 1 data don’t fit assumed parametric distribution

Ha: Sample 1 data fit do fit…

2. Test reliability, presumably with an independent sample!

HoHo: Sample 2 null hypothesis about reliability is true

HaHa: Sample 2 null hypothesis is rejected 😉

I.e., you should test the distribution assumption AND do the reliability tests! Nonparametric reliability estimation make no unwarranted assumptions, so the estimates don’t require the you to test distribution assumptions. Walter Shewhart said, “Original data should be presented in a way that will preserve the evidence in the original data for all the predictions assessed to be useful.” [Shewhart rule #1] Nonparametric estimates preserve information in original data.

Generally accepted accounting principles require statistically sufficient data to make nonparametric, population reliability estimates of all products and their service parts, https://fred-schenkelberg-project.prev01.rmkr.net/want-field-reliability-without-life-data/#more-408510/. Nonparametric field reliability estimates are inputs to the Bayes prior distribution of a new product. 

Bayes Credible Reliability Test

Jim Bagley, former Applied Materials President, said in 1993, “Our job is to amaze and delight customers with our products and services.” Demonstrated test reliability leads to word-of-mouth benefits and competitive advantage. Demonstrated test reliability helps increase availability by: reducing spares’ back-orders, opportunistic maintenance, correlation of TBF and TTR, service planning, condition monitoring, optimal maintenance and replacement, diagnostics, etc.

Reliability test planning depends on reliability, and reliability is determined in the Field! “Those who cannot learn from history are doomed to repeat it.” George Santayana. So use field reliability or failure rate function estimates to construct a nonparametric Bayes prior for credible reliability test planning. Plan the tests to maximize bang-per-reliability-related buck subject to budget limitations. Plan and allocate costs such as testing, proposed sample sizes and allowable failures using a workbook and VBA: WeitestD.xlsm. Minimize total costs of units, test time, consumer and producer risk. [Bajeel and Kumar, Coit and Smith, Ferryanto, Gerokostopoulos et al., Kleyner et al. ] 

Estimates have uncertainty, which could be described by prior probability distributions. Alternative Bayes prior distributions for failure probability P[Fail], MTBF, or reliability or failure rate functions could be: 

  • P[Fail}: Assume beta-binomial conjugate prior on P[Fail] and binomial test data: beta posterior distribution of P[Fail] e.g., solve P[Life > MTBF|test]=63% for LCL on MTBF [Gerokostopoulos, Guo, and Pohl (2015)]
  • MTBF: Gamma prior (of constant failure rate) and constant failure rate in test data; posterior distribution of MTBF (for LCL on MTBF), 8.4.6. How do you estimate reliability using the Bayesian gamma prior model? (NIST.gov) [Coppolo (1981), Kensler, Xu and Chen]
  • Assumed reliability function: WeiBayes [ReliaWIki], Jeffreys prior, or Dirichlet prior for discrete pdf or percentiles. Multivariate normal prior for mean and inverse Wishart for covariance matrix of normally distributed pdf, percentiles, or actuarial failure rates
  • Semi-parametric: Linear failure rate function a(t) = a+b*t (two parameters). Sen, Kanan, and Kundu use a linear hazard rate with gamma priors on linear parameters: slope and intercept. Hameed and Alwan use approximately linear hazard rate and Rayleigh-Logarithmic distribution. (Don’t assume Weibull reliability with a linear failure rate, because the only Weibull with linear failure rate is constant failure rate!) 
  • Credible nonparametric: Dirichlet or multivariate normal prior for discrete reliability or failure rate function estimates. Use credible, new-product, reliability predictions made from scaled nonparametric field failure rate function estimates, using comparable other product or part field failure rate function estimates multiplied by the ratio MTBF(old)/MTBF(new). I.e., a(t}new) = a(t|old)*MTBF(old)/MTBF(new). [George, “Credible Reliability Prediction”]

What is Credible Reliability Prediction?

Generations of products and their parts have similar reliabilities, because they are designed similarly, use same or similar parts, are shipped through same channels, have operators trained same ways, are operated in similar environments, and are maintained in same ways. [Kaaniche et al., Dogan, Jais et al.] It is reasonable to expect generations of products to have proportional failure rate functions. This is known in biostatistics as a proportional hazards model [George, Dogan]. The proportionality factor may be estimated by MTBF(old)/MTBF(new), because, at the design stage, all that is known about a new product is an MTBF(new) prediction, presumably predicted in the same way as MTBF(old) for older products and service parts. Why not make a Bayes prior the same way you make an MTBF prediction, given a reliability block diagram or prospective bill of materials?

Estimate actuarial (age specific) failure rates for a new product’s parts based on comparable parts. Use field actuarial failure rate estimates of comparable older parts from company field data (required by GAAP) or vendor population data. Use scaled field actuarial failure rates, new product BoM, and its reliability block diagram (RBD) to compute credible reliability prediction for new product, and use it as the Bayes prior distribution!  Figure 2 shows nonparametric field failure rates for modules of older wafer processing chambers. The new product consists of comparable parts in series. 

pastedGraphic.png

Figure 2. Field reliability functions for an existing wafer processing chamber components

Bayes prior field reliability leads to Credible Reliability Test plans, because it uses (credible) observed reliability of older, comparable products and service parts! Subsystem reliability test plan depends on system configuration, failure modes,  and dependence! Test plan depends on costs: consumer risk, producer risk, and FUD.

Bayes Reliability Critics say…

“In some applications solid prior information, based on a combination of physics of failure and previous empirical experience, is available. This is particularly true in some engineering applications such as reliability, for which there are known, well-understood failure mechanisms. Engineers working in certain areas of industry will have previous experience with particular failure mechanisms and test and product operating environments that will allow them, in some situations, to provide strong prior information about aspects of the failure-time model.” [Li and Meeker]

“The Bayesian approach is somewhat controversial in reliability engineering, particularly as it can provide a justification for less reliability testing. For example, Kleyner et al. (1997) proposed a method to reduce sample size required for a success run test in order to demonstrate target reliability with a specified confidence [mixture of beta and uniform prior]. Choosing a prior distribution based on subjective judgement, expert opinion, or other test or field experience can also be contentious. Combining subassembly test results in this way also ignores the possibility of interface problems.”

If you want more pros and cons, see section 8.1.10 of the NIST “Engineering Statistics Handbook.” I admit there could be problems with prior data, so do the best you can with what you have and compute the marginal costs and rewards for more, better data! And don’t forget to deal with dependence.

Bayes Law, P[A|B] = P[B|A]*P[A]/P[B]

Here is how to incorporate field reliability using Bayes Law. The “posterior” distribution of reliability conditional on test data, P[Reliability|Test data], is L[Test data|Reliability]*P[Reliability]/P[Test data], where P[.] are probability density functions (pdf), and L[.] = likelihood function. 

P[Reliability] is pdf of observed, FIELD reliability. [See “Credible Reliability Prediction” (CRP) and “User Manual for Credible Reliability Prediction”]. The distribution of observed field reliability could be the assumed distributions listed above for alternative Bayes priors, sample distribution from components’ lifetime data, or from the asymptotic multivariate normal distribution of field reliability maximum likelihood or least squares estimates from ships and returns counts. P[Test data] (denominator) is ∫L[Test data|Field R(t)]*P[Field R(t)]dt; where the integral is from 0 to warranty or useful life.

Field Reliability function R(t) estimates differ depending on whether you have sample life data (Weibull? Kaplan-Meier? or ???) or population ships and returns counts [George])

Simple Example Compares Bayes vs. Credible Bayes MTBF Test Results

NIST “Engineering Statistics Handbook”  uses Bayes to update a constant failure rate failure rate (or MTBF): assuming a Gamma(a, b) prior distribution of failure rate and exponential test time to failure (or Poisson number of failures) [NIST-SEMATECH].  For a credible prior, fit a Gamma(a,b) distribution to PVD chamber field failure rate estimates, –d ln(R(t))/dt, where R(t) are observed chamber reliability estimates at different ages t. 

Use the observed PVD chamber field failure rate Gamma distribution to compute a new test chamber failure rate posterior distribution for some test alternatives. For example, consider testing some new PVD chambers to failure or 1000 hours. Table 1 gives some potential test times and Gamma posterior MTBF parameters. The Gamma posterior distribution is Gamma(a+r,b+SUM[t(i),i=1,2,…,N) r = number of failures, and if N = 1, posterior is Gamma(a+r,1000). This is also in “Practical Reliability Engineering,” [O’Connor and Kleyner]

Table 1. NIST simple method: Gamma prior and constant failure rate. Numerical entries are some example times to failure of chambers (units) https://www.itl.nist.gov/div898/handbook/apr/section2/apr25.htm.

Units on testTest TimeTest TimeTest TimeTest Time
1 Unit1000


2 Units7561000

3 Units5927561000
4 Units2955927561000

Table 1 continued. Gamma shape parameter k = number of field failures = 883, gamma scale parameter theta = total time on test (field) = 17442 hours (PVD chamber field MTBF =19.75 months)

Units on testk’theta’ hoursMTBF median months80% LCL monthsLCL on MTBF hours
1 Unit8831744319.7619.2114030
2 Units8841744419.7319.1914015
3 Units8851744519.7119.1714000
4 Units8861744619.6919.1413985

The prior gives ridiculous MTBFs, because it was not adjusted for differences from field PVD chambers to test chamber. So, credibly adjust Gamma prior shape parameter theta. Gamma mean is k*theta, so adjust theta using ratio of test chamber MTBF prediction and field PVD chamber MTBF. Example assumes test chamber MTBF is 1000 hours.  Table 2 show result, using same times on test as in table 1. Reality substantially changes MTBF median and its LCL. 

Table 2. Same as table 1 except for rescaling the mean by the ratio of test chamber MTBF and field PVD chamber MTBF. Gamma mean is k*theta, so theta is rescaled. 

Units on testk’Theta’ monthsMTBF median months80% MTBF LCL monthsLCL Hours
1 Unit88312101.371.33973
2 Units88412111.371.33973
3 Units88512121.371.33973
4 Units88612131.371.33972

“Balance” Consumer and Producer “Risks” [Kensler]

An alternative test plan that balances both confidence and power or consumer and producer risk. MTBFo=goal, d= excess MTBF you want to detect, a = P[Type I error] consumer risk (Sell a product with less-than specified MTBF), and b = P[Type II error] producer risk (Don’t sell product because MTBF is too low.). I describe this method to show how consumer and producer risk may be incorporated in credible reliability test planning.

The chi-square statistical test method assumes constant failure rate, chi-square distribution for probabilities, and “Risk” = probability of error times cost of error. [See also MIL-HDBK-781A]

MTBFA is the MTBF required to have a high probability of passing the demonstration. In other words, MTBFA= MTBFo + δ, where δ > 0 is the difference we want to be able to detect. A test plan with a 100(1 − 𝛼)% confidence level and power of at least 1 − β has 𝑟 allowable failures where 𝑟 is the smallest nonnegative integer such that

χ2(β, 2r+1)/χ2(1−α, 2r+1) ≥ MTBFo/MTBFA.

Table 3. Hypothesis test description of type I and II errors and consumer and producer risks 



DecisionDecision


Ho is trueHa is true
TruthHo is truepastedGraphic_1.pngconf. = 1-αType I error: Consumer risk = α
TruthHa is trueType II error Producer risk = βpastedGraphic_1.pngPower = 1-β

Table 4. Example of test plan to balance producer and consumer risk: MTBFo=1000, δ=200, α=5%, β=10%. Pick the minimum total test time as a function of failures that has MTBF2 greater than MTBFo. Total test time is MTBFo*χ2(1−α, 2r+1)/2.

FailuresMTBFo/ MTBFARatio of
Chi-Sqs 
>MTBFA/ MTBFo?Total Test time
00.8330.769FALSEFALSE
10.8330.820FALSEFALSE
20.8330.845TRUE11070
30.8330.862TRUEFALSE
40.8330.873TRUEFALSE
50.8330.882TRUEFALSE
60.8330.889TRUEFALSE

Credible Reliability Test Implementation

This section describes nonparametric, credible reliability test using Bayes law to incorporate field reliability and test results. The prior distribution f(θ) of field reliability: θ could be vector of a discrete reliability function R(t) or its percentiles. Maximum likelihood and least squares are the two methods I use to estimate field reliability and failure rate functions for parts and products comparable with the new product to be tested. 

The Kaplan-Meier (grouped time-to-failure counts) or ships and returns [George] maximum likelihood estimators of field reliability are asymptotically unbiased with Cramer-Rao lower bound on variance-covariance, and asymptotically multivariate normally distributed. The least-squares estimator from ships and returns is asymptotically multivariate normally distributed due to martingale central limit theorem [Borgan and Langholz, Muller and Quintana] 

For the new product to be tested, scale the failure rate function corresponding to θ using “Credible Reliability Prediction,” a(θ; new) = a(θ;old)*MTBF(new)/MTBF(old) (proportional hazards) a(θ;.) is actuarial failure rate (estimate). Functions of maximum likelihood estimators are also maximum likelihood estimators.  

Compute L(Test Data|θ) for ∫L(Test Data|θ)*f(θ)dθ. Apply Bayes formula to compute posterior reliability pdf [Sun and Berger]: f(θ|Test Data)  = L(Test Data|θ)*f(θ) / ∫L(Test Data|θ)*f(θ)dθ where L{.|θ) is conditional likelihood of test data. Compute or simulate the distribution of MTBF corresponding to posterior reliability pdf =Σθ(t)*t. Pick off LCL on MTBF with desired confidence. Proceed to optimization step…

Optimize “Credible Reliability Test” plan

The optimal reliability test plan should minimize costs, subject to physical, time, political, and budget constraints. Cost of test chambers + number of test chambers*(cost of test times per chamber) + Consumer risk (expected cost of unavailability + chip shortages + ??? under posterior reliability if test passes with Type I error + Producer risk = expected service cost, warranty extension, recalls, word-of-mouth, etc. under posterior reliability + if test fails with Type II error.

Cost of FUD! (Fear, Uncertainty, and Doubt!), proportional to variance of posterior MTBF or reliability. Need costs. When I asked a professor where I could get cost data, he replied, “Ask your accounting Department.” FDL-RITA (Falling Down Laughing-Rolling In The Aisles)

Why not plan credible reliability tests to maximize bang-per-reliability-related buck subject to budget constraint? Planning and cost allocation to: RnnCmm, costs, proposed sample sizes and allowable failures. WeitestD.xlsx workbook assumes a Weibull TTF distribution. Input costs of units, test time, consumer and producer risk. The workbook computes costs for test units, times, and allowable failures for optimization. Feel free to use your own credible posterior distribution in the workbook.

I made workbook WeitestD.xlsx for RnnCmm reliability test planning. It includes Weibull time to failure posterior distribution and discounting. The Cmm Sample spreadsheet includes test time corresponding to a Weibull confidence limit on reliability for specified parameters, confidence level, and sample size. Taken from www.weibull.com/LifeDataWeb/test_design.htm (Weibull reliability test formula).

RnnCmmWeibull spreadsheet:  Sample size corresponding to a RnnCmm reliability test with specified error and cost parameters. Enter Rnn, Cmm, costs and some proposed sample sizes and allowable numbers of failures. Binomial probability of test failure under Ho: reliability ≥ Rnn% and power of test. Weibull time on test and test cost corresponds to the sample sizes.

Table 5. Typical RnnCmm reliability test plan Input with Weibull reliability function as posterior distribution: WeiTestD.xlsx!RnnCmmWeibull

RangeValueDescription
Rel90%Reliability specified that you want to demonstrate
Conf90%Confidence limit, close to 100%, 1–consumer risk
Eta1732Weibull scale parameter. estimate 
Beta2Weibull shape parameter estimate
AF15Acceleration factor for Eta;
hope beta doesn’t change due to acceleration.

Table 5 continued

RangeValueDescription
Delta1.00%Excess reliability, Rel = nn%+Delta%, you’d like to detect,
for test power computation, 1–producer risk
CostI$100,000 Cost of type I error, accepting a bad product
with reliability less than specified
CostII$100,000 Cost of type II error, rejecting a good product
with reliability equal to specified reliability + Delta
Fixed$10,000 Fixed cost test, setup, fixtures, and so on
Prod Cost$100 Cost per unit tested, including cost of product
and product-specific test fixture
Uncert$10,000 Cost per percent of standard deviation of
reliability estimate nn% in RnnCmm test plan 
TimeCost$10 Cost per unit tested of time on test,
used in to compute test cost with Weibull reliability function 
`$10Cost per unit time of eta uncertainty, stdev

Table 6. RnnCmm test plan output. Pick the plan with least total cost in bottom row. Least-cost sample plan looks like test 38 and accept if no more than one fails. 

Failures01234<-Allowable test failures, k
Sample size2238526578<-Sample to show Rel=nn% with Conf~mm% and ≤ k failures
Power94.1%98.7%99.7%99.9%99.97%<-Power = 1-P[Type II error], accept Ho: P[Rel < nn%|Rel = nn%+delta
Stdev Rel6.40%4.87%4.16%3.72%3.40%<-Unconditional stddev of reliability estimate from sample size above
Total costs$118k$115k $115k $116k $117k <-Unconditional E[costs of errors, tests, and uncertainty]

Take-Aways

Want a Credible Reliability Test plan? Use field reliability data, not just tests, not just armchair exercises such as FMECA. If a medical company required FDA approval, it would do Bayes’ testing [Muller and Quintana] https://www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials. See also FDA 21 CFR 820.50 “Sampling plans shall be written and based on a valid statistical rationale.” [Ferryanto]

FMECA Risk Priority Numbers (RPNs) hide the fact that risk is (statistical) expected cost, perhaps discounted. Reliability test planning depends on politics, costs, (statistical) risks, and uncertainty. Trade uncertainty, costs, and risks to equate bang-per-buck. Balance consumer and producer risk. Costs include: units on test, test time, test facilities and manpower, numbers of failures, probabilities, consumer and producer risk, and uncertainty. WeiTestD.xlsx is one way of using a posterior distribution. Change Weibull to the posterior distribution of new product life incorporating test and field reliability. 

Ask pstlarry@yahoo.com if you want Bayes posterior distribution for linear, Dirichlet, multivariate normal, or other prior distributions and tests. Ask for WeiTstD.xlsx workbook.

References

Ørnulf Borgan and Bryan Langholz, “Using Martingale Residuals to Assess Goodness-of-Fit for Sampled Risk Set Data”, Chapter 4 “Advances in statistical modeling and inference. Essays in honor of Kjell A Doksum” World Scientific, Singapore, 2007

Coit, David W. and Alice E. Smith. “Stochastic Formulations of the Redundancy Allocation Problem,” Proceedings of the 1996 5th Industrial Engineering Research Conference – Minneapolis, MN, USA 

Jerome Collet, “Some Remarks on Rare-Event Approximation,”   IEEE Transactions on Reliability, DOI: 10.1109/24.488924, April 1996 

Gokhan Dogan, “Carryover Parts and New Product Reliability,” August 2010, MIT PhD thesis, Carryover parts and new product reliability (mit.edu)

Liem Ferryanto, “Statistical Sampling Plan for Design Verification and  Validation of Medical Devices,” Journal of Validation Technology, May 2015

Athanasios Gerokostopoulos, Huairui Guo, Ph. D. & Edward Pohl, Ph. D., “Determining the Right Sample Size for Your Test: Theory and Application,” RAMS 2015

L. L. George, “Credible Reliability Prediction,” ASQ RD monograph (CRP), 2nd ed., https://sites.google.com/site/fieldreliability/credible-reliability-prediction/ 2019

L. L. George, “User Manual for Credible Reliability Prediction,” https://sites.google.com/site/fieldreliability/user-manual-for-credible-reliability-prediction/ 2019

M. J. Haire, J. G. Maltese, and R. G. Sohmer, “A System Availability “Top-Down” Apportionment Method,” RAMS, 1985

Christopher Jais, Benjamin Werner, and Diganta Das. “Reliability Predictions – Continued Reliance on a Misleading Approach”, RAMS 2013, Orlando, Florida, 28-31

Mohamed Kaaniche, Karama Kanoun, Michel Cukier, and M. B. Martini, “Software reliability analysis of three successive generations of a Switching System”, January 2006, DOI:  10.1007/3-540-58426-9_153, EDCC 1994: Dependable Computing — EDCC-1 pp. 471-490

Jennifer Kensler, “Reliability Test Planning for Mean Time Between Failures Best Practice”, STAT T&E COE-Report-09-2013, www.AFIT.edu/STAT/

MIL-HDBK-781A, ‟Handbook for Reliability Test Methods, Plans, and Environments for Engineering, Development Qualification, and Production,” 1996

NIST, “Engineering Statistics Handbook,” Chapter 8, “Assessing Product Reliability”  https://www.itl.nist.gov/div898/handbook/apr/section1/apr1a.htm/

References to Bayes Life Testing

P. N. Bajeel and M. Kumar, “Design of Optimal Bayesian Reliability Test Plans for a Series System,” Int’l J. of Pure and Applied Mathematics, Vol. 109 # 9 2016, pp. 125–133

Anthony Coppolo, “Bayesian Reliability Tests Made Practical,” RADC-TR-81-106, 1981

Alaa Faris Hameed and Iqbal Mahmood Alwan, “Bayes estimators for reliability and hazard function of RayleighLogarithmic (RL) distribution with application,” Periodicals of Engineering and Natural Sciences, Vol. 8, No. 4, October 2020, pp.1991-1998 

Tongdan Jin, Reliability Engineering and Services, Wiley, Nov. 2018

Ming Li and William Q. Meeker, “Application of Bayesian Methods in Reliability Data Analyses,” (2013). Statistics Preprints. 84. http://lib.dr.iastate.edu/stat_las_preprints/84/ 

Jason R. W. Merrick, Refik Soyer, and Thomas A. Mazzuchi, “A Bayesian Semiparametric Analysis of the Reliability and Maintenance of Machine Tools.” Technometrics, February 2003, VOL. 45, NO. 1 DOI 10.1198/004017002188618707 

Ananda Sen, Nandini Kanan, and Debasis Kundu, “Bayesian planning and inference of a progressively censored sample from linear hazard rate distribution,” prog-lhr_CSDA_revised.pdf (iitk.ac.in)

Patrick D. T. O’Connor and Andre Kleyner, Practical Reliability Engineering, 4th ed., Wiley, 2012

Peter Muller and Fernando A. Quintana, “Nonparametric Bayesian Data Analysis,” MQ02.pdf (utexas.edu) 

Andre Kleyner, Shrikar Bhagath, Mauro Gasparini, Jeffrey Robinson and Mark Bender, “Bayesian Techniques to Reduce the Sample Size in Automotive Electronics Attribute Testing,” Microelectronics Reliability, Vol. 37. No.6, pp. 879-883,1997

Dongchu Sun and James O. Berger, “Objective Bayesian Analysis for the Multivariate Normal Model,” Proc. ISBA 8th World Meeting on Bayesian Statistics, June 2006

ReliaWiki, “Bayesian-Weibull Analysis”, http://reliawiki.com/index.php/Bayesian-Weibull_Analysis/

Tian-Qun Xu and Yue-Peng Chen, “Two-sided M-Bayesian credible limits of reliability parameters in the case of zero-failure data for exponential distribution,” Applied Mathematical Modeling 38 (2014)2586-2600

Filed Under: Articles, on Tools & Techniques, Progress in Field Reliability?

About Larry George

UCLA engineer and MBA, UC Berkeley Ph.D. in Industrial Engineering and Operations Research with minor in statistics. I taught for 11+ years, worked for Lawrence Livermore Lab for 11 years, and have worked in the real world solving problems ever since for anyone who asks. Employed by or contracted to Apple Computer, Applied Materials, Abbott Diagnostics, EPRI, Triad Systems (now http://www.epicor.com), and many others. Now working on actuarial forecasting, survival analysis, transient Markov, epidemiology, and their applications: epidemics, randomized clinical trials, availability, risk-based inspection, Statistical Reliability Control, and DoE for risk equity.

« Understanding the Binomial Distribution
How to Calculate Replacement Asset Value for Assets and Equipment Purchased in Foreign Currency »

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Articles by Larry George
in the Progress in Field Reliability? article series

Join Accendo

Receive information and updates about articles and many other resources offered by Accendo Reliability by becoming a member.

It’s free and only takes a minute.

Join Today

Recent Articles

  • Gremlins today
  • The Power of Vision in Leadership and Organizational Success
  • 3 Types of MTBF Stories
  • ALT: An in Depth Description
  • Project Email Economics

© 2025 FMS Reliability · Privacy Policy · Terms of Service · Cookies Policy