Prototypes are precious items. A vendor sample or beta version of factory parts may provide necessary insights.
They are not real though.
The prototypes and samples we hold, admire, examine and test are just a representation of the final product or system.
It is extremely rare we will learn exactly what we need to know with these few items.
The problem with prototypes
Beyond scarcity, prototypes are a faint realization of the final product.
We may be pending component selection, vetted software, and assembly decisions.
The aim of the prototype is to learn as much as we can before finishing the design and assembly process development.
Each prototype is a start of measurements and experiments in order to make decisions and changes.
We often want to know how the product will perform for customers. On a shifting platform, it is difficult to get a clear view of the future.
Each prototype is a start of measurements and experiments in order to make decisions and changes.
One factor that impacts field reliability that is missing in prototypes is the variability of components, parts, assembly practices, and use conditions.
Keep in mind that the future performance is less promising than the prototypes may reveal.
Prototypes are still useful when treated appropriately
Learning about reliability performance when given prototypes is possible.
You have to start well before the first samples arrive.
A focus on reliability, early and often in the design process, determines the areas of reliability risk.
This allows you to either learn more about that specific risk. How it will manifest and how often it may occur.
Knowing these elements then permits sorting out how long before the failures occur.
Prototypes are informative.
If and only if you ask the right questions and prepare to find those answers.
Prototypes provide surprises
When dealing with prototypes it is too easy to dismiss failures are being due to the failure occurring on a prototype.
Doing so, you miss an opportunity to learn something about your design and assembly process.
Every failure is gold.
Avoiding the work to determine the root cause of a failure with a wave of your hand assuming it was due to a prototype only issue, has led to way too many field issues.
If something unexpected occurs with a prototype, that is worth taking the time to examine and understand.
Sure, some failures may be due to the substitute material, alternate assembly process, or lack of the final version of the software.
Sometimes it is the only time you will witness a 10% field failure issue during the development process.
You don’t know if the failure is major or not till you understand what caused the failure.
Reliability estimates based on prototypes
Running a test, any test to examine reliability performance has a couple of barriers to creating an accurate reliability estimate.
- First, we’re using prototypes which may not fully represent the final product.
- Second, the interpretation of the test conditions to use conditions may have limiting assumptions.
- Third, sampling error – we get a particularly good or poor batch of prototypes.
- Fourth, we excuse failures on ‘it’s a proto’
- Fifth, it’s an extrapolation of data to the future, thus making the assumption the same pattern of reliability performance will continue
- Sixth, the stresses applied to the prototype will excite the failure mechanisms we will eventually see in the field.
And more, you get the idea. Making reliability estimates is tricky business.
Understand the risks, develop models and expectations, evaluate and experiment to find failures, understand all failures, test and measure time to failure, and finally, expect your results will be wrong.
Monitor field performance and look to be surprised.
The fewer surprises from field failures, the better you did during the development process.
The more surprises, well you may have another chance.
How well have your reliability estimates compared to actual field performance?
What has worked or not worked for you?
Leave a comment below and join the conversation.
Leave a Reply