Well, we are going COTS, so there is no point creating (or demanding) reliability specifications for it.
Really?
I was just speaking to a couple of engineers – from different organizations – who were coincidentally struggling with the reliability of Commercial Off The Shelf (COTS) systems. COTS is a funny term. Big contractors, governmental organizations and militaries use the term ‘COTS’ a lot. But they tend to be the only ones who do.
So what is ‘COTS’ … really? A system that is designed and manufactured just for you is called ‘developmental.’ A system that has been designed and manufactured for someone else is called ‘COTS.’
Why would anyone buy something designed and manufactured for someone else?
(Perceived) lower risk.
The idea is that a COTS system is tried and true. All its onboard technologies are assumed to be ‘mature.’ You, as customer #2, will not be a ‘guinea pig.’ And by this simple categorization, we can immediately wave away ‘risk’ to the extent that we become enamored with something designed and manufactured for someone else.
And then of course there are ‘COTS’ systems that have really disappointed all previous buyers. These design atrocities or manufacturing experiments – by virtue of having previously been bought by someone else – can now be anointed as ‘COTS.’
I remember back in my military days when the army I was in decided to purchase some ‘COTS’ armored vehicles. They were ‘tried and true’ and designed for the ‘harshest of environments.’ There was obviously no need to test them at all. So testing was only done to be able to say that testing was done. But it turned out they were totally inappropriate for our training areas and deployments and kept over-heating, breaking their suspension and otherwise kept falling apart in our unique terrain (everyone’s terrain is unique by the way.) So they had to be hastily, expensively, and haphazardly modified. It was almost as if they had been designed and manufactured for someone else.
Risk was actually multiplied by applying the term ‘COTS.’
The only true COTS items are those physically sitting on shelves in places like Wal-Mart, Target, or Coles. There are no comparably vast big box stores of MRI machines, main battle tanks or liquid fuel rocket engines waiting for you to put into a shopping trolly. They are often going to be made and manufactured for you after you ask for them. Sure, they might be based on an extant design, but the number of tweaks, amendments, changes, and better manufacturing approaches that can be incorporated is quite large.
Even the most specialized or expensive machines in the world have options. There are always different suppliers, different models and different variants of existing designs – all of which can have those tweaks, amendments and changes applied to them.
This is a given for many other organizations who routinely create reliable systems, comprising lots of different components from suppliers. And it starts with them demanding reliability.
This brings it back to my two engineering colleagues who were struggling with the idea of ‘COTS.’ They had admitted defeat before even knowing there was a game to play. They were both working in ‘prime contracting’ organizations, delivering a substantial and complex system to governmental customers. Contracts with reliability performance clauses had been signed. But when it came time for the prime contractor to become the customer and procure all those really important sub-systems, there was no intention of including reliability clauses in the supplier documentation pack. Why? … because they were caught up in the illusion of COTS. They thought all their options were ‘done’ and saw the whole idea of coming up with specifications as futile.
You must ALWAYS ask for reliability. You can ALWAYS demand data and documentation to validate claims. You can ALWAYS create your own test regimes to identify preferred suppliers. You can ALWAYS expect contractors to incorporate characteristics and other things specific to you when they make a so-called ‘COTS’ thing.
You might be surprised by what some prospective contractors and suppliers are willing to do to make you happy. These are the contractors and suppliers you want to do business with. But you can never find the ones you want if they don’t get a chance to show how eager they might be to meet your demands.
You don’t get what you don’t ask for.
And you can’t ask for anything if you don’t know what you want.
Do you have any similar stories or experiences? Share them with all of us!
[display_form id=396]
This is a very weird comment. To class a whole armoured vehicle as COTS is a first.
Also your definition of COTS is wrong.
COTS refers to
1. components and parts which did not consider your Requirements & Specifications in its design and manufacture.
2. The onus is on you to demonstrate its suitability.
3. The manufacturers are absolved from liability if you use it outside its performance window.
4. The manufacturer may refuse to be drawn into a supply contract and rather use their own business model.
5. The component is made for possible use by many customers
There is a cut of point. It can’t be a whole system.
It can be a subsystem if it fits your application. It’s integration into yours application is for you to do and you may be legally prevented from unduly criticising it, unless it is dangerous to life and or health, in the short or long term.
Generally COTS give promise only the bare minimum Reliability in a most benign use condition.
COTS are generally components, parts, subsystem (vehicle climate control subsystem).
Picking your car , a whole system, from the forecourt doesn’t make it a COTS in Reliability because you can’t do anything about integrating it into a bigger system that is yours. Despite the fact that it is made for multiple clients and their, each singular requirements and specifications were not considered
For it to be COTS it must be commercially available, not made for you, must be integrated into your system.
Integrating to the environment (Vehicle to Environment) is not part of COTS. Thsts a Use Case Analysis / Evaluation.
Thank you for your comment … it is always good to get a dialogue going.
I can’t disagree with you from your perspective of COTS having a ‘cut off’ point that lies somewhere between ‘subsystem & components’ and the ‘entire system.’ If this is your experience and how you and those around you define COTS to be, then that is your reality.
However, I have seen COTS been used (rightly or wrongly) to describe entire systems. And that includes cars and other vehicles that have been colloquially referred to as ‘COTS options.’ Now you might argue that this is incorrect – and you may be able to win that argument.
But the point of my article is that there is a wider set of experience bases where what is or isn’t COTS might deviate from your experience. And it is often the case that this categorization is used to avoid any further discussions about reliability, and how we can get that from the manufacturer.
Thanks for taking the time to add the conversation. It is always better when we have a dialogue going where readers can learn about different takes on the same subject.