Around about this time in a normal year (where normal is the old normal, not this new normal), we’d be packing up for World ATM Congress in Madrid. Once there we’d paint a 5m x 10m stand pretty much head to toe in pink and throw up the word “Validation” in big white letters. For those 3 days, I am in the business of selling Validation. Or at least I try, because the thing is, validation is an incredibly hard sell.
Over the last 17 years, we have “sold” our fair share of validation, perhaps more than most other consultancies. Even when we started it wasn’t exactly new and yet, all these years later it is a wonder (and also a frustration) that it hasn’t really established itself as a concrete part of every ATM project in the same way things like Safety Assessment has. A lot of the time it does still feel like a bit of a “sell” to convince people to take it seriously.
So seeing as how we aren’t going to be in Madrid this March, allow me to summarise a few of the arguments and counter-arguments about validation we would usually encounter in that big, windowless exhibition hall.
“Validation is a cost overhead.” This is often partnered with “We would have done it if we had the budget, but we don’t so we didn’t”.
OK so I can’t deny that the expertise and resource required for validation has to be paid for. But it shouldn’t be seen as an overhead. A well run validation process will often save money in the long run by helping avoid delay, making sure you don’t make expensive mistakes, and getting stakeholder buy in sooner.
“It all sounds a bit expensive.”
See above – it might cost but it should pay for itself. And considering the cost of most complex ATM systems, it is probably only a tiny percentage of the overall project cost.
“It’s the sort of thing we’d like to have as an option and see if we have time at the end.”
I’m afraid, by that point, it’ll be too late. Verification can be done near the end but validation needs to be an integral part of your project from day one to be fully effective. And no – it shouldn’t be an option. It’s my job to convince you it is a necessity.
“Validation just produces paper. There’s a library in {insert place here} where validation reports are just gathering dust. No-one ever reads them.”
Well, maybe if they’d got us to write them more people might read them! More seriously though, validation produces evidence. Subjective feedback, quantitative performance results, inputs to safety cases – validation provides all of these, not just isolated simulation reports.
“It’s fine for research projects but doesn’t really belong in real world implementation.”
You’ve no idea how many times I hear this one! Yes there is a correlation in that validation often goes hand-in-hand with concept development and early lifecycle. But it’s not the preserve of research and the impact of good validation is probably most keenly felt in implementation projects where stakeholder and user buy in is critical to success.
“It’s OK – we have a pretty good verification process.”
Good for you. I hope you managed to get all those requirements you’re going to check against written clearly enough right back when you first thought of this idea. Let me know how it goes.
“If it was that good, it would have its own Wikipedia Entry.”
Don’t get me started on this one……
“It’s all just real time simulation.”
Maybe historically, but not today. In ATM, validation certainly has its roots in real time simulation but a good validation strategy will take in RTS, FTS, workshops, desktop research, live trials and many more techniques. It’s very powerful.
“I saw some of the validation stuff in SESAR. It all looked a bit rigid and a bit theoretical”.
Yes, sorry about that! The SESAR Validation process is based on the E-OCVM which Think (in)famously helped to write. What is less well known, is that the E-OCVM was written with clear instructions to be applied flexibly and according to the needs of the programme. The sheer size and scale of SESAR means a fairly uniform and structured approach is necessary but outside of that, the process should bend and shape according to your project. Horses for courses….
“You can’t prove to me that it works.”
Yes, this one is tough. It’s a bit like raising a child. You only know the outcome of your process after the event and without having eyes into a parallel universe, we’ll never know what would have happened if we had done things differently. What we do have though are dozens (hundreds!) of well investigated instances where not applying the principles of validation has led to cost over-run, schedule delay, lack of acceptance and ultimately failure.
Those are just some of the conversations I enjoy having and there are plenty more. We are still in the era of having to convince a lot of people that validation is actually quite a useful – necessary – part of any ATM project.
30 or 40 years ago validation wasn’t quite as critical. The ATM system had plenty of capacity and systems were less complex. We could afford to get things “close enough” in the fitness-for-purpose sense. We have no such luxury these days. We can’t rely on getting lucky.
Progress is being made. Internally at Think we talk about it in a similar way to Safety Culture and maturity. In other industries, there are organisations for whom even thinking about safety is an afterthought. They need to be convinced that it is important. In ATM we have had that awareness from day one. Today in ATM there are a mix of organisations who each have different levels of “Validation Culture”. There are those who have practised validation for years and who don’t need to be sold on including it and there are others who have yet to be convinced, probably because they haven’t yet had a disaster which they now regret. As an Irishman I am naturally quite melancholy and morose but I don’t want “regret” to be my main sales tactic!
We won’t be in Madrid this March, but we will be there when World ATM Congress happens again and in the meantime, having read this far you obviously have some interest in validation so please do pick up the phone and let me convince you how useful validation is. Don’t hesitate.
Author: Conor Mullan, MD, Think Research
Recent Comments