Early and late stage formatives – what to do?

PUBLISHED 1st February 2018

I often get asked ‘when should we do a formative study’? Should we wait until we have got the product designed? Can we use a prototype for the study? Can we get away without doing any formatives – and just go straight to the validation stage?

All seem to be grappling with one central question – what is the purpose of formative evaluations?

Our experience over many years now tells us that formative evaluations are vital. And there is a big difference between the role of early stage formatives compared to late (or ‘pre-validation’) stage formatives.

Early stage formatives – Early stage formatives should be done BEFORE you have made the big decisions about design; while you are still at the prototype stage, and when you still have the flexibility to make big changes to the design of the user interface if needed. The purpose of the early stage formative is to explore design-based usability questions. In fact when we design the protocol for early stage formatives we design it around the key usability questions. Questions such as ‘how can we design out the known use problems with these types of devices‘. A good example would be the known problem of holding auto-injectors upside down. In an early stage formative you should explore design-based options for guiding users to hold the auto-injector correctly. Another Usability-based design question could be “how long/wide/heavy should the handle of my surgical device be so that it enables surgeons to hold it correctly and comfortably during the surgical procedure?” Again, this should be addressed early, before you have designed the handle.

Late stage formatives – Late stage formatives should be performed when you are contemplating entering design freeze; when you are certain that you have got the design right. The purpose of late stage formatives is two-fold. Firstly, it should be used as a dress rehearsal for the validation study. It gives you an opportunity to test out the product in a simulated scenario and for any unexpected use errors to surface. We all want to avoid any nasty surprises in the validation study, and late stage formatives are an excellent way to ‘kick the tyres’. The second purpose of late stage formatives is to test out your risk control measures. What do I mean? Well, in your uFMEA you will have identified the measures by which you intend to reduce use-related risk. To go back to the example of auto-injectors, you may have included colour-coded signalling to help users to identify the correct orientation. In a late stage formative you can test whether users can correctly interpret the colours to identify the end from which the needle will emerge. This gives you data with which to reflect back on the effectiveness of your risk control measure, and update your uFMEA accordingly. This will be invaluable when you come to analyse the residual use-related risk.

So, early and late stage formatives are vital; they address fundamental questions about designing safe and effective products. They must be included on your critical path; they will save you money, heartache and time. They enable you to make small decisions early rather than big decisions late.