Harm in relation to Use error and device error, as relevant to Usability testing, and how to effectively assess it

PUBLISHED 21st August 2017

ISO 14971:2007 ‘deals with processes for managing risks’ and the concept of risk is outlined within the standard as;

  1. The probability of occurrence of harm;
  2. The consequences of that harm, that is, how severe it might be.

This concept is logical and systematic; it can be agreed that determining the benefit/risk balance associated with a medical device should rely on the severity of harms that can occur and the likelihood of them presenting themselves. An example of the tangible assessment of hardware would be that there are tests that can be done to estimate the degradation over time of a material. The likelihood of a potential hazardous situation arising from use of the device prior to this time-frame is low, and can be made lower by inherent safety design.

However, the assessment of risk from a usability perspective has the less reliable ‘Human Factor’ thrown into the mix, and standards and guidance have tried to accommodate for this. The ANSI/AAMI IEC 62366 standard; 62366-1:2015 favours hazards and hazard-related use scenarios in lieu of risk because hazard-related use-scenarios ‘can be based on the severity of the potential consequences of the associated hazards’ whereas risk relies partly on ‘probability of occurrence of encountering a hazard, which is one component of risk, can be very difficult to estimate.’ The MHRA released a draft guidance in June 2016, in which they outline that ‘Critical tasks must be defined and prioritised by design teams based on the severity of harm presented’. The FDA also acknowledges in their guidance – UCM259760 – that probability is ‘very difficult to determine for use errors’ and their opinion is;

‘Many use errors cannot be anticipated until device use is simulated and observed, the severity of the potential harm is more meaningful for determining the need to eliminate or reduce resulting harm. If the results of risk analysis indicate that use errors could cause serious harm to the patient or the device user, then the manufacturer should apply appropriate human factors or usability engineering processes.’

So ‘risk’ is used to determine potential harm associated with a medical device in ISO 14971:2007.  Conversely, ‘hazardous use scenarios’ are used to determine potential harm associated with user-device interaction in IEC 62366-1:2015 and by the USA and the UK regulatory bodies in their recommended approaches to usability testing. The question is, what does this mean when assessing potential harm, and how should manufacturers evaluate use-related risk?

Manufacturers usually use a ‘usability Failure Modes and Effects Analysis’ (uFMEA) which is – in principle – a document that is an altered version of a FMEA. The documents although titled similarly and designed similarly by most manufacturers, have differences in end goals and techniques used to reach the end goals. The reason for this is that the FMEA is used to analyse potential risk with a device as per ISO 14971:2007. However, the uFMEA should be used to analyse potential hazardous use scenarios, as per IEC 62366-1:2015 and supporting national guidance.

If you would like to discuss this methodology further, contact Medical Device Usability. We look forward to collaborating with you.


Standards and guidance mentioned in this blog;

IEC 62366-1:2015 – Medical devices – Part 1: Application of usability engineering to medical devices

ISO 14971:2017 – Medical devices – Application of risk management to medical devices

Human Factors and Usability Engineering – Guidance for Medical Devices Including Drug-device Combination Products, MHRA (Draft for comment June 2016)

Applying Human Factors and Usability Engineering to Medical Devices, Guidance for Industry and Food and Drug Administration Staff (UCM259760, February 2016)