Determining a device’s potential use errors, a mini-series. Part 5 (final): Designing the uFMEA and how to populate it, a brief introduction

PUBLISHED 12th March 2018

 

Medical Device Usability has decided to post a mini-series in relation to understanding different areas of research that feed in to the ‘risk management file’ or the usability failure modes and effects analysis (uFMEA). In the fifth and final instalment of this mini-series, we will discuss the concept of a ‘uFMEA’ – the most common form of a living medical device usability risk management document. In the previous instalments, we discussed;

The intended user (Click here for Part 1)

The intended use environment (Click here for Part 2)

Investigating ‘known use problems’ (Click here for Part 3)

The terms ‘Normal use’ and ‘Abnormal use’ (Click here for Part 4)

The uFMEA is a method of risk analysis. It is commonly used in the usability engineering process, however common does not equate to simple. The method is complex, the documentation is complex and the outputs are only effectively utilised if the inputs are solid. Thus, this blog can in no way hope to cover everything, but here is a crash course in the uFMEA as a concept and why it is necessary to produce a medical device that is usable as intended by the intended user safely and effectively.

What is a usability failure modes and effects analysis (uFMEA)?

The ‘Failure Modes and Effects Analysis’ (FMEA) is a method used to analyse risk. If the primary focus of usability engineering is to ensure risk associated with use of the device is reduced as far as possible, then the method of risk assessment is the ticket that carries the device to that outcome. A common method used is an FMEA adapted for usability, often titled ‘usability Failure Modes and Effects Analysis’ (uFMEA). The uFMEA analyses individual tasks for potential use errors that can occur during interaction between the user and the device’s user interface. Potential hazardous scenarios, harms and severity of harms are determined for each use error. The MHRA describe the FMEA technique in their guidance ‘Human Factors and Usability Engineering – Guidance for Medical Devices including Drug-device combination products’. Their description is outlined below in Table 1.

Table 1: Excerpt from ‘Table 1: Key features of principal engineering techniques for medical devices; alternative approaches could be used if justified.’ MHRA Human Factors and Usability Engineering Guidance.

Table 1 Excerpt from ‘Table 1: Key features of principal engineering techniques for medical devices; alternative approaches could be used if justified.’ MHRA Human Factors and Usability Engineering Guidance

Table 1 outlines that the uFMEA ‘needs expertise in human factors to be effective’, this is a pertinent point. The context of the uFMEA is crucial to ensure the use errors included in the document are appropriate for usability testing. It is not uncommon to see that some use errors outlined in the uFMEA would be caused by abnormal use. This generally occurs when manufacturers assume that the uFMEA is populated using the same principles as a design FMEA or an engineering FMEA, however the name ‘FMEA’ is where the similarities end.

Starting the uFMEA

Early in the device development process, investigation into the intended users, use environments and known use problems can be used to contribute to an early task list or ‘task analysis’. This task analysis can be used to populate the uFMEA with foreseeable use errors. The uFMEA should be a living document, meaning the uFMEA should be updated regularly as new information becomes available. ‘New information’ includes data such as;

  • Deciding on a design concept for the device
  • Understanding who the user is and in what use environments they will use the device
  • Understanding known use problems with predicate devices and current competitor products

It is our experience at MDU that if the uFMEA is not updated throughout the design process then when the manufacturer wants to proceed with validation one of two instances take place;

  1. The uFMEA is over saturated with critical tasks, which results in a complicated validation study but also makes the device appear to have great risk. Some critical tasks may have been removed along the process, as formative testing could have resulted in solid justification to state no further investigation needed for some tasks.
  2. The uFMEA is over saturated with critical tasks that are then all removed at once prior to the validation study, resulting in a possible flag to the regulating body that there was a sharp change in the manufacturer’s view of the amount of risk associated with the device. This looks like risk has not been considered thoroughly but that tasks have been removed to increase the likelihood of passing the validation test.

‘Figure 1’ taken from FDA guidance ‘Applying Human Factors and Usability Engineering to Medical Devices’ demonstrates the Human Factors considerations that contribute to use outcomes.

Figure 1 can translate to uFMEA outputs – ‘what is the task? Will the lighting levels interfere with the user’s capabilities? What must the user do? What will the user interface communicate?’ considering questions such as these for each task can allow the manufacturer to determine where use errors are possible and how to avoid them. Once use errors have been considered for each task, it is the responsibility of the manufacturer to outline what they intend to do to reduce the risk of a use error occurring. ISO14971 outlines that risk control (or ‘mitigation’) can be achieved through one or more of the following ‘in the priority order listed; inherent safety by design, protective measures in the medical device itself or in the manufacturing process, [or] information for safety’. If the uFMEA is created early in the development process, it can be used to consider appropriate mitigation for potential use errors. The earlier a use error is identified, the easier and cheaper it is to apply ‘inherent safety by design’.

Use of uFMEA through formative testing

The usability engineering process should incorporate formative testing iteratively throughout the development of the device. The uFMEA can provide the data to design formative studies that investigate the effectiveness of intended risk mitigation methods for use of a device.

Once a formative study is complete and the data analysed, the conclusions can be used to inform the reiteration of the uFMEA. This usually means including newly observed use errors that weren’t considered previously, or stating previously known use errors need no further investigation. This cycle is continued through the design process for a medical device.

Use of uFMEA in validation

The purpose of a validation study is to determine if risk has been mitigated effectively. Therefore, the risk documentation (the uFMEA) is used to design a validation test and should not be amended at the validation stage.

With all devices there is a risk/benefit ratio that needs to be considered, therefore there will always be risk associated with using a device – the Regulators do not expect you to be able to conclude ‘there is no risk associated with this device’. If formative testing is done iteratively through design and the uFMEA kept alive through this process, the uFMEA should have more risks that would result in low severity harm than high severity harm by validation. Any remaining risks with potential for high severity harm should have appropriate mitigation outlined. The use errors that risk hazards with a high severity of harm are dubbed the ‘critical tasks’. Human Factors will be used to design a study where the critical tasks are assessed in relation to however the uFMEA communicates they may occur and how they should be mitigated. The more thorough and consistent the uFMEA, the greater the likelihood that the validation study will effectively assess the user-user interface interactions.

 

In conclusion

Implementing the usability engineering process at the beginning of the device’s development process should result in reduced risk. Documenting the risk associated with the usability of a medical device will allow Human Factors specialists to design studies that will provide risk mitigation recommendations that follow a logical sequence based on the design of the device. In contrast, if the usability engineering process or the uFMEA is started in late stages of device development the risk mitigation in relation to user-user interface interactions may not be as effective as they could be.

Bottom line, don’t let a lack of robust risk analysis methods be the reason your device falls short of safety in use – apply appropriate human factors methods from the beginning of the development process.

 

 

This concludes our mini-series on uFMEA design, if you would like to talk further or have any comments of your own please don’t hesitate to contact us!

References

  1. Food and Drug Administration, Center for Devices and Radiological Health UCM259760, Applying Human Factors and Usability Engineering to Medical Devices.

MHRA Human Factors Task and Finish Group, 2017, Human factors and usability engineering – guidance for medical devices including drug-device combination products, MHRA.