Regulation (EU) 2017/745, what does it mean for usability testing?
For those of you out there who thought usability testing was a small-time, unimportant aspect of medical device development (or are dealing with upper management who believe so), this new Medical Device Regulation (MDR) is further evidence (62366 and UCM259760 already making the point) to prove otherwise.
Chapter 1 of Annex 1 of the Regulation (General safety and performance requirements) pages 96 – 97 outline the risk management that is required of manufacturers through the lifecycle of their medical devices. A lot of these outlined requirements can be directly measured with usability engineering, which is handy because the Regulation goes on to say;
‘The documentation shall contain the results and critical analyses of all verifications and validation tests and/or studies undertaken to demonstrate conformity of the device with the requirements of this Regulation and in particular the applicable general safety and performance requirements’ – Page 111, point #6.
By ‘May 26th 2020’ all manufacturers aiming to market their medical devices within the EU will be required to conform to the Regulation and all harmonised standards. This does not include ISO standard 62366-1:2015 – ‘Application of Usability Engineering to Medical Devices’, but does include ISO14971 (‘application of risk management to medical devices’), which 62366 refers to throughout. This means reducing risk to a residual amount and then providing evidence of validation of these risk reduction methods – a roundabout way of saying a usability engineering file should be a part of submission documentation. In this article, we will look at some of the General Requirements outlined in Chapter 1 of the ‘General safety and Performance Requirements’ and how they can be most optimally completed through iterative usability testing.
General requirements point 3 (sub-points ‘b’ and ‘c’) ‘Establish, implement, document and maintain a risk management system’
Sub-points b through d of General Requirements point 3 are echoed in ISO standard 62366-1:2015. Point b; ‘identify and analyse the known and foreseeable hazards associated with each device’ reads as though it has been inspired by section 5.3 of ISO standard 62366-1:2015; – ‘The manufacturer shall identify known or foreseeable hazards and hazardous situations’. The Regulation and the standard are looking for manufacturers to do the same thing. Techniques commonly used to foresee hazards associated with use are Task Analysis, ‘Perception, Cognition and Action (PCA)’ and literature reviews of similar devices. These techniques can be implemented at any stage of the design process, and ideally they are re-applied through development of the device.
Point c: ‘Estimate and evaluate the risks associated with, and occurring during, the intended use and during reasonably foreseeable misuse’. Simulated use is an accepted way to assess the effectiveness of risk mitigation strategies. Risk is defined in the Regulation as the combination of severity and probability of a hazard. Neither 62366 nor FDA guidance UCM259760 (‘Applying Human Factors and Usability Engineering to Medical Devices’) condone the use of probability when determining the tasks that are critical for safe use. 62366 states that it is ‘difficult to estimate with accuracy the probability of a use error occurring’ (page 18). The usability engineering process therefore relies on severity, rationalising that if a use error may result in a harm with a severity of ‘5’ (Death), it doesn’t matter how likely it is that it may happen, it needs to be considered, mitigated and that mitigation assessed. Alternatively, if there are severity ‘1’ harms with high probability, they will likely be explored through simulated use anyway – as there is a high chance of occurrence.
Usability specialists have adapted the Failure Modes and Effects Analysis technique (the uFMEA) to organise risk related to use-errors into something comprehensive. The uFMEA is intended as a living document through development and production of a medical device, and if utilised properly in conjunction with usability testing the uFMEA can be used to determine the final critical tasks for assessment in validation testing. A good use of the uFMEA technique can be the difference between 30 or 5 critical tasks for validation but also provides something physical that you can submit within the usability engineering file as part of submission.
General requirements point 3, sub-point ‘d’ and point 4
Point d: ‘Eliminate or control the risks referred to in point ‘c’ in accordance with the requirements of Section 4’. Section 4 of the general requirements states that risk control measures shall ‘conform to safety principles’ and ‘manage risks so residual risk associated with each hazard as well as overall residual risk is judged acceptable’. Usability testing is primarily concerned with managing risk associated with hazards and hazardous use scenarios and therefore is compelling as the primary way to conform to this requirement.
The sub-points of Point 4 are also in-line with current requirements for usability testing as recommended by the ISO 14971, where both recommend prioritising elimination/reduction of risk as far as possible through design, then ‘protective measures’ (such as alarms) and finally information for safety. The use of usability testing iteratively throughout the design process for a medical device will result in greater risk mitigation measures through design, recognised as the gold standard of risk mitigation by the FDA and in ISO 14971, and now in EU Regulation 2017/745. When conducting formative usability testing, we here at Medical Device Usability ensure to produce constructive solutions to risks that are observed and acknowledged as being a potential hazard to users and patients.
The EU and the USA
The EU Regulation and the Code of Federal Regulations (CFR) are now more harmonised, showing that the EU is now catching up with the USA on usability testing – Section 820.30 (Design Controls) of CFR points ‘f’ and ‘g’ outline that manufacturers must verify and then validate the design of their device through simulated use with a high-fidelity/production line prototype of the device. If the EU is following suit of the USA, then the need for usability testing will increase from here – onwards.
Ultimately, this application of ergonomic principles to the design of medical devices can only serve to improve usability of devices and therefore:
- Reduce time spent on training (through intuitive design)
- Reduce risk of use errors
- Increase likelihood of adherence of medication
- Increase patient independence
To name a few of the positives of usability testing.
We at Medical Device Usability Ltd. practice the methods required to conform to the general requirements of the Regulation as discussed above. This is regardless of whether we are conducting usability testing for our clients in the USA and complying with UCM259760 or for our clients in the EU whilst complying with 62366. This Regulation gives us and all HF practitioners the opportunity to work more closely with manufacturers to ensure the methods used comply with EU Regulation.
2012. International Organisation for Standardisation 14971:2009, Medical devices – Application of risk management to medical devices.
2015. International Electrotechnical Commission 62366-1:2015, Medical devices – Part 1: Application of usability engineering to medical devices.
2016. International Electrotechnical commission Technical Report 62366-2:2016, Medical devices – Part 2: Guidance on the application of usability engineering to medical devices.
2016. Food and Drug Administration, Center for Devices and Radiological Health UCM259760, Applying Human Factors and Usability Engineering to Medical Devices.
2017. Electronic Code of Federal Regulations, Part 820 – Quality System Regulation.
Official Journal of the European Union, 2017. Legislation, Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC.