Safe and Effective Use
“Safety is 30% Common sense; 80% Compliance and the rest is good luck.” – Barry Spud
Regulations particular to medical device User Interface development, are clear on the need to develop a device or application that is Safe and Effective. The FDA, in the very first paragraph of the guidelines for Applying Human Factors and Usability Engineering to Medical Devices, defines ‘safe and effective use’ as their main concern
FDA has developed this guidance document to assist industry in following appropriate human factors and usability engineering processes to maximize the likelihood that new medical devices will be safe and effective for the intended users, uses and use environments.
The IEC 62366-1 2015 guideline on the Application of usability engineering to medical devices, in sometimes slightly different wording, expresses the same sense of focus:
Some quotes from 62366-1 to set the scene:
- USABILITY; characteristic of the User Interface that facilitates use and thereby establishes Effectiveness, Efficiency, and User satisfaction in the intended use environment
- EFFECTIVENESS; accuracy and completeness with which users achieve specified goals
- NORMAL USE; operation, including routine inspection and adjustments by any User according to the instructions for use or in accordance with generally accepted practice for those Medical Devices provided without instructions for use … Use error can occur in normal use.
- USE ERROR, User action or lack of User action while using the Medical Device that leads to a different result than that intended by the manufacturer or expected by the user.
Medical practice is increasingly using Medical Devices for observation and treatment of patients. Use errors caused by inadequate Medical Device usability have become an increasing cause for concern. Many of the Medical Devices developed without applying a Usability Engineering (Human Factors Engineering) process are non-intuitive, difficult to learn and difficult to use…. The design of the User Interface to achieve adequate usability requires Usability Engineering , intended to identify and minimise use errors and thereby reduce use-associated risks. — IEC 62366-1:2015
Three layers of defense
Human Factors Engineering adds three layers of defense that ensure that your product is designed to support safe and effective use and stands a much better chance of receiving regulatory approval.
“Human Factors Engineering predicts where foreseeable use errors will occur and implements safety features that prevent those normal use errors to result in harm to the user or the patient.”
Usability risk DETECTION
Risk comes from not knowing what you're doing
The first step is to understand your product and determine where use errors leading to patient harm can occur. It requires a through understanding of the Use environment, Users and the (intended) User Interface.
A common starting point is to perform an initial user interface risk assessment, often called a Usability Failure Mode Effect Analysis (uFMEA) but for the FDA you might be better of calling it a URRA. Using techniques like cognitive walk through and Heuristic UI evaluations, it is definitely possible to establish a good baseline understanding of your product and start predicting foreseesable user errors that could lead to patient harm and think about a designing that could mitigate those use errors.
However, as soon as possible and reasonable, you should start interacting with users through interviews and observation studies on prototypes, wireframes, mockups or existing devices to be able to detect the unforeseen use errors that could lead to patient harm. These are the errors you want to detect early. These are the big oopses that, if undetected will come and bite you in the back late in your design cycle, or even worse during your final validation or, uber-worse, after you have launched your product.
Human factors/Usability engineering is about detecting the big oopses early — Henk Peels
Usability risk MITIGATION
When you have successfully detected use errors, it simply is a matter of update risk management file and feedback the information to your design team to enable them to identify use risk mitigating design solutions.
During the design and develop phases measures to eliminate or reduce use-related hazards that could result in harm to the patient or the user can be designed and implemented. The Human Factors Engineer works with Design, Engineers and Software engineers to identify design solutions that actually mitigate the identified use errors.
The Human Factor Engineer can help the design team by performing formative use tests (observations) to get the confidence, and the evidence that the improved design indeed eliminates or sufficiently mitigates the use error. These Formative evaluations typically can be done in a very quick and efficient fashion on a very limited group of users (2-8) and can be as simple as A/ testing of 2 possible designs of a software UI screen to testing Use Error scenarios.
Formative tests and safety related use scenarios are a great source of input to design and scope the protocol for the final summative evaluation testing.
“Patient safety is the absence of preventable harm to a patient during the process of health care and the reduction of risk of unnecessary harm associated with health care to an acceptable minimum” — World Health Organization (WHO)
Usability risk EVALUATION
When you have followed the correct process, you should feel quite confident by now. You understand your product User Interface and you have established (backed up by substantial proof) that your device user interface design supports safe and effective use.
Based on the identified risks, the potential for use errors that can lead to patient harm and the evidence gathered during fomative testing you can design, plan and execute the summative usability evaluation to gather the definitive evidence your device is ready for launch.
The Summative evaluation is typically done in parallel to the clinical tests (if applicable) and has to be performed on real users, on the final design product in a (simulated) use environment. The regulations are quite clear on the minimum requirements for such studies.
Presenting the evidence that your device User Interface is safe and effective is done by means of reports that are to be included in the pre-market submission file. The format of the evidence differs depending on whether you are presenting to European (‘Human Factors Engineering file‘) or USA authorities (usability evaluation report.
Summative evaluation; User Interface evaluation conducted at the end of the development with the intent to obtain objective evidence that the User Interface can be used safely
Human factors validation testing: "Testing conducted at the end of the device development process to assess user interactions with a device user interface to identify use errors that would or could result in serious harm to the patient or user. Human factors validation testing is also used to assess the effectiveness of risk management measures. Human factors validation testing represents one portion of design validation. "
