user research:
a triangulated approach
Client: an award winning mobile healthcare company, partnered with Mad*Pow x Bentley U
Role: UX Researcher, Moderator, Recruitment Survey Design
Skills: Usability Testing, ethnography / Contextual inquiry, Study Design, Moderating, Heuristic Review
Tools: Qualtrics, zoom
description
Examine the usability of a new mobile application from a cloud-based platform that allows hospitals and emergency departments to communicate, send, and triage incoming ER cases, in order to accelerate life-saving care.
This platform operates as a system: EMS providers can use a smartphone app to share patient demographics, vital signs, EKGs, photos, and videos on-route to the ER. Each case is tracked by GPS and sent to hospitals in real-time via a corresponding dashboard that tracks incoming ambulances. On-call physicians receive real-time alerts via the latest product in this series, and app allowing them to view patient clinical data and communicate with the rest of the care team instantaneously. This project was completed as a team of 5 user researchers.
challenge
Examine app usability and options for customization within the user interface for varying end users, paying special attention to notifications and filters, in order to support business goals of user adoption and retention. Study the needs of three types of users that our client had identified as critical personas (Emergency Department [ED] Managers, Specialists, and Charge Nurses).
method
Triangulation of three research approaches to learn the effectiveness and learnability of the app and determine current obstacles:
Expert Review - conducted by each team member, then aggregated
Contextual Inquiry - observational visit to a hospital that is currently using the system to gain better understanding of user needs
1:1 Usability Sessions - in-depth interview sessions with representative users
PROCESS OVERVIEW:
In-Depth Expert Review —> Contextual Inquiry —> Usability Study —> Final Findings & Recommendations
expert review
Each team member conducted an in-depth expert review individually, using Nielsen's 10 Heuristics and Rubin’s Problem Severity Scale. Findings were discussed, aggregated, and presented to our stakeholders.
Conducting an expert review first allowed us to:
Familiarize ourselves with the user journey
Identify surface level problems quickly so that valuable user testing time could be used to take a deeper dive on specific usability issues
Use best practices and established heuristics to develop hypotheses to test
Helped shape the order of tasks and prompts in testing
Sample finding from my individual Expert Review:
Bolded items in the data section appear clickable, but are not.
To find the data indicated here, the user must know to look inside the ‘details’ tab.
Recommendation: Make the bolded data elements active links that connect to the details page, and open to the corresponding item.
discovering this allowed us to test the hypotheses that users would struggle to access the data shown here:
——> during usability testing, all 8 participants unsuccessfully tried to click the bolded items
CONTEXTUAL INQUIRY
An observational visit was made to a hospital Emergency Department (ED) that currently uses this technology alongside traditional radio communication. I observed the context of use for the application, differences in workflow between the two technologies, and varying user needs in the ED and ambulance environments.
This involved observation and informal interviewing of several types of end users including:
Charge nurse
EMS dispatcher
EMS Director
EMT
contextual inquiry highlights
USABILITY TESTING
Using insights from our heuristic review and contextual inquiry, a moderator's guide was developed, with tasks specifically intended to:
Investigate the workflow for receiving notifications and filtering emergency cases
Understand the experience for first-time users
Eight 1:1 Usability sessions were conducted (in-person and remote) using test accounts within the app.
recruit: 1 ems manager; 3 neurology residents, 2 physicians, 1 emergency physician, 1 registered nurse
Key tasks included:
Viewing incoming ER Cases
Receiving Notifications from other members of the ER/EMT staff
Filtering for ER Cases relevant to user type
Filtering Notifications
Setting a logout/end time
USABILITY RESULTS
QUALITATIVE
QUANTITATIVE
selected quotes helped build the story of the user experience for the target demographic
“These categories aren’t quite [specific] enough...but if they were too narrow I’m worried I’d miss a patient.”
“I got an alert! So...I press View?
...I would have thought it would bring me right to that case.”
Average Ratings of Task Difficulty
At the end of each task, users were asked to rate the process according to level of difficulty, where 1= Easy and 5 = Difficult.
FINAL FINDINGS & RECOMMENDATIONS
finding
Overall, the app is intuitive to navigate and use, even for first time users
Certain features only apply to specific user types and confuse others. Slowing down or confusing users in the context of emergency care can be costly and impact life-saving outcomes
Following the data journey can be challenging for users; viewing media files and incoming cases can sometimes be difficult
Filters for cases and notifications do not have a clear distinction, leading users to wonder what they will be notified of
Recommendation
Maintain simplicity of the design, especially considering use context
Support several “profile types” that are more tailored to user needs. Allow users to sign into profile types as part of their initial signup; allow them to change their profile type if necessary
Resolve situations where interface does not meet user expectations & add search and sort functionality to cases to increase data traceability
Expand customization options, including clear choices for which notifications get pushed to the user’s phone and clear sign-out notifications