In this human-written article I’ll describe how design teams innovate by paying more attention to what people do and giving skeptical attention to what they say they do. These insights are derived from Engenious Design work helping clients design and develop medical devices.

The flossing disconnect

Most people agree that dental flossing is a good idea. But how many people actually flossed this morning?^[1] That’s a disconnect between the ideals we report to believe, our good intentions, and our real actions. This is an example of how asking people what they do will miss important aspects of what they really do. I call this gap between sincere belief and practice the flossing disconnect. This disconnect translates to healthcare work, where there is a wealth of research-driven data on best clinical practices. And also gaps between what clinicians agree are best and what clinicians really do in clinic.

Medical device implications

We see this flossing disconnect in our work at Engenious Design where our team of 100 helps clients create new medical devices. When we do field research for a new project and ask clinicians about practice, the answers sound pretty good. But when we observe what really happens in clinic, the reality is often different from what we are told. Sometimes much different! This is why we observe first and ask questions second.

Observe first, question second

When studying workflows, observing human factors, usability engineering or product management research for a project, we emphasize getting into clinic. Being in the clinical environment in person, as early as possible is one of the most important early investments a project can make in overall success.

For this reason our ID/UX/HF/UE teams maintain vaccinations and credentials across multiple agencies, so we can see for ourselves what clinicians do, sometimes at a moments notice.

This can look like recent visits to observe surgeons, nurses and support team in ambulatory surgery centers and operating rooms. When we observed surgeries, we were able to discover a wealth of insights into workflows, pain points and where a new system could leapfrog the competition.

Or another set of visits to home care settings where we observed patients and caregivers in homes, skilled nursing facilities, retirement homes and hospital settings. Our design team was able to develop data-driven design constraints that affected part design, alert indicators and other nuances that make big differences in usability.

The Hawthorne effect (being watched)

Professor Henry Landsberger coined the “Hawthorne effect”, where people being observed modify their behavior^[2]. This stemmed from a 1920’s study conducted at the Western Electric factory (Hawthorne Works) that made electrical relays. They set out to conduct productivity studies on factory lighting, but were surprised to discover a more impactful productivity increase resulted from being observed^[3,4].

The Hawthorne effect is on our mind when conducting field research and it’s nearly impossible to eliminate. But we can take measures to reduce the effects, specifically by not letting subjects know exactly what we are studying and being intentionally vague about our fieldwork. If we are studying how clinicians grip rotary tools in surgery, we won’t specifically call out those study aims. Instead we might say that we are studying surgical process, and even ask general background questions while quietly observing and video recording (with permission) rotary tool grips.

Steps that work, in order:

Here’s the sequence we have found works best in practice. In summary, we do our homework, observe clinicians, then return with informed questions.

1. Discover: We do our homework, learning the lingo, the players and what research says we should see.
2. Observe: Get into clinical environments and quietly observe. Taking notes, pictures and video where we can.
3. Diagram: Distill the observations into a workflow diagram. We have a format that works well; send a note to request a copy.
4. Question: We then talk directly to clinicians, armed with our observations to ask the right questions about what we saw.
5. Show: When we have early concepts for innovation, we show them to clinicians, then observe how they try using them, following up with questions.
6. Iterate: We learn a lot when we watch people using our prototypes. And we always plan to integrate improvements into the next version, so we can do it all again.

Pain drives innovation

It’s often said that it’s easier to sell pain killers than vitamins. I’m of the opinion that the best innovations are ones that address user pain points. Sometimes innovation simply improve a product, and that’s okay. But game-changers aren’t just a little better, they often address pain points the user couldn’t even voice on first contact.

The apocryphal Henry Ford saying "If I had asked people what they wanted, they would have said faster horses" is perhaps a myth, but the idea is very real. It’s better to observe pain points and address them, because designers can’t always trust users to be objective about what they need and how they work.

Finding pain

A documented workflow analysis is a discipline that we find helpful to quickly communicate what’s happening and also capture pain points. This is a scalable system for capturing the key steps, hazards and pain points for someone doing something. The format is deceptively simple, and it took years to boil down to the essential elements. But the system is an efficient way of finding pain points.

The system also helps satisfy the needs of a formative usability study, required by IEC 62366-1^[5]. The system provides important inputs to meet ISO 14971 for risk management^[6]. And it’s also a great way for product managers, designers and engineers to work from a common understanding of workflow, user needs and hazards. Message us for a copy of our format, or expert help executing a workflow study.

Seeing risks

Risk management articulated in ISO 149716^[7]. treats use errors as hazards, including reasonably foreseeable misuse. By observing clinical practice with an eye towards use hazards, design teams meet the intent of identifying “reasonably foreseeable misuse”. Simply relying on what clinicians say they do, will not surface all hazards.

Leapfrogging the competition

In our experience innovation is almost embarrassingly easy once we know the pain points. Often the solutions are obvious after we’ve done our homework. But it’s only obvious because the insights are earned through diligent work and a smart process. In other cases, the innovation isn’t obvious and is more hard-won. But knowing the pain points first is the best way to nearly guarantee innovation will happen in product design. Many a patent has come from this observing-first methodology, because the system helps discover non-obvious opportunities.

Formative human factors credit

This whole article could be devoted to IEC 62366-15^[8], but instead I chose to focus on the pragmatic benefits of watching first, asking later. If you work in medical devices like we do, all of this work helps a team satisfy the formative human factors and usability engineering study required by the standard. If your target includes USA markets, FDA’s 2016 guidance, Applying Human Factors and Usability Engineering to Medical Devices still applies^[9,10]. Look for a future article on how our system ties into IEC 62366-1, and how that standard has benefits for even non-medical designers and R&D teams.

References:

1. Fleming EB, Nguyen D, Afful J, Carroll MD, Woods PD. "Prevalence of daily flossing among adults by selected risk factors for periodontal disease — United States, 2009–2014." Journal of Periodontology, 2018. https://pmc.ncbi.nlm.nih.gov/articles/PMC6434526/

2. Landsberger HA. Hawthorne Revisited: Management and the Worker, Its Critics, and Developments in Human Relations in Industry. Cornell University, 1958.

3. Levitt SD, List JA. "Was there really a Hawthorne effect at the Hawthorne plant? An analysis of the original illumination experiments." American Economic Journal: Applied Economics, 3(1), 2011, 224–238.

4. McCambridge J, Witton J, Elbourne DR. "Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects." Journal of Clinical Epidemiology, 67(3), 2014, 267–277.

5. IEC 62366-1:2015+AMD1:2020, Medical devices — Part 1: Application of usability engineering to medical devices. https://webstore.iec.ch/en/publication/67220

6. ISO 14971:2019, Medical devices — Application of risk management to medical devices. https://www.iso.org/standard/72704.html

7. ISO/TR 24971:2020, Medical devices — Guidance on the application of ISO 14971.

8. IEC/TR 62366-2:2016, Medical devices — Part 2: Guidance on the application of usability engineering to medical devices. https://webstore.iec.ch/publication/24664

9. FDA CDRH. Applying Human Factors and Usability Engineering to Medical Devices: Guidance for Industry and Food and Drug Administration Staff. Issued February 3, 2016. https://www.fda.gov/media/80481/download

10. FDA CDRH. Content of Human Factors Information in Medical Device Marketing Submissions. https://www.fda.gov/media/163694/download

‍