Research Methodology

Cycle 1: Concept Testing & Desirability Study

Study Design

I led a concept testing and desirability study to reveal the values that are important to patients when onboarding onto a digital medical offering.

We created storyboard concepts to present to patients as stimuli to elicit reactions.

Key questions we were addressing in this study:

-       Do patients want an onboarding experience that is transactional or relational? (transactional or relational)

-       How important is it that onboarding is integrated with their existing care? (care integration)

-       How much product education needs to be experiential? (sample/pre-product experience or instructive)

To answer these questions, I presented a diverse sample of patients with 8 mid-fidelity concepts (storyboards with titles, descriptions, and an illustration) that captured these factors at opposite ends of the spectrum. Interviews were semi-structured and 60 minutes in duration. I asked participants these questions about each concept:

■      What are any thoughts about this? Why?

■      What do you like about this if anything? Why?

■      What do you not like about this if anything? Why?

■      Walk me through how you might use this if at all. Why?

■      What problems might you run into with this?

Synthesis and analysis was conducted using Google Spreadsheets, Miro, Transcripts.

Cycle 2: Desirability & Comprehension Study

Study Design

In Cycle 2, we narrowed down to (2) concepts for which we tested desirability and comprehension.

The first concept was a “Take a tour” concept, and the second was an “Experiential” concept.

This study assessed Desirability (motivation, curiosity, and trust) through a concept evaluation and also assessed comprehension.

It was a (2) part interview with (10) participants. Each part was an hour for a total of (2) hours per participant.

Hypotheses:

We hypothesized that patient mental models and previous experience with prescription medicine, mental health treatment, and digital health products will influence their expectations about onboarding and telemedicine onto our PDT.  

We further hypothesize that patients are still not familiar with the concept of PDT and will need clear explanations as to the mechanism and benefits of PDTs. Based on prior research, it will be important to give patients visibility into the clinical backing of the product and safety, how it works, and who is behind the product.

Research Questions:

Desirability

-       What are each patient’s level of curiosity, trust, and motivation at each step of onboarding in each onboarding version (Take a Tour, and Experiential)? What feelings and concerns are evoked with each concept?

-       How do patients feel about each onboarding version, which do they prefer, and why?

Comprehension

-       Do patients understand the product and the onboarding as a whole? If not, where are the gaps that we need to fill in in order to give a more complete and easy-to-understand picture of the what and why of onboarding?

-       Do patients understand what we are trying to get across at each step of onboarding?

Discussion Questions (sample):

Based on what you see here, how would you describe the app to a friend?

[success criteria] Patient mentions at least (3)

-       App is a treatment

-       App needs a prescription

-       It’s FDA authorized

-       Doctor has access to my progress

-       You can review treatment lessons

-       You can track triggers/cravings

-       Earn rewards

[testing] comprehension of screen content and reSET value props

At this point, what is your level of motivation to continue? What makes you feel that way?

[testing] desirability/motivation

Synthesis & Analysis

My teammates took notes during interviews and we rated them pass/fail for each

comprehension task. We also took qualitative notes on Miro Board. Finally, I

ordered transcripts for each interview, which I analyzed and coded.

Results

In Cycle 2, our team learned that a take-a-tour pre-product experience combined with limited experiential features would be most promising. We also learned that we could improve our descriptions of the product offerings in the first 3 screens, and that peer support through video testimonials is emotionally compelling and critical for conversion,

Cycles 3 & 4: Desirability & Usability Studies

Study Design

Cycles 3 and 4 were identical in terms of research methodology. However, we created different research artifacts and prototypes since we tested with the chronic insomnia population, and the opioid use disorder population. Our hypothesis and onboarding strategy for both communities was the same:

Emotional engagement and product clarity/comprehension drive motivation, leading to greater conversion.

We evaluated:

-       Usability of the interface (ui) and comprehension of the product offerings and the onboarding process.

-       Desirability through an understanding of emotional responses and engagement at each step of the introductory and onboarding experience.

Results

The prototypes in Cycles 3 and 4 tested positively and led to our proposed roadmap for optimal patient success.

We proposed:

1. Supportive videos: video production to increase patients’ interest and understanding of how a PDT can help patients through supportive and empathic videos.

2. In-app experience: turn customer interest into action with an easy-to-navigate in-app home screen experience and low-friction account creation.

_________________________________

Research Artifacts Cycles 1-4

With my outstanding and visionary team, we prototyped a variety of ideas that generated strong reactions.

Cycles 1 and 2 were discovery-based studies and helped narrow down the best ideas to move forward with.

In cycles 3 and 4, we evaluated the solution through usability testing.

We initially prototyped a variety of onboarding concepts to test values such as relational vs transactional onboarding, integrated vs standalone onboarding, and experiential vs instructive onboarding.

Thereafter, prototyping evolved to engage patients’ varied learning styles, with a particular focus on video-based learning. We prototyped a video series of experts and patients talking about the benefits of the product.

_________________________________

Participant Recruitment Cycles 1-4

We recruited 40 participants total with diverse clinical indications (chronic insomnia, SUD, OUD, MDD). I wrote screeners for inclusion and exclusion criteria and I worked closely with a recruitment agency to secure participants for the studies.

In each clinical indication segment, we secured participants along extremes of health condition severity, healthcare service utilization, technological savviness, experience with telemedicine, and different types of Insurance coverage (private commercial insurance, and other types of insurance).

Importantly, as part of an initiative to represent more diversity in research and in healthcare, we secured a diverse sample of participants (ethnicity, gender, educational levels, and income levels).