Katherine L. Milkman

Aneesh Rai, Marissa A. Sharif, Edward H. Chang, Katherine L. Milkman, Angela Duckworth (2022), A Field Experiment on Subgoal Framing to Boost Volunteering: The Trade- Off Between Goal Granularity and Flexibility, Journal of Applied Psychology. https://doi.org/10.1037/apl0001040

Abstract: Research suggests that breaking overarching goals into more granular subgoals is beneficial for goal progress. However, making goals more granular often involves reducing the flexibility provided to complete them, and recent work shows that flexibility can also be beneficial for goal pursuit. We examine this trade-off between granularity and flexibility in subgoals in a preregistered, large-scale field experiment (N = 9,108) conducted over several months with volunteers at a national crisis counseling organization. A preregistered vignette pilot study (N = 900) suggests that the subgoal framing tested in the field could benefit goal seekers by bolstering their self-efficacy and goal commitment, and by discouraging procrastination. Our field experiment finds that reframing an overarching goal of 200 hr of volunteering into more granular subgoals (either 4 hr of volunteering every week or 8 hr every 2 weeks) increased hours volunteered by 8% over a 12-week period. Further, increasing subgoal flexibility by breaking an annual 200-hr volunteering goal into a subgoal of volunteering 8 hr every 2 weeks, rather than 4 hr every week, led to more durable benefits.

L John, Hayley Blunden, Katherine L. Milkman, L Foschini, Bradford Tuckfield (2022), The Limits on Inconspicuous Incentives, Organizational Behavior and Human Decision Processes, 172 ().

Katherine L. Milkman, Linnea Gandhi, Sean F. Ellis, Heather N. Graci, Dena Gromet, Rayyan S. Mobarak, Alison Buttenheim, Angela Duckworth, Devin Pope, Ala Stanford, Richard H Thaler, Kevin Volpp (2022), A Citywide Experiment Testing the Impact of Geographically Targeted, High-Pay-Off Vaccine Lotteries, Nature Human Behaviour. https://doi.org/10.1038/s41562-022-01437-0

Abstract: Lotteries have been shown to motivate behaviour change in many settings, but their value as a policy tool is relatively untested. We implemented a pre-registered, citywide experiment to test the effects of three high-pay-off, geographically targeted lotteries designed to motivate adult Philadelphians to get their COVID-19 vaccine. In each drawing, the residents of a randomly selected ‘treatment’ zip code received half the lottery prizes, boosting their chances of winning to 50×–100× those of other Philadelphians. The first treated zip code, which drew considerable media attention, may have experienced a small bump in vaccinations compared with the control zip codes: average weekly vaccinations rose by an estimated 61 per 100,000 people per week (+11%). After pooling the results from all three zip codes treated during our six-week experiment, however, we do not detect evidence of any overall benefits. Furthermore, our 95% confidence interval provides a 9% upper bound on the net benefits of treatment in our study.

Mitesh Patel, Katherine L. Milkman, Linnea Gandhi, Heather N. Graci, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Jake Rothschild, Modupe Akinola, John Beshears, Jonathan E. Bogard, Alison Buttenheim, Christopher F. Chabris, Gretchen B. Chapman, James J. Choi, Hengchen Dai, Craig R. Fox, Amir Goren, Matthew D. Hilchey, Jillian Hmurovic, Leslie K. John, Dean Karlan, Melanie Kim, David Laibson, Cait Lamberton, Brigitte C. Madrian, M. Meyer, Maria Modanu, Jimin Nam, Todd Rogers, Renante Rondina, Silvia Saccardo, Maheen Shermohammed, Dilip Soman, Jehan Sparks, Caleb Warren, Megan Weber, Ron Berman, Chalanda N. Evans, Seung Hyeong Lee, Christopher K. Snider, Eli Tsukayama, Christophe Van den Bulte, Kevin Volpp, Angela Duckworth (2022), A Randomized Trial of Behavioral Nudges Delivered through Text Messages to Increase Influenza Vaccination Among Patients with an Upcoming Primary Care Visit, American Journal of Health Promotion, 37 (3), pp. 324-332.

Abstract: Purpose: To evaluate if nudges delivered by text message prior to an upcoming primary care visit can increase influenza vaccination rates. Design: Randomized, controlled trial. Setting: Two health systems in the Northeastern US between September 2020 and March 2021. Subjects: 74,811 adults. Interventions: Patients in the 19 intervention arms received 1-2 text messages in the 3 days preceding their appointment that varied in their format, interactivity, and content. Measures: Influenza vaccination. Analysis: Intention-to-treat. Results: Participants had a mean (SD) age of 50.7 (16.2) years; 55.8% (41,771) were female, 70.6% (52,826) were White, and 19.0% (14,222) were Black. Among the interventions, 5 of 19 (26.3%) had a significantly greater vaccination rate than control. On average, the 19 interventions increased vaccination relative to control by 1.8 percentage points or 6.1% (P = .005). The top performing text message described the vaccine to the patient as “reserved for you” and led to a 3.1 percentage point increase (95% CI, 1.3 to 4.9; P < .001) in vaccination relative to control. Three of the top five performing messages described the vaccine as “reserved for you.” None of the interventions performed worse than control. Conclusions: Text messages encouraging vaccination and delivered prior to an upcoming appointment significantly increased influenza vaccination rates and could be a scalable approach to increase vaccination more broadly.

Harsha Thirumurthy, Katherine L. Milkman, Kevin Volpp, Alison M. Buttenheim, Devin G Pope (2022), Association Between Statewide Financial Incentive Programs and COVID-19 Vaccination Rates, PLoS One, 17 (3).

Abstract: To promote COVID-19 vaccination, many states in the US introduced financial incentives ranging from small, guaranteed rewards to lotteries that give vaccinated individuals a chance to win large prizes. There is limited evidence on the effectiveness of these programs and conflicting evidence from survey experiments and studies of individual states’ lotteries. To assess the effectiveness of COVID-19 vaccination incentive programs, we combined information on statewide incentive programs in the US with data on daily vaccine doses administered in each state. Leveraging variation across states in the daily availability of incentives, our difference-in-differences analyses showed that statewide programs were not associated with a significant change in vaccination rates. Furthermore, there was no significant difference in vaccination trends between states with and without incentives in any of the 14 days before or after incentives were introduced. Heterogeneity analyses indicated that neither lotteries nor guaranteed rewards were associated with significant change in vaccination rates.

Alison Buttenheim, Katherine L. Milkman, Angela Duckworth, Dena Gromet, Mitesh Patel, Gretchen B. Chapman (2022), Effects of Ownership Text Message Wording and Reminders on Receipt of an Influenza Vaccination: A Randomized Clinical Trial, Journal of the American Medical Association Network Open, 5 (2).

Abstract: Importance  Despite the availability of safe and effective vaccines, many people fail to get vaccinated. Messages using behavioral science principles may increase vaccination rates. Objective  To determine the effect on influenza vaccination rates of a text message telling patients that an influenza vaccine had been reserved for them. Design, Setting, and Participants  As part of a larger influenza vaccine messaging megastudy, in this randomized clinical trial, 11 188 patients in 2 large health systems were assigned to receive a text message that stated “a flu shot has been reserved for you,” a text message that stated “flu shots will be available,” or no text message. Both messages included the option to reply yes (Y) or no (N) to indicate that the patient wanted to get vaccinated. Patients 18 years or older were included if they had new or routine (nonsick) primary care appointments scheduled from September 20, 2020, through March 31, 2021. Interventions  The evening before the scheduled appointment, patients in the 2 message conditions were sent 3 back-to-back SMS messages containing the study wording. Patients in the usual care control group did not receive any study messages. Main Outcomes and Measures  Receipt of an influenza vaccine on the date of the patient’s scheduled appointment. Results  A total of 11 188 patients were randomized to the reserved or the available message conditions or to usual care. The 10 158 patients analyzed in the study had a mean (SD) age of 50.61 (16.28) years; 5631 (55.43%) were women; and 7025 (69.16%) were White. According to health records, 4113 (40.49%) had been vaccinated in the previous influenza season, and 5420 (53.36%) were patients at Penn Medicine. In an intent-to-treat analysis, changes in vaccination rates in response to the reserved message did not reach statistical significance (increase of 1.4 percentage points, or 4% [P = .31]) compared with the message conveying that influenza vaccines were available. Relative to the usual care control, the reserved message increased vaccination rates by 3.3 percentage points, or 11% (P = .004). Patients in the reserved message condition were more likely to text back Y (1063 of 3375 [31.50%]) compared with those in the available message condition (887 of 3351 [26.47%]; χ2 = 20.64; P < .001), and those who replied Y were more likely to get vaccinated (1532 of 1950 [78.56%]) compared with those who did not (749 of 4776 [15.68%]; χ2 = 2400; P < .001). Conclusions and Relevance  This study found that patients who received text messages regarding flu vaccination had greater vaccine uptake than those who received no message. Messages that increase the likelihood that patients will indicate their intention to be vaccinated may also increase vaccination behavior.

Katherine L. Milkman, Linnea Gandhi, Mitesh Patel, Heather N. Graci, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Jake Rothschild, Jonathan E. Bogard, Ilana Brody, Christopher F. Chabris, Edward Chang, Gretchen B. Chapman, Jennifer E. Dannals, Noah J. Goldstein, Amir Goren, Hal E. Hershfield, Alexander Hirsch, Jillian Hmurovic, Samantha Horn, Dean Karlan, Ariella Kristal, Cait Lamberton, M. Meyer, Allison H. Oakes, Maurice Schweitzer, Maheen Shermohammed, Joachim H. Talloen, Caleb Warren, Ashley Whillans, Kuldeep N. Yadav, Julian J. Zlatev, Ron Berman, Chalanda N. Evans, Rahul Ladhania, Jens Ludwig, Nina Mazar, Sendhil Mullainathan, Christopher K. Snider, Jann Spiess, Eli Tsukayama, Lyle Ungar, Christophe Van den Bulte, Kevin Volpp, Angela Duckworth (2022), A 680,000-Person Megastudy of Nudges to Encourage Vaccination in Pharmacies, Proceedings of the National Academy of Sciences, 119 (6). 10.1073/pnas.211512611

Abstract: Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was “waiting for you.” Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

Katherine L. Milkman, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Predrag Pandiloski, Yeji Park, Aneesh Rai, Max Bazerman, John Beshears, Lauri Bonacorsi, Colin Camerer, Edward Chang, Gretchen B. Chapman, Robert Cialdini, Hengchen Dai, Lauren Eskreis-Winkler, Ayelet Fishbach, James J. Gross, Samantha Horn, Alexa Hubbard, Steven J. Jones, Dean Karlan, Tim Kautz, Erika Kirgios, Joowon Klusowski, Ariella Kristal, Rahul Ladhania, George Loewenstein, Jens Ludwig, Barbara Mellers, Sendhil Mullainathan, Silvia Saccardo, Jann Spiess, Gaurav Suri, Joachim H. Talloen, Jamie Taxer, Yaacov Trope, Lyle Ungar, Kevin Volpp, Ashley Whillans, Jonathan Zinman, Angela Duckworth (2021), Megastudies Improve the Impact of Applied Behavioural Science, , 600 (), pp. 478-483.

Abstract: Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

Katherine L. Milkman, Mitesh Patel, Linnea Gandhi, Heather N. Graci, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Modupe Akinola, John Beshears, Jonathan E. Bogard, Alison Buttenheim, Christopher F. Chabris, Gretchen B. Chapman, James J. Choi, Hengchen Dai, Craig R. Fox, Amir Goren, Matthew D. Hilchey, Jillian Hmurovic, Leslie K. John, Dean Karlan, Melanie Kim, David Laibson, Cait Lamberton, Brigitte C. Madrian, M. Meyer, Maria Modanu, Jimin Nam, Todd Rogers, Renante Rondina, Silvia Saccardo, Maheen Shermohammed, Dilip Soman, Jehan Sparks, Caleb Warren, Megan Weber, Ron Berman, Chalanda N. Evans, Christopher K. Snider, Eli Tsukayama, Christophe Van den Bulte, Kevin Volpp, Angela Duckworth (2021), A Megastudy of Text-Based Nudges Encouraging Patients to Get Vaccinated at an Upcoming Doctor’s Appointment, Proceedings of the National Academy of Sciences, 118 (20). 10.1073/pnas.2101165118

Abstract: Many Americans fail to get life-saving vaccines each year, and the availability of a vaccine for COVID-19 makes the challenge of encouraging vaccination more urgent than ever. We present a large field experiment (N = 47,306) testing 19 nudges delivered to patients via text message and designed to boost adoption of the influenza vaccine. Our findings suggest that text messages sent prior to a primary care visit can boost vaccination rates by an average of 5%. Overall, interventions performed better when they were 1) framed as reminders to get flu shots that were already reserved for the patient and 2) congruent with the sort of communications patients expected to receive from their healthcare provider (i.e., not surprising, casual, or interactive). The best-performing intervention in our study reminded patients twice to get their flu shot at their upcoming doctor’s appointment and indicated it was reserved for them. This successful script could be used as a template for campaigns to encourage the adoption of life-saving vaccines, including against COVID-19.

John Beshears, Hengchen Dai, Katherine L. Milkman, Shlomo Benartzi (2021), Using Fresh Starts to Nudge Increased Retirement Savings, Organizational Behavior and Human Decision Processes, 167 (), pp. 72-87.

Abstract: We conducted a field experiment to study the effect of framing future moments in time as new beginnings (or “fresh starts”). University employees (N = 6,082) received mailings with an opportunity to choose between increasing their contributions to a savings plan immediately or at a specified future time point. Framing the future time point in relation to a fresh start date (e.g., the recipient’s birthday, the first day of spring) increased the likelihood that the mailing recipient chose to increase contributions at that future time point without decreasing their likelihood of increasing contributions immediately. Overall, fresh start framing increased retirement plan contributions in the eight months following the mailing. Our findings represent the first experimental demonstration of the benefits of fresh start framing in a consequential field setting.