The Doctors for Coronavirus Prevention Project Thanksgiving / Christmas Messaging Campaign

• ClinicalTrials.gov Identifier: NCT04644328
• Recruitment Status: Completed
• First Posted: November 25, 2020
• Results First Posted: December 21, 2021
• Last Update Posted: December 21, 2021
• Sponsor: National Bureau of Economic Research, Inc.
• Collaborators: Massachusetts Institute of Technology; Harvard University; Massachusetts General Hospital; Yale University; Stanford University
• Information provided by (Responsible Party): Esther Duflo, National Bureau of Economic Research, Inc.

Study Description

Brief Summary:

Facebook ads with physician-delivered videos were shown before the Thanksgiving and Christmas holidays and focused on staying safe during the COVID pandemic by limiting travel and mask-wearing.

Condition or disease	Intervention/treatment	Phase
Covid19	Behavioral: Facebook Ads on the importance of staying safe during the Thanksgiving / Christmas holiday	Not Applicable

Detailed Description:

The investigators used Facebook ads to show a 20 second video clip recorded by Massachusetts General Hospital, Harvard and Lynn Community health center doctors (6 people in all) to approximately 20,000,000 Facebook users. The ads will be shown before the Thanksgiving and Christmas holiday and will focus on staying safe - limiting travel and mask-wearing. The investigators will randomize exposure to the ad campaign at the ZIP code or county level to ask:

Do the videos change mobility and Thanksgiving and Christmas holiday travel? Do they reduce the spread of COVID-19?

Are there spillover impacts from the video messages? For example, if individuals decide to stay home, then do the geographical regions that tend to be visited by people from treated areas experience any effects, either through information spillovers or through a reduction in travel?

Study Design

• Study Type: Interventional (Clinical Trial)
• Actual Enrollment: 820 participants
• Allocation: Randomized
• Intervention Model: Sequential Assignment
• Intervention Model Description: A total of 13 states, comprising 829 counties, centrally report COVID-19 cases at the ZCTA level. Two separate experiments were run within the study, one around Thanksgiving and one around Christmas. For each experiment, approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not. In treated ZCTAs, Facebook users received ads with messages from physicians and other health personnel urging them to stay home and wear masks to stop the spread of COVID-19. Users in non-treated ZCTAs received no such ads. After exclusions, 820 counties were randomized to high- or low-intensity treatment during Thanksgiving and 767 counties were during Christmas.
• Masking: None (Open Label)
• Primary Purpose: Prevention
• Official Title: The Doctors for Coronavirus Prevention Project Thanksgiving / Christmas Messaging Campaign
• Actual Study Start Date: November 12, 2020
• Actual Primary Completion Date: February 11, 2021
• Actual Study Completion Date: February 11, 2021

Arms and Interventions

Arm	Intervention/treatment
Experimental: High-Intensity Treatment Arm; Treatment: Individuals received approximately 3 Facebook ads over a 2-week period. Each ad contained a short video recorded by a physician using a script that discusses staying safe during the holidays by considering not traveling and using a mask when appropriate. Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.	Behavioral: Facebook Ads on the importance of staying safe during the Thanksgiving / Christmas holiday; Facebook ads to show a 15 second video clip recorded by MGH, Harvard and Lynn Community health center and other health care professionals. The ads will be shown before the Thanksgiving/Christmas holiday and will focus on staying safe - limiting travel and mask-wearing. The investigators will randomize exposure to the ad campaign at the ZIP code or county level.
No Intervention: Low-Intensity Control Arm; Control: Individuals did not receive ads containing short physician-recorded videos. Randomization to treatment: The 13 states which centrally report COVID-19 cases at the ZCTA level contained 829 counties. At Thanksgiving, 9 counties were excluded due to data limitations. At Christmas, 62 counties were excluded due to data limitations and potential negative impacts in rural, conservative areas resulting from polarization in the wake of the 2020 presidential election. Approximately half of the counties were randomized to high-intensity treatment with the remaining randomized to low-intensity treatment. In high-intensity counties, 3/4 of ZCTAs were treated (i.e., Facebook users in those ZCTAs received ads) and 1/4 were not. In low-intensity counties, 1/4 of ZCTAs were treated and 3/4 were not.

Outcome Measures

Primary Outcome Measures :

1. Facebook Movement Metric [ Time Frame: November 26, 2020 (Thanksgiving); December 24-25, 2020 (Christmas); February 2-29, 2020 (Baseline benchmark) ]
   The change in movement metric is the percent change in distance covered in a day compared to the same day of the week in the benchmark period of February 2-29, 2020, by people who started the day in a particular location. We define holiday travel as travel during the three days preceding each holiday, since the available data does not allow us to compute the impact of the intervention on the return travel (after the holiday). The reason is that the mobility data describes the behavior throughout the day, for people who were in each county that morning. Since the campaign was targeted based on home location, we can only capture its impact on travel away from home, not back home.
2. Percentage Leaving Home on Day of Holiday [ Time Frame: November 26, 2020 (Thanksgiving); December 24-25, 2020 (Christmas) ]
   The Facebook stay put metric is the percentage of people who stay within a small geographical area (a "tile" of 600m*600m in which they started the day). We use it to compute the share of people leaving home on the day of the holiday (i.e. this variable = 1 - stay put on the day of the holiday).
3. Inverse Hyperbolic Sine of COVID-19 Cases [ Time Frame: December 1-14 (Thanksgiving arms) and January 1-14 (Christmas arms) ]
   Inverse hyperbolic sine of COVID-19 cases during a 14 day period starting 5 days after each holiday

Secondary Outcome Measures :

1. Knowledge of COVID-19 Prevention Message (Recall of ad, Intent to Travel, Mask Wearing and Beliefs About Travel) [ Time Frame: up to one month ]
   Facebook will ask a set of users in treated zip codes to answer four survey questions (each user answers only one question each). A small within-zip control group will be held from being treated, enabling comparisons between those who see the videos and those who don't. These questions will be asked a few days after the ad is shown on the User's feed and will include 1) recall of the ad; 2) intention of traveling over holiday; 3) intention of wearing a mask; and 4) beliefs about whether people should travel over holiday.
2. Average Number of Tiles People Occupy (Mobility Measure) [ Time Frame: up to one month ]
   this measures each day "how much people move around by counting the number of level-16 Bing tiles they are seen in within a day. People seen in more tiles are probably moving around more, while people seen in fewer are probably moving around less. Each day take eligible people in a given region and compute the number of distinct tiles they were seen in." This is aggregated to the county level.
3. Percentage of Eligible Participants Only Observed in a Single Level-16 Bing Tile (no Change in Movement) [ Time Frame: up to one month ]
   Facebook "Stay-put" data: this measures each day "the percentage of eligible people who are only observed in a single level-16 Bing tile (600m x 600m) during the course of a day" aggregated to they county level.

Eligibility Criteria

• Ages Eligible for Study: 18 Years and older (Adult, Older Adult)
• Sexes Eligible for Study: All
• Accepts Healthy Volunteers: Yes

Criteria

Inclusion Criteria:

- Individuals viewing the ads must be Facebook users, 18 years and older. Facebook will decide specifically which users receive the messages within a target geographical area (zip or county).

Exclusion Criteria:

-

Contacts and Locations

Locations

United States, Massachusetts

JPAL North America

Cambridge, Massachusetts, United States, 02138

Sponsors and Collaborators

National Bureau of Economic Research, Inc.

Massachusetts Institute of Technology

Harvard University

Massachusetts General Hospital

Yale University

Stanford University

  Study Documents (Full-Text)

Documents provided by Esther Duflo, National Bureau of Economic Research, Inc.:

Study Protocol and Statistical Analysis Plan  [PDF] December 11, 2020

More Information

Publications of Results:

Breza E, Stanford FC, Alsan M, Alsan B, Banerjee A, Chandrasekhar AG, Eichmeyer S, Glushko T, Goldsmith-Pinkham P, Holland K, Hoppe E, Karnani M, Liegl S, Loisel T, Ogbu-Nwobodo L, Olken BA, Torres C, Vautrey PL, Warner ET, Wootton S, Duflo E. Effects of a large-scale social media advertising campaign on holiday travel and COVID-19 infections: a cluster randomized controlled trial. Nat Med. 2021 Sep;27(9):1622-1628. doi: 10.1038/s41591-021-01487-3. Epub 2021 Aug 19.

• Responsible Party: Esther Duflo, Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics, MIT; Co-Director, Abdul Latif Jameel Poverty Action Lab, National Bureau of Economic Research, Inc.
• ClinicalTrials.gov Identifier: NCT04644328
• Other Study ID Numbers: 2003000118A010
• First Posted: November 25, 2020
• Results First Posted: December 21, 2021
• Last Update Posted: December 21, 2021
• Last Verified: December 2021

Individual Participant Data (IPD) Sharing Statement:

• Plan to Share IPD: Undecided
• Plan Description: The pre-analysis plan will be posted publicly. The ability to share IPD will be based on discussions with our partner organization.

• Studies a U.S. FDA-regulated Drug Product: No
• Studies a U.S. FDA-regulated Device Product: No

Additional relevant MeSH terms:

Coronavirus Infections; Coronaviridae Infections; Nidovirales Infections; RNA Virus Infections; Virus Diseases; Infections