This pilot study tested the feasibility of tracking e-cigarette initiation over three months using a burst-of-measurement approach and examined reactivity to the ecological momentary assessment (EMA) data capture technique in this context.
MethodsAdults who smoked daily and were interested in trying e-cigarette use received an e-cigarette starter kit and were randomized to complete EMA reports (n&#¿;=&#¿;30) vs. not (n&#¿;=&#¿;29). Participants (Mage=44.1 years [SD=12.6], 61.0&#¿;% male, 54.3&#¿;% White, 38.9&#¿;% Black/African American) smoked 13.0 cigarettes per day on average (SD=8.4) and reported mild-to-moderate nicotine dependence (MFTND=4.0 [SD=2.3]). Biochemical assessment and surveys were conducted at enrollment, e-cigarette initiation (1 week later), and 1-week, 1-month, and 3-month post e-cigarette initiation.
ResultsA subset (17&#¿;%) of participants did not achieve the minimum 75&#¿;% EMA adherence at Week 1. The remaining participants showed excellent adherence with the EMA protocol, with a consistently high response rate to audibly prompted mini-surveys during screening (93&#¿;%) and follow-up weeks (93&#¿;%, 93&#¿;%, and 92&#¿;%, respectively). No consistent trend emerged in comparing the two randomized groups, either on variables relevant to the public health impact of e-cigarette initiation (i.e., cigarettes/day, exposure to smoke, exposure to nicotine, motivation to quit smoking), or in e-cigarette use (i.e., bouts/day, puffs/bout), with average Cohen's d values across timepoints ranging from d&#¿;=&#¿;0.02 for exposure to smoke to d&#¿;=&#¿;0.06 for cigarettes per day.
ConclusionsConducting EMA with adults who smoke daily and are initiating e-cigarette use appears feasible, with some caveats. Reactivity to EMA during e-cigarette initiation appears to be small or non-significant.
Electronic cigarettes (e-cigarettes) are nicotine delivery devices that have demonstrated remarkable uptake worldwide in recent years. The WHO, FDA, as well as the United States Surgeon General have made urgent calls for more research on e-cigarettes.1-3 Despite concerns about health risks, from January 2017 to March 2022, the total monthly e-cigarette unit sales increased by 293.6&#¿;% to 23.3 million units and sales of products containing ≥5&#¿;% nicotine strength increased by 1486.3&#¿;%, representing 80.9&#¿;% of all e-cigarette sales.4 Ecological momentary assessment (EMA)5 is a data capture technique that samples experiences and behaviors as they occur in real life. Thus, this methodology can provide critical insight into how people who smoke use e-cigarettes.
One worry in using EMA, however, has always been the potential of reactivity to frequent assessments, that is, the possibility that answering EMA questions will cause meaningful – and potentially negative – changes in behaviors, thoughts or feelings. In some instances, EMA reactivity is the goal of repeatedly asking about a behavior and its context. For instance, in EMA-based clinical interventions, behavioral or psychological experience monitoring or prompts to engage in behaviors based on EMA data are meant to change behavior.6 The goal would be for a patient to develop insight into their behavior based on repeated measurement of that behavior and its context, thereby increasing their ability to change that behavior. Studies to date, however, have typically shown non-significant effects of EMA on targeted behavioral outcomes, and only modest effects on related psychosocial constructs.7,8 These results suggest that measuring behaviors with EMA does not systematically change established behaviors under investigation.
Behavior change could be one process whereby EMA shows meaningful influence on behavior and specifically on smoking. During behavior change, the behavior of interest and related constructs are in flux, and thus are potentially more vulnerable to outside influences such as EMA. Three studies to date have addressed EMA reactivity in the context of behavior change by testing the impact of EMA on people who smoke undergoing a quit attempt.8-10 Compared to no EMA reports, Rowan and colleagues9 found that one week of EMA reporting prior to planned quit dates had no effect on smoking abstinence rates 7 and 28 days later (N&#¿;=&#¿;96). Group differences did exist, however, on 7-day and 28-day outcomes of the 20 assessed subscales of self-efficacy and withdrawal, but patterns were inconsistent. To estimate the impact of the duration of EMA reporting, McCarthy and colleagues randomized participants (N&#¿;=&#¿;70) to complete EMA reports during 4 versus 7 weeks leading up to a quit attempt as well as during 3 weeks following the quit attempts, and found no group differences in withdrawal symptoms during the 6 overlapping weeks.10 To estimate the impact of EMA intensity, McCarthy and colleagues8 randomized participants (N&#¿;=&#¿;110) to complete one versus six EMA reports per day over a period of four weeks, with EMA assessments beginning one week prior to smoking cessation, and ending three weeks following smoking cessation. They found no impact of EMA on smoking cessation or prolonged abstinence over a period of 10 weeks. This study did, however, show lower craving, anxiety, anger, hunger, and positive affect in participants randomized to the high-frequency EMA condition. Taken together, these studies provide evidence that EMA reporting is unlikely to impact conventional smoking behavior outcomes, even during a period of behavior change, but that it may impact constructs relevant to smoking cessation.
Three studies to date have reported EMA data on e-cigarette use, to our knowledge.11-13 Though these studies help understand the context of dual use of cigarette and e-cigarette use among established dual users, no studies to our knowledge to date have examined e-cigarette initiation, nor have they reported on potential EMA reactivity. In addition, both Huh and Camenga focused on dual cigarette and e-cigarette use in young adults,11,12 who may have different e-cigarette habits relative to established adults who smoke. In addition, these three studies11-13 collected EMA data for 7 to 14 consecutive days, which does not capture whether e-cigarette use, cigarette use, and their context change over longer periods of time.
Given the demonstrated usefulness of EMA in smoking research,14,15 and increasing calls for research on e-cigarettes,1-3 we extended previous work and conducted a pilot study to test the feasibility of using EMA to capture e-cigarette initiation over a three-month time course using a burst-of-measurement approach16 and examine EMA reactivity in e-cigarette initiation. Though a recent meta-analysis of EMA studies assessing substance use (including nicotine) with people who use substances did not find a different in adherence rates with the length of assessment period,17 completing EMAs reports for three consecutive months is likely burdensome to participants, costly to researchers, and likely to result in lower adherence over time. Understanding if this burst-of-measurement approach, with five individual weeks of assessments spread over the course of three total months, is feasible is important to understand contextual influences on e-cigarette use over longer periods of time.
MethodParticipantsParticipants were people who smoke and who were interested in trying out e-cigarettes. Eligibility criteria were: (a) 18+ years of age, (b) currently smoking daily for at least the last 3 months with 100+ lifetime cigarettes smoked, (c) currently not using non-cigarette tobacco (e.g., cigars, chewing tobacco, pipe), (d) currently not using any smoking cessation pharmacotherapy (though use of non-pharmacological smoking cessation aids was not exclusionary), (e) interested in trying out e-cigarettes, with no previous e-cigarette experience, (f) willing to initiate e-cigarette use during the study, (g) not pregnant, planning to become pregnant, or breast feeding, and (h) fluent in English.
ProceduresStudy participation lasted 3.5 months, and consisted of five lab visits and, for participants randomized to the EMA condition, 4 weeks of EMA (see Fig. 1). At each lab visit, participants completed biochemical tests and questionnaires. All procedures were approved by the first author's affiliated institutional review board.
Study timeline for ecological momentary assessment protocol
Notes. Timeline of study participation showing ‘burst of measurement’ ecological momentary assessment (EMA) approach in which EMA was delivered in distinct, 1-week epochs immediately prior to e- cigarette initiation, immediately following e-cigarette initiation, and 1 month and 3 months following e- cigarette initiation.
After phone screening, interested individuals came to the lab to provide written informed consent, and to biochemically verify smoking status. If expired breath carbon monoxide was < 6&#¿;ppm,18,19 a dipstick urine test (NicAlert) was used to confirm smoking status. Participants were then randomized to either complete EMA or not on a 1:1 schedule with 4 participants per block using SAS PROC PLAN syntax.20 EMA participants were given a handheld device (i.e., Hewlett Packard iPAQ 110) to complete EMA reports, and were asked to practice using the device for 1–2 days. Data collection started upon confirming by phone that participants were comfortable with the EMA reporting. EMA participants had to achieve a score of 75&#¿;% or higher on the study's EMA adherence summary score, which consisted of completing morning reports (yes/no per day; 20&#¿;% of the EMA score), evening reports (yes/no per day; 20&#¿;% of the EMA score), and audibly-prompted mini-surveys (percent of audibly prompted reports participants completed per day; 60&#¿;% of score), while not exceeding allotted time for ‘sleep’ (–10&#¿;% of the EMA score) or ‘suspend’ (–10&#¿;% of the EMA score) modes or failing to initiate at least one report of smoking or vaping (–30&#¿;% of the EMA score), unless participants reported being abstinent from either smoking or e-cigarette use. Audibly-prompted reports occurred up to six times per day, randomly occurring between 0:00–4:00, 4:00–8:00, 8:00–12:00, 12:00–16:00, 16:00–20:00, and 20:00–24:00 when the device was not in ‘sleep’ or ‘suspend’. At the end of each week, participants data were downloaded and their adherence was calculated with an excel file. A research assistant then gave the participant feedback on the types of prompts (morning report, evening report, random prompts, cigarette reports, e-cigarette reports, sleep time, suspend time) that decreased their adherence and encouraged them to increase their adherence. Participants were required to pass the EMA adherence check at the end of their first week of the study because the EMA protocol was the independent variable in the study.
Participants returned to the lab one week later (Visit 2 – e-cigarette initiation) and received a 2nd generation e-cigarette starter kit with the flavor of their choice (i.e., Classic Tobacco, Magnificent Menthol, or Variety Pack). Female participants needed to produce a negative urine pregnancy test to receive the e-cigarette starter kit. EMA participants were asked to continue to complete EMA reporting on cigarette use and e-cigarette use for the following week.
All participants returned to the lab for 1-week, 1-month and 3-month follow-ups post e-cigarette initiation. EMA participants completed EMA reporting during the week immediately preceding lab visits. For the 1- and 3-month follow-ups, staff called EMA participants to remind them to turn on their EMA devices.
Participants were compensated for their participation, with escalating compensation per lab visit (i.e., $10 for enrollment, $35 for 3-month), and EMA reporting (i.e., $40 for the baseline week, and $55 for the 3-month, directly proportional to EMA adherence), with a total possible compensation of $120 for non-EMA and $310 for EMA participants. Participants who decided to leave the study early were asked to complete an early termination questionnaire by phone, email, or in person.
MeasuresBiochemical assessmentsExpired breath carbon monoxide was assessed with a Bedfont piCO+ Smokerlyzer. Cotinine was assessed via saliva samples collected with cotton dental rolls (SalivaBio OralSwabs) placed inside the cheek. Samples were frozen on site and shipped to a laboratory (Salimetrics LLC) for analysis using duplicate enzyme immune assay.
EMA itemsShort, logic-branched, multiple-choice and slider-bar items were used. Audibly prompted mini-surveys (21 items) assessed current mood (6 items), other subjective states (e.g., hunger; 4 items), craving (2 items),21 environmental (e.g., inside/outside), social (e.g., alone/with others; with children), and physiological (i.e., caffeinated or intoxicating substance) context (9 items). The same items were used in user-initiated reports of cigarette and e-cigarette consumption, which also assessed mood and other subjective states retrospectively (i.e., right now vs. right before smoking or vaping), reason for this instance of smoking / vaping (1 item), number of cigarettes smoked/ number of e-cigarette puffs taken (1 item), and, for e-cigarettes only, nicotine strength used (1 item).
QuestionnairesQuestionnaires were administered via REDCap (Research Electronic Data Capture).22 Logic-branching was used to reduce participant burden. Surveys addressed several constructs. Of relevance here are the assessment of motivation to quit smoking, as measured on a 0–10 Likert scale ranging from “Stopping smoking is not at all important to me” (0) to “stopping smoking is the most important goal in my life” (10),23 past week e-cigarette and cigarette use, and nicotine dependence as assessed by the Fagerström Test for Nicotine Dependence (FTND).24
Data analytic planFirst, t-tests (for continuous variables) and chi-square tests (for categorical variables) examined whether there were differences between EMA and non-EMA groups at baseline with respect to demographic and smoking history variables. Then, to summarize adherence to the EMA protocol, we calculated percentages describing EMA behaviors. These analyses used data from the EMA group only, as the non-EMA group did not have EMA data. We calculated these summaries separately per EMA week, as each week represented a new burst of measurement, and thus it was of interest to see if compliance would differ across different assessment weeks. For each person, we calculated the percentage of time they met the EMA compliance criterion (e.g.,% of days they completed the morning report;% of times they responded to an alert to complete a random mini-assessment), and then averaged across participants to provide the overall summary percentage.
To examine if there were differences between randomized groups in our key variables of interest that might be impacted due to EMA reactivity, we used a descriptive approach. Our sample size was quite low, meaning a non-significant p-value would be uninformative. Instead, we calculated means with standard deviations, and based on these, calculated Cohen's d to provide insight into the magnitude of group differences. Because our sample size was low, we then also calculated 95&#¿;% confidence intervals around this effect size, as recommended,25 using the formula provided by Hedges and Olkin.26
ResultsParticipants were predominantly male (61.0&#¿;%), White (54.3&#¿;%) or African American (38.9&#¿;%) and were an average 44.1 (SD&#¿;=&#¿;12.6) years of age. The sample was socio-economically disadvantaged. Specifically, few participants were employed full-time (17.0&#¿;%) or had a college degree or more (23.7&#¿;%); most had an annual household income of under $25,000. At baseline, participants smoked 13.0 (SD&#¿;=&#¿;8.4) cigarettes per day on average and reported mild to moderate nicotine dependence (MFTND&#¿;=&#¿;4.0; SD&#¿;=&#¿;2.3). Participants randomly assigned to EMA and non-EMA groups did not significantly differ on any demographic or smoking characteristics at baseline (see Table 1).
Sample characteristics by EMA vs Non-EMA group.
Notes. FTND= Fagerstrom Test for Nicotine Dependence; † assessed at e-cigarette initiation visit rather than enrollment visit; ppm= parts per million; ng/ml= nanograms per milliliter; all p>.05.
Requests for study enrollment exceeded staff capacity; we completed 20 successful enrollments per month. A subset (17&#¿;%; n&#¿;=&#¿;5) of participants randomized to the EMA group failed the EMA adherence check in the first week; and an additional 10&#¿;% (n&#¿;=&#¿;3) chose to discontinue the study for other reasons. The EMA protocol was therefore not feasible for approximately a third of participants randomly assigned to EMA.
For participants in the EMA group who passed the initial adherence check, EMA adherence was consistently high during the three subsequent EMA assessment bursts (Table 2), with response rates to audibly-prompted mini-surveys, and morning and evening reports exceeding 90&#¿;%. When not abstinent from smoking, participants reported at least one cigarette and/or e-cigarette use on average on 84–92&#¿;% of days. The sleep function was somewhat problematic, with participants blocking off more than the allotted 10 h of ‘sleep’ per day in the screening week, though adherence to the 10 h of sleep expectation improved each week as participants received feedback and decreased compensation for putting their device in sleep mode for more than 10 h. The suspend function was used as intended. Retention (Fig. 2) for this burst-of-measurement EMA study was acceptable for the EMA group that passed the adherence check at Week 1 (82&#¿;%, 77&#¿;%, and 68&#¿;% for 1-week, 1-month and 3-month follow-up), but not for the non-EMA group (75&#¿;%, 64&#¿;%, and 53&#¿;% for 1-week, 1-month and 3-month follow-up). Exit interviews indicated that compensation was not sufficient to offset time required to come in for office visits, which was particularly true for non-EMA participants.
EMA adherence rates for persons who failed the screening test and those who completed the screening test across assessment periods.
Notes. EMA= Ecological momentary assessment.
Consort diagram showing participant enrollment and attrition for ecological momentary assessment group, and no ecological momentary assessment group.
Note. Of the n&#¿;=&#¿;5 who were excluded between consent and randomization, one was excluded at baseline because we were unable to confirm their status as a current daily smoker at baseline (i.e., their CO reading was 4&#¿;ppm). The remaining four consented but were not randomized, as the choice to randomize participants to complete EMA versus not complete EMA was not reflected in the original study protocol (the purpose of the study was to examine the feasibility of using EMA to characterize e-cigarette initiation among adults who smoke daily in one arm only). An IRB amendment approving randomization was approved after these four participants had consented to participate. Thus, they completed the study but are not reflected here because they did not agree to be randomized.
Comparisons in cigarette and e-cigarette outcomes between the EMA and non-EMA groups were not suggestive of EMA reactivity (Table 3). No consistent trend emerged in comparing the EMA vs non-EMA groups, either on variables relevant to the public health impact of e-cigarette initiation (i.e., cigarettes per day, exposure to smoke, exposure to nicotine, motivation to quit smoking), or in terms of the newly forming behavior, e-cigarette use (as measured in number of bouts of e-cigarette use per day and puffs per bout of e-cigarette use). Due to the limited sample size of this study, confidence intervals around group difference estimates (i.e., Cohen's d) were large, but means across time periods suggest that systematic group differences were either small or negligible.
Cigarette and e-cigarette smoking outcomes for EMA vs non-EMA groups.
Notes. EMA&#¿;=&#¿;ecological momentary assessment. CI&#¿;=&#¿;Confidence Interval. Standard deviations in parentheses; CO&#¿;=&#¿;carbon monoxide, ppm= parts per million; ng/ml= nanograms per milliliter; * p<.05.
The biggest group difference was small in size (Cohen's d&#¿;=&#¿;0.29, CI&#¿;=&#¿;0.01 − 0.59) and was in exposure to nicotine measured via salivary cotinine assessed during Visit 2, after 1 week of EMA reporting, but immediately prior to e-cigarette initiation. Specifically, EMA participants showed higher salivary cotinine (M&#¿;=&#¿;499.2, SD&#¿;=&#¿;315.2) than non-EMA participants (M&#¿;=&#¿;334.3, SD&#¿;=&#¿;240.1) at Visit 2. In Visit 3, one week later, the pattern was reversed with non-EMA participants (M&#¿;=&#¿;429.3, SD&#¿;=&#¿;412.3) showed higher salivary cotinine than EMA participants (M&#¿;=&#¿;332.2, SD&#¿;=&#¿;210.7; Cohen's d&#¿;=&#¿;−0.15, CI&#¿;=&#¿;−0.46 – 0.22). EMA and non-EMA groups did not significantly differ in salivary cotinine at Visit 4, one month after e-cigarette initiation, (Cohen's d&#¿;=&#¿;−0.02, CI&#¿;=&#¿;−0.35 – 0.38) or at Visit 5, three months after e-cigarette initiation (Cohen's d&#¿;=&#¿;0.05, CI&#¿;=&#¿;−0.31 – 0.41).
The only other difference that was small in size (rather than negligible) was motivation to quit smoking, which was higher in the EMA group (M&#¿;=&#¿;8.6, SD&#¿;=&#¿;1.7) relative to the no-EMA group (M&#¿;=&#¿;7.7, SD&#¿;=&#¿;2.4) at Visit 5 (Month 3) only and was not statistically different (Cohen's d CI&#¿;=&#¿;−0.17 – 0.64).
DiscussionOur findings suggest that EMA reactivity within the context of e-cigarette initiation is either small or negligible. The largest group difference we observed was on salivary cotinine assessed during Visit 2, after 1 week of EMA reporting, but before e-cigarette initiation, where EMA participants had higher cotinine than non-EMA participants. Subsequently, however, the directionality reversed, with non-EMA participants having higher cotinine values. The inconsistency of these group differences suggests random fluctuations around largely similar means. Given the limited sample size of our study, we cannot, of course, rule out the existence of small effects. Our results do suggest, however, that medium to large effects due to engaging in EMA are unlikely to occur on variables relevant to measuring the public health impact of e-cigarette initiation, or on variables summarizing e-cigarette use. This constellation of findings is concordant with previous research that utilized EMA to study substance use,7,9,27 and implies that the act of engaging in EMA does not contaminate the behaviors under assessment in the context of e-cigarette initiation.
Results of our pilot study also support the feasibility of using an EMA burst-of-measurement approach to capture e-cigarette initiation, with some caveats. First, there was a subset of participants – almost a third of participants who were randomized to complete EMA – who did not complete the EMA protocol, either because they were withdrawn by the study staff for having low adherence during the screening week (17&#¿;%) or the participants electively withdrew (10&#¿;%). The screening and consent process may need to be even more explicit and include examples of what EMA prompts sound like and how often they may occur. More work is needed to understand the reasons for low adherence among this subset of participants. For the group who remained in the study, generally, response rates to EMA mini-surveys were consistently high across EMA periods, suggesting that participants are unlikely to succumb to assessment fatigue when needing to participate in multiple EMA periods. Retention rates were also acceptably high in the EMA group, though notably not in the no-EMA group, who received less compensation. There was compensation escalation at each visit; however, the $10 for enrollment was deemed unacceptably low and $35 for Month 3 was not escalated enough to achieve desired retention in the non-EMA group. To maximize retention rates, our results indicate that compensation rates needs to be higher or could be offered entirely remotely to offset travel costs and time. We further recommend against the use of user-initiated sleep and suspend functions, as these features appeared to complicate rather than enhance the end-user experience.
Study limitations include the limited sample size, and a largely low socioeconomic status sample of adults recruited in an urban academic medical center. This sample, therefore, may not represent the breadth of e-cigarette users. This study also used iPAQs given the low socioeconomic status of participants, which may have resulted in lower adherence than app-based EMA monitoring. Results presented here may not be an accurate reflection of future EMA adherence rates with app-based monitoring on personal cell phones or study cell phones. In addition, though e-cigarette use was self-reported for both EMA and non-EMA participants, these may be underestimates, and could be enhanced with blue-tooth enabled e-cigarettes to more accurately measure bouts.28 Finally, there was a significant difference in smoking experience between randomized groups (i.e., longer smoking careers in those assigned to the EMA group), but given a lack of significant difference in age and other smoking characteristics, it is unclear if this difference could have meaningfully impacted results.
ConclusionsResults of this study have some preliminary clinical implications. Though not currently approved as medical devices in the United States29 or any other country to the authors knowledge, the United Kingdom has provided guidance for how e-cigarettes and other nicotine delivery systems could apply for consideration as medical devices.30 In addition, a recent review has shown using e-cigarettes with nicotine results in higher quit rates than NRT or non-nicotine e-cigarettes31 and persons may prefer e-cigarettes given their similarity in inhalation to traditional cigarettes, as well as their customizations for nicotine level and flavor.32 Results of this study suggest that asking repeated questions about cigarette use, e-cigarette use, and their context will not necessarily increase either behavior or exposure to nicotine levels – meaning that providers who are asking their patients to monitor their e-cigarette use during an attempt to quit traditional cigarettes is unlikely to worsen smoking or quit outcomes. Simply monitoring traditional smoking and e-cigarette use may not change either behavior, and so enhancing motivation quit outcomes with behavioral therapies33 may be needed for persons wishing to replace traditional cigarettes with e-cigarettes. Though motivation to quit smoking was higher in the EMA group three months after e-cigarette initiation, this effect was small and would need to be replicated in additional studies to encourage providers to assign smoking and e-cigarette monitoring in order to increase future motivation to quit smoking.
Results from the present investigation support the feasibility of EMA for the study of e-cigarette initiation and use, within the subset of participants who passed the EMA adherence test within the first week – that is, they completed at least 75&#¿;% of the EMA protocol in their first week. This suggests that the participants who are adherent to the EMA protocol do not become over-burdened or have other negative reactions to the EMA protocol. For this study, the independent variable was EMA reporting, and so participants were only included if they could complete the EMA reporting. However, for future EMA studies recruiting people who smoke daily initiating e-cigarette use, more incentives, additional training on the EMA reporting protocol particularly around user-initiated sleep mode, or additional information at screening or consent on the burden of EMA procedures may be needed, in order to prevent recruiting participants who are not able to complete the EMA protocol. Traditional concerns associated with the EMA methodology such as participant reactivity, did not appear to be active in people who smoke who are initiating e-cigarette use. In addition, this study tested a novel burst-of-measurement approach, which proved to be effective for capturing behavior under investigation across a several brief (7 day) time-periods over the course of three months, without dramatically increasing participant burden.