- © 2007 by American Society of Clinical Oncology
Long-Term Toxicity Monitoring via Electronic Patient-Reported Outcomes in Patients Receiving Chemotherapy
- Ethan Basch,
- Alexia Iasonos,
- Allison Barz,
- Ann Culkin,
- Mark G. Kris,
- David Artz,
- Paul Fearn,
- John Speakman,
- Rena Farquhar,
- Howard I. Scher,
- Mary McCabe and
- Deborah Schrag
- From the Departments of Epidemiology and Biostatistics, Medicine, Nursing, and Information Services, and the Office of the Physician-in-Chief, Memorial Sloan-Kettering Cancer Center, New York, NY; and the National Cancer Institute Center for Bioinformatics, Rockville, MD
- Address reprint requests to Ethan Basch, MD, MSc, Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, 307 East 63rd St, New York, NY 10021; e-mail: basche{at}mskcc.org
Abstract
Purpose In cancer treatment trials, clinicians traditionally report patient toxicity symptoms. Alternatively, patients could provide this information directly.
Patients and Methods The Common Terminology Criteria for Adverse Events (CTCAE) is the mandated instrument for tracking patient toxicity symptoms in National Cancer Institute (NCI)-sponsored cancer treatment trials. We adapted CTCAE symptom items into patient language and uploaded these to an online platform. Lung cancer outpatients receiving chemotherapy were invited to self-report selected symptoms at visits via waiting area computers or optional home access. Symptom reports were printed for nurses at visits, but no instructions were given with regard to use of this information.
Results From June 2005 through March 2006, 125 patients were invited to participate, and 107 chose to enroll. Mean length of participation was 42 weeks (range, 1 to 71 weeks), by which time 35% died. The average number of clinic visits was 12 (range, 1 to 40 visits). At each consecutive visit, most patients (mean, 78%) logged in without significant attrition. Reasons for failure to log in included having no reminder and having inadequate time. Although 76% of enrollees had home computers, only 15% self-reported from home. Satisfaction with the system was high (90%), but only 51% felt communication was improved. All participating nurses understood the reports and felt this information was useful for clinical decisions, documentation, and discussions. However, only one of seven nurses discussed reports with patients frequently, with insufficient time being the most common barrier to discussions.
Conclusion Online patient self-reporting is a feasible long-term strategy for toxicity symptom monitoring during chemotherapy, even among patients with advanced cancer and high symptom burdens. However, without explicit reminders and clinician feedback, patients demonstrated limited voluntary interest in self-reporting between visits.
INTRODUCTION
There is increasing interest among regulatory agencies, industry sponsors, and clinical investigators in the use of patient-reported outcomes (PROs) in cancer treatment trials and routine care.1-3 Toxicity symptom monitoring has been identified specifically as an area in which patient self-reporting might improve the quality and completeness of collected data, might provide a more comprehensive picture of the patient experience, and might improve the efficiency of clinical operations.4-7 This information can also be used to inform clinical interventions, with the potential to alleviate symptoms and address toxicities early.
Patient reporting has become a common method of data collection in nononcology trials and has served as the basis for labeling claims of US Food and Drug Administration–approved drugs with primary symptom end points.8,9 PROs are also the preferred method for collecting health-related quality of life (HRQOL) data.10-12 However, clinician-based reports rather than PROs remain the standard approach for symptom toxicity monitoring in oncology.13,14
Recently, the US Food and Drug Administration (FDA) has specified standards for the development and interpretation of PRO instruments in registration-track trials.15 Representatives of the US FDA and the National Cancer Institute (NCI) have publicly stated that, for subjective domains including symptoms and HRQOL, the patient's own account, rather than a clinician's interpretation, should be considered the gold standard.1,15-18
Several factors coincide to make evaluation of patient self-reporting of symptoms and toxicity in the context of chemotherapy treatment a timely area for investigation. First is the receptiveness of the US FDA and the NCI to consider this information as part of the drug approval process. Second is the increasing familiarity of patients with Web-based interfaces and the relative ease involved in designing and maintaining such systems to systematically track the patient experience.
To explore the use of a computer interface for eliciting cancer patients' symptoms, we previously developed a patient-language adaptation of the NCI's Common Terminology Criteria for Adverse events (CTCAE), which is the mandated instrument for clinician-based toxicity reporting in NCI-sponsored cancer treatment trials.19 In pilot research, we found that women with gynecologic malignancies were able to self-report symptoms online by using a patient version of the CTCAE during an 8-week period.20 However, this experience did not extend to men or to patients with other cancers, nor did it determine whether reporting over prolonged periods is sustainable among patients with high symptom burdens while receiving treatment. Therefore, we designed a study to assess the extent to which lung cancer patients receiving cytotoxic chemotherapy are willing and able to electronically self-report symptoms at clinic visits over an extended period.
PATIENTS AND METHODS
Toxicity Symptom Questionnaire
A patient-language adaptation of CTCAE symptom items was previously developed through focus groups, cognitive interviewing, and comparisons of patient versus clinician responses.20-22 A lung cancer–specific questionnaire was developed that included seven core items (anorexia, constipation, diarrhea, fatigue, nausea, pain, vomiting), identified based on salience for patients receiving chemotherapy regardless of cancer type, and two disease-specific items (cough, dyspnea). Item selection was based on a paper survey of 40 patients, a literature review, and expert consultation. The goal was to identify a parsimonious set of symptoms so as to maximize the response while minimizing the patient burden. The questionnaire also included one item to measure the Karnofsky Performance Status23 and five items, comprising the EuroQoL EQ-5D questionnaire, to measure HRQOL.24
Online Platform
The questionnaire was uploaded to an online portal called STAR (Symptom Tracking and Reporting), which allows patients to securely log in, enter information, and view reports of previously entered data. User acceptance testing in 20 patients was conducted to ensure ease of navigation and content comprehension. STAR also generated clinician reports that highlighted severe (grade 3) or disabling (grade 4) symptoms. The technology configuration was approved by institutional security and privacy officers and by the institutional review board at Memorial Sloan-Kettering (New York, NY). Computers were set up in clinic waiting areas.
Enrollment
This feasibility study sought to enroll 100 patients, diagnosed with thoracic malignancies and starting new chemotherapy regimens, who were not enrolled in a clinical treatment protocol and were able to read and understand English. Potentially eligible patients were identified by reviewing a list of scheduled visits. All participants signed informed consent and research authorization forms. Patients remained enrolled until death, voluntary disenrollment, transfer of care, or study completion. We selected a 16-month observation period on the basis of a projected 50% mortality rate by this time point, as estimated from prior patient care data.
The sample size was based on the available resources for conducting this study. On the basis of prior research in this area, participants were considered adherent if they completed online questionnaires at more than 60% of follow-up visits during the study.20 The feasibility threshold for this study was set so that, if 80% of participants were adherent, this strategy of data collection would be considered worth pursuing further, and alternative means would be employed for capturing symptom information from those unwilling or unable to self-report online. With a sample size of 100, the power to detect a difference between 0.8 and 0.9 in the proportion of patients completing questionnaires with an alpha of 5% (one-sided test for binomial proportions) was 87%, and that required to detect a difference between 0.7 and 0.8 was 73%. Given a type I error of 10%, the power was 85% between 0.7 and 0.8.
Study Intervention
All participants underwent a 10-minute, computer-teaching session at enrollment. At subsequent clinic visits, patients were reminded of the opportunity to log in to STAR, but such logins were not mandatory. STAR was also available via the internet outside the hospital, but there were no reminders for home log-in sessions between visits. At each follow-up visit, a STAR report was added to the printed chart provided to clinical nurses. Whether to and how to incorporate a discussion of STAR reports into visits was left to patient and clinician discretion.
Patients were informed that no clinician would regularly monitor STAR reports between visits, so patients should use traditional means of contacting staff (eg, telephone) if severe or disabling symptoms occurred at home. Nonetheless, an automated alert system was integrated into STAR so that, when a patient entered a symptom grade ≥ 3, an on-screen message encouraged contacting a clinician and an e-mail warning was automatically sent to a designated nurse.
Outcomes
Feasibility.
As an assessment of patient willingness to self-report toxicity symptoms, the proportion of approached patients who consented to participate and reasons given for nonparticipation were recorded. For those who enrolled, we tracked the total number of attended clinic visits during the study period, the frequency of completed online sessions at visits (adherence), and the specific reasons for failure to complete online sessions. A completed online session was defined as an event in which a patient logged in to the system and pressed the on-screen submit button at the end of the questionnaire. For the calculation of adherence, an online session was considered to correspond to a specific visit if the session was submitted from a clinic-based computer on the day of the visit or from a home computer within 24 hours of the visit. A time-specific adherence rate was calculated by summing the number of completed online sessions by all patients during each month of enrollment and dividing that sum by the number of visits that month.
To assess for relationships between adherence and specific patient characteristics, a nonlinear mixed model with binary response was used. Binary covariates were entered as fixed effects in the model. Univariate and multivariate models were considered. Month on the study, as a continuous covariate, was assessed as a fixed effect to determine whether the duration of participation affected adherence rates. A random intercept to account for a unique baseline compliance rate for each patient was added in the model as a random effect. A sensitivity analysis was performed using General Estimating Equations (GEE). The estimates from the GEE model were used as initial parameters for the nonlinear mixed model.
Patient satisfaction survey.
To assess patient satisfaction with online CTCAE self-reporting, a paper questionnaire was administered after 8 weeks of participation, with items adapted from validated measures used in similar research.25
Nursing survey.
Following study completion, an anonymous survey was administered to participating nurses to assess their understanding of the patient-reported information; whether they found it valuable towards clinical decisions, documentation, or discussions with patients; and the possibility of specific barriers to reviewing this information with patients.
RESULTS
Enrollment
From June 2005 through March 2006, 130 consecutive lung cancer patients were approached; 125 were eligible, and 107 consented to participate (Fig 1). Eighteen (14%) refused because they saw no benefit, felt too distressed, or were unwilling to use a computer.
Patient accrual and status at study completion. MSK, Memorial Sloan-Kettering Cancer Center.
Patients
Baseline characteristics are presented in Table 1. Most enrollees were between the ages of 50 to 69 years, although 28% were older than 70 years. Most (69%) were diagnosed with metastatic non–small-cell lung cancer that had predominantly good baseline performance status. The majority of patients received cyctotoxic chemotherapy on an every-3-weeks schedule. In a baseline paper survey, most enrollees (76%) reported having a computer at home, although only 47% reported regular internet use. Although 43% held a college or graduate degree, 38% were not educated beyond high school.
Patient Characteristics
Visits and Online Sessions
Mean length of enrollment was 42 weeks (median, 47; range 1 to 71 weeks). The mean number of clinic visits was 12 (range, 1 to 40 visits), and the mean number of completed online sessions was 11 (range, 1 to 144 sessions). The mean duration of online sessions was 4.5 minutes (median, 3.5; range, 0.5 to 20 minutes).
Adherence
Figure 2 shows the proportion of patients who completed an online session at each consecutive visit. Although there was some fall-off after the first visit, adherence rates remained high at all subsequent visits (up to visit number 40), with an overall mean adherence rate of 78%. To determine if there was attrition of online activity over time, we analyzed the proportion of patients completing sessions during each month of their follow-ups (Fig 3). Fall-off was observed between the first and second months because of greater adherence at the initial visit compared with subsequent visits. After the first month, the proportion remained relatively stable (between 70% and 80%) for any given month, which reflected a lack of significant attrition overall.
Proportion of patients completing online questionnaires at a given clinic visit (does not show clinic visits attended by fewer than 10 patients). (*) During the study period, there were up to 40 return visits, and the mean proportion of patients completing questionnaires at follow-up visits 25 through 40 was 78%.
Proportion of patients who completed online questionnaires at clinic visits attended (*) during each month of follow-up (defined for each patient from the time of enrollment).
Most patients completed online sessions at the majority of visits: 84% logged in at more than 60% of visits, and 57% logged in at more than 80% of visits. Therefore, this study met the predetermined feasibility threshold to consider this data-collection strategy worthy of further evaluation (ie, ≥ 80% of participants logged in at > 60% of visits).
Logistic Barriers to Adherence
Seventy patients failed to log in during at least one visit. The most common reason given, which accounted for 32% of all misses, was that the clinic staff failed to remind the patient to use the system. Other reasons included a lack of adequate time before the physician visit (20%) and treatment or schedule changes that altered the duration of the appointment (15%). Few refusals were due to technical problems (5%), and few resulted from patients feeling too sick (5%), feeling too distressed (3%), or perceiving no benefit (3%).
Patient Characteristics Associated With Adherence
Prior computer experience was associated significantly with greater adherence (P = .017). No significant relationship was observed between adherence and age, sex, level of education, disease type, disease stage, therapy type, or baseline performance status. Although the duration of enrollment was significantly associated with adherence (P = .0006), this relationship was attributable to the fall-off in adherence between the first and second visits, and no significant difference in adherence between visits was observed after that time (P = .42).
Home Reporting
In contrast to the high adherence rate for self-reporting at office visits, only 16 enrollees (15%) actively self-reported from home between visits. Home users did access STAR more frequently than clinic-only users, with a mean of 23 online sessions (range, three to 144 sessions) versus nine (range, one to 36 sessions), respectively.
Toxicity Alerts
During the study, 179 alerts were triggered by severe (grade 3) or disabling (grade 4) toxicity symptoms, as reported by 57 different patients. The most commonly reported severe or disabling symptoms were pain (73 grade 3; 10 grade 4), shortness of breath (65 grade 3; 17 grade 4), and anorexia (32 grade 3; no grade 4). Severe or disabling nausea, vomiting, diarrhea, and constipation were less common (< 10 each). These toxicities were distributed across the 57 patients, with 24 reporting only one severe or disabling symptom, 22 reporting two to four, and 11 reporting more than five.
Patient Satisfaction Survey
As presented in Table 2, most respondents found STAR easy to use (98%) and useful (90%). Although most (77%) felt it improved the quality of discussions with clinicians, only 51% thought communication was improved. Nonetheless, the majority wished to continue using STAR (95%) and would recommend it to others (90%).
Patient Impressions of the STAR Online System (N = 63)
Nursing Survey
All nurses stated that they understood the content of the STAR reports and felt the information was useful for clinical decisions, documentation, and discussions with patients. All noted that they had altered management because of patient-reported information in STAR; management alterations included medication changes (made by five of seven nurses), lifestyle modification recommendations (five of seven nurses), and new physician consultations arranged (two of seven nurses). However, when asked how often such management changes were made, only one nurse reported frequently, whereas four reported changes occasionally and two reported them rarely. The most commonly cited barrier to acting on or discussing patient-reported information was insufficient time.
DISCUSSION
Our results demonstrate that online patient self-reporting is a feasible long-term strategy for symptom monitoring in most lung cancer patients receiving chemotherapy, including in those with end-stage disease and high symptom burdens. This finding lays the foundation for future research to explore strategies that maximize patient response rates; to assess whether patient reporting improves the efficiency and comprehensiveness of toxicity monitoring in clinical trials; to measure if this approach improves the quality of routine cancer care by expediting detection of severe or disabling toxicities; and to refine the technology platform and questionnaire design.
In a prior study in women with gynecologic malignancies, we found that most patients voluntarily self-reported symptoms online at appointments during an 8-week observation period (up to five clinic visits).20 We have now built on these results by following patients for up to 16 months and 40 visits. Unlike the prior study, this investigation also included men, and no significant associations were seen between levels of adherence and sex. Results of the current study are particularly promising towards the broader use of patient self-reporting in oncology, because the lung cancer population is generally more ill, lower in socioeconomic status, and more apt to drop out of studies than other groups because of disease progression.
The most common barrier to participation in the clinic was a lack of a personalized reminder to log in. The low observed rate of home participation (15%) may similarly have resulted from a lack of regular reminders and the perception among participants that self-reported information was not being explicitly addressed by clinicians. Indeed, we did not require that STAR reports be discussed at visits, and only 51% of patients felt STAR enhanced communication with clinicians (despite 90% overall satisfaction). This hypothesis is supported by the nursing survey results. Although nurses valued the content of STAR reports and used them as the basis for management decisions or discussions, only one of seven did so frequently. The most common barrier to more frequent action or discussion was insufficient time.
These observations underline the importance of systematically involving staff in the patient-reporting process, both in terms of reminding patients to log in and of providing active feedback about entered information. In this study, we focused primarily on encouraging patients to become engaged in recording their own symptoms. This effort included little response to these data; the flow of information was largely unidirectional. For example, there was no dedicated nurse review of the symptom burden and no targeted information provided back to patients. We found that, even without such interventions or feedback, a high proportion of patients was willing to self-report. Future adherence rates may improve with an added incentive of explicit clinical review of self-reported information.
Prior research suggests that integration of electronic patient self-reporting into the flow of routine care delivery improves adherence.26 In a follow-up study, patient-reported symptoms are being explicitly addressed by a nurse during and between visits. We are measuring effects on adherence and the extent to which e-mail alerts and printed reports are used as the basis for specific clinical decisions.
Even if a PRO approach to toxicity symptom data collection can be refined and optimized for cancer patients, what are the potential benefits in oncology trials and routine care? In clinical research, the ability to accurately and consistently collect toxicity data is essential. PROs offer an automated means to collect this information and symptom end point data in real time (including between visits). This approach has recently been advocated by regulatory agencies.1-3,15-18,27 In routine oncology care, PROs may assist with symptom management. A model in which patient-reported information is regularly monitored by dedicated clinical personnel may expedite the management of symptoms and may potentially avoid hospitalizations.
Although we speculate that clinic-based patient reporting of toxicity symptoms may increase efficiency by reducing time for clinicians to elicit symptoms, between-visit reporting may increase the work burden by generating information that must be reviewed or acted upon. It is not clear if the number of actionable symptoms reported from home via an online system will exceed those reported by patients using traditional methods (eg, the telephone). This is being assessed in an ongoing, randomized trial of home-based, patient self-reporting versus standard care.
Limitations of this study include enrollment only of English-speaking patients in a single urban cancer center and a restricted number of measured symptoms. Different populations variably may be willing to complete online forms and should be evaluated. This study was not designed to distinguish between symptoms of early and late treatment effects, and most patients remained on active treatment throughout enrollment.
CTCAE items have not undergone formal validation, and the CTCAE's development was largely consensus-based.13 We selected the CTCAE for this study because it remains the regulatory standard tool for monitoring toxicities in cancer treatment trials.
Only a limited number of the CTCAE items were included in this feasibility study. Future evaluations should include the ability to capture a greater number of symptoms. Approximately 127 of the 1,059 CTCAE items can be classified as subjective and are amenable to patient self-reporting. Although it is not practical to ask patients to respond to this many items, an electronic platform could be designed to custom-select appropriate symptoms based on diagnosis, treatment, patient characteristics, and prior responses. A checklist or search capability could allow patients to add additional symptoms.
Future PRO studies of toxicity monitoring should also consider combining use of the CTCAE with more comprehensive instruments to provide more granular information about symptoms of interest. For example, a multitiered questionnaire system can be envisioned in which responses to the CTCAE screening items trigger longer questionnaires: if a patient screens positive for the presence of anxiety, that patient then could be presented with a further questions that hone in on that area of concern.
As our understanding of the patient experience improves, we foresee a paradigm shift whereby more objective toxicities will continue to be tracked by professionals but symptoms will be self-reported by patients or by both patients and clinicians (eg, PRO with clinician review). Ultimately, when we inform patients about potential adverse effects of therapies, they may prefer to be presented with data based on the impressions of their peers.
AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST
The authors indicated no potential conflicts of interest.
AUTHOR CONTRIBUTIONS
Conception and design: Ethan Basch, Alexia Iasonos, Ann Culkin, Mark G. Kris, David Artz, Paul Fearn, John Speakman, Howard I. Scher, Mary McCabe, Deborah Schrag
Administrative support: Allison Barz
Provision of study materials or patients: Ethan Basch, Ann Culkin, Mark G. Kris, David Artz, John Speakman
Collection and assembly of data: Ethan Basch, Allison Barz, Ann Culkin, Rena Farquhar
Data analysis and interpretation: Ethan Basch, Alexia Iasonos, Allison Barz, Ann Culkin, Howard I. Scher, Mary McCabe, Deborah Schrag
Manuscript writing: Ethan Basch, Alexia Iasonos, Deborah Schrag
Final approval of manuscript: Ethan Basch, Alexia Iasonos, Allison Barz, Ann Culkin, Mark G. Kris, David Artz, Paul Fearn, Rena Farquhar, Howard I. Scher, Mary McCabe, Deborah Schrag
Acknowledgments
We thank the physicians and nurses of the Memorial Sloan-Kettering Cancer Center, Thoracic Medical Oncology clinic for assistance with patient recruitment; to Kai Lin, Kevin Shannon, and Charmaine Pun for technology development; and to Howard Thaler and Venkatraman Seshan for assistance with data analysis.
Footnotes
-
Supported in part by the Steps for Breath Fund for Lung Cancer Research and the Society of Memorial Sloan-Kettering Cancer Center.
-
Presented in part at the 11th Annual Meeting of the International Society for Pharmacoeconomics and Outcomes Research, Philadelphia, PA, May 20-24, 2006.
-
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
- Received February 8, 2007.
- Accepted July 10, 2007.
