- © 2009 by American Society of Clinical Oncology
Improving the Methodologic and Ethical Validity of Best Supportive Care Studies in Oncology: Lessons From a Systematic Review
- From the Shaare Zedek Medical Center, Department of Oncology, Cancer Pain and Palliative Medicine Unit, Jerusalem, Israel; Division of Medical Oncology, Department of Internal Medicine, Duke University Medical Center, Durham, NC; Palliative and Supportive Services, Flinders University, South Australia, Australia; and Oncological Palliative Care, Oncology Department Internal Medicine and Palliative Care Center, Cantonal Hospital, St Gallen, Switzerland.
- Corresponding author: Nathan I. Cherny, MBBS, FRACP, FRCP, Cancer Pain and Palliative Medicine Service, Department of Medical Oncology, Shaare Zedek Medical Center, Jerusalem, Israel 91031; e-mail: chernyn{at}netvision.net.il.
Abstract
Purpose To systematically review the best supportive care (BSC) literature and to evaluate the ethical and methodologic validity issues by using widely acknowledged criteria.
Methods Two search strings that included both cancer and supportive as terms (with random article type, or review or meta-analysis) explored databases from 1966 to 2008. Citations, abstracts, and papers were reviewed for inclusion criteria, and relevant data were extracted by two independent researchers. Data were validated for accuracy. Ethical and methodologic validity were evaluated by using the criteria derived from the Helsinki Requirements of the WMA; CONSORT statements for the evaluation of reports of randomized, controlled trials; and the universal requirements for ethical clinical research.
Results Forty-three published papers were identified that described 32 studies, 20 of which incorporated the design of treatment plus supportive care (SC) versus SC alone, and 12 of which incorporated the design of treatment versus SC. Most of the studies had poor compliance to critical Helsinki requirements, to methodologic precautions derived from the CONSORT statement for studies involving a nonpharmacologic arm, and to four of seven universal requirements for ethical clinical research.
Conclusion Lack of rigor in BSC studies has contributed to a generation of research with widespread ethical and methodologic shortcomings. Ad hoc SC and lack of standardization of SC delivery may be sources of systematic bias or error in BSC trials. Rectifying these shortcomings in future studies demands greater vigilance toward these issues by researchers, institutional review boards, editors, and peer reviewers. Given the prevalence of overlooked problems that are later identified, currently open BSC studies should be reevaluated by institutional review boards and researchers to check for ethical and methodologic validity, and identified shortcomings should be addressed.
INTRODUCTION
Since the mid 1980s, best supportive care (BSC) studies have emerged as the dominant research paradigm for randomized, controlled trials of new anticancer treatments for diseases hitherto unresponsive to treatment. The studies aim to identify therapies that improve the duration of survival or the quality of life compared with the current standard of care, which is the provision of palliative care. They emerged as an alternative to studies of treatment versus no treatment, as these studies had hitherto been the dominant research paradigm to address this issue.
BSC studies are heterogeneous, and they have involved two different designs: incorporation of supportive care (SC) as a control arm (ie, treatment v SC design) or incorporation as a component of both arms (ie, treatment plus SC v SC alone; Fig 1). Since the early 1980s, more than 30 of these studies have been published, most commonly in non–small-cell lung, pancreatic, and gastrointestinal cancer. This constitutes an important body of literature that has been published in high-impact journals; has been subjected to systematic reviews for specific diseases; is widely cited; and has contributed to substantial changes in practice, policy, and the funding of specific therapies.
Two designs of best supportive care studies. SC, supportive care.
The clinical value and scientific validity of this research approach has been criticized. It has been clamed that the SC in these studies is inadequately described,1–4 is not consistent with any validated standard,1–4 is routine rather than best,1–4 is substandard,1 and is delivered by physicians who were possibly inadequately skilled for the task.1,3 There is substantial evidence to indicate that routine supportive care often features inadequate evaluation of pain,5–8 inability to appreciate the presence and/or severity of symptoms,8–11 lack of consultation with experts in pain or palliative care,12,13 substandard pain treatment,5,6,8,9,14,15 poor symptom control,8,9,16 and a lack of attention to psychological or existential distress9,16–18 or to family support.9,17,18 In addition, evidence indicates that oncologists frequently feel inadequately prepared for this aspect of their work.19–22 In the United States, evidence to this effect was summarized in a report by the Institute of Medicine.23
To critically evaluate these concerns, we sought to systematically review BSC studies in solid tumors to address their ethical and scientific validity and to identify pertinent implications that derive from those findings.
METHODS
MEDLINE and CANCERLIT searches were carried out for the period of 1966 to 2008 by using the following terms (in combination of keywords, title, and text): cancer (as keyword), randomized (as text), and supportive (as title); cancer (as keyword), random allocation (as keyword), and supportive (as text); and cancer (as keyword), random allocation (as keyword), and palliative (as text). These searches yielded 100, 32, and 106 papers, respectively, and incuded 146 distinct papers. Of those, 32 papers that satisfied each of the following inclusion criteria were selected: English language; published in full; involved patients with advanced or metastatic cancers for which there were no standard disease-modifying treatments and for which SC was the standard of care; investigated the use of a disease-modifying treatment that was either chemotherapeutic, biologic agent, radiation, or any combination of these; and either supportive or palliative care as the control arm.
MEDLINE and CANCERLIT searches with the following terms were performed to identify systematic reviews: cancer (as keyword), metastatic (in any usage), systematic (in any usage), supportive (in any usage), and review (in any usage); and cancer (as keyword), metastatic (in any usage), meta-analysis (in any usage), supportive (in any usage), and review (in any usage). This search yielded 38 papers, 16 of which addressed the role of anticancer treatments for advanced or metastatic cancer when SC was the standard of care. Hand searches of the bibliographies of the systematic reviews, identified papers, and specialist texts identified 11 additional papers that met the inclusion criteria.
Forty-three papers that described 32 studies were identified. All of these were reviewed independently by at least two researchers (N.C. and either R.S., Y.Z., or F.S.) to extract data regarding the study background, design, methodology, results, and discussion (Data Supplement, online only). The studies were classified according to their study design, as follows: treatment plus SC versus SC alone, or treatment versus SC. Study design was extracted from both the study title and the methodological description. Extracted data were tabulated and verified. Discrepancies were resolved by consensus between the investigators. Data were analyzed for each of the study designs.
Ethical and methodological validity were evaluated by using the criteria derived from the Helsinki Requirements of the World Medical Association24–30; relevant methodological items derived from the updated CONSORT standards for the reporting of randomized, controlled studies31,32; and the seven universal requirements for ethical clinical research explicated by Emanuel et al.33
RESULTS
Published Papers
Of the 43 papers that matched the inclusion criteria, many of the papers were published in high impact journals, including the following: Journal of Clinical Oncology (n = 12),34–45 British Journal of Cancer (n = 5),46–50 Cancer (n = 4),51–54 Annals of Oncology (n = 3),55–57 Journal of the National Cancer Institute (n = 3)58–60 and Lancet (n = 2).61,62
Twenty studies incorporated the design of treatment plus SC versus SC alone, and 12 involved the design of treatment versus SC.
Studies of Treatment Plus SC Versus SC Alone
Twenty studies tested treatment plus SC versus SC alone, and these included studies with non–small-cell lung cancer (n = 10), colorectal cancer (n = 4), mesothelioma (n = 2), small-cell lung cancer (n = 1), pancreatic cancer (n = 1), gastric cancer (n = 1), and assorted gastrointestinal cancers (n = 1; Tables 1 and 2). Most of these studies evaluated the role of chemotherapy as a first-line therapy adjunctive to SC, and three studies evaluated biologic agents. These 20 studies evaluated the incremental benefit of the study treatment compared with SC alone as measured by primary outcome of survival duration (n = 14), survival duration and quality of life (QOL; n = 2), QOL (n = 2), symptom severity (n = 1), and time to progression (n = 1). Sixteen studies included QOL assessment; of these, 14 used a validated QOL tool. These 20 studies involved a total of 4,628 participants; 2,620 were randomly assigned to receive SC with a study treatment, and 2,008 received SC alone.
Studies of Treatment Plus SC v SC Alone
Studies of Treatment Plus SC v SC Alone: Methodological Findings
None of the 20 studies reviewed contemporaneous standards for SC; two included a limited review of SC for the disease being addressed.34,61 One fully described the SC treatment that was used,58 two had partial descriptions,55,61 15 had minimal descriptions,34,36–40,46,47,51,52,56,57,59,62–65,68 and two had no description.35,66,67 Examples of minimal description included the following: “supportive care, which included palliative radiation, psychosocial support, analgesics, and nutritional support”51(p1271). “BSC was defined as the best palliative care per investigator excluding antineoplastic agents”39(p1659); and “palliative care, including radiotherapy, antibiotics, cough suppressants, analgesics, and so on, was given to all patients without restriction according to the standard practice of the collaborating centers.”34(p3189)
In half of the studies,10 SC delivery was at the discretion of the treating investigator, and there was no reference standard.36,37,39,46,47,59,63,64,67,68 Three invoked a national reference standard.55,56,61 Six studies invoked local or institutional standards34,38,51,62,65,66 without any additional details regarding the content or validity of those standards or regarding whether they represented a written document or customary practice. One study used an idiosyncratic standard that was inconsistent with contemporaneous standards and that included use of lorazepam for pain and no use of oral opioids for severe pain.58 Only one of the 20 studies gave any indication of the training and experience of the participants in the provision of SC.61 Three of the studies included participating sites in countries with recognized shortcomings in opioid availability and accessibility.36,64,67
Sixteen studies did not present any details of the care actually delivered in the SC arm34,35,37–40,46,51,52,55,56,58,59,61–63,66–68; in four papers, limited details of SC delivery were presented.36,47, 64,65 Only two papers addressed SC in the discussion51,61; one highlighted the need for a standardized SC control arm for future studies.51 No paper included a disclaimer regarding the quality of SC as a potential source of bias.
Studies of Treatment Versus SC
Twelve studies tested treatment versus SC, and these included studies in non–small-cell lung cancer (n = 6), gastric cancer (n = 2), colon cancer (n = 1), pancreatic cancer (n = 1), advanced refractory cancer (n = 1), and metastatic cancer with brain metastases (n = 1; Tables 3 and 4). Most of these evaluated the role of chemotherapy as a first-line therapy relative to SC. These studies evaluated the relative efficacy of disease-modifying treatment to SC as measured by primary outcome of survival duration (n = 8) or survival duration and QOL (n = 4). Seven studies included an assessment of QOL; of these, two used a validated QOL tool, four used performance status as a surrogate indicator of QOL,54,71,74,75 and one used a nonvalidated idiosyncratic scale.73 These 12 studies involved 1,023 participants, 584 of whom received some form of disease-modifying treatment, and 539 of whom received supportive care.
Studies of Treatment v SC
Studies of Treatment Plus SC v SC Alone: Methodological Findings
Although this design implies that participants randomized to active treatment would receive SC if, and only if, the participants no longer benefited from anticancer treatment, in actuality this was not the case. The studies by Rapp et al,42 Cellerino et al,43 the Elderly Lung Cancer Vinorelbine Italian Study (ELVIS) group,49,60 Sheperd et al,44,72 and Pyrhonen et al50 documented the provision of off-study SC to the participants in the treatment arm, and no study explicitly precluded administration of SC to participants in the treatment arm.
None of the 12 papers reviewed contemporaneous standards for the administration of SC. Nine papers included minimal description of SC,41,44,49,50,54,71,73–75 and three had no description.43,53,70 In 11 studies, SC delivery was at the discretion of the treating investigator, and there were no reference standards.41,43,44,50,53,54,70,71,73–75 The ELVIS study invoked recognized standards for pain management and antiemetic use.49 No study described the training and experience of the participants in the provision of SC, and one multinational study included centers from a country with recognized shortcomings in opioid availability and accessibility.53
Eight studies did not present any details of the care actually delivered in the SC arm,43,53,54,70,71,73–75 and limited details were presented in four studies.42,44,49,50,60,72 No paper included a disclaimer regarding the standard of SC administration as a potential source of bias.
DISCUSSION
On the basis of the findings of the systematic review, we will address three questions: Are these studies compliant with the Declaration of Helsinki? Are these studies scientifically valid? Are these studies ethical?
Are These Studies Compliant With the Declaration of Helsinki?
The Declaration of Helsinki, developed by the World Medical Association is the most widely cited set of ethical principles for research in humans. Originally formulated in 1964,24 it has undergone six amendments in 1975, 1983, 1989, 1996, 2000, and 2008.25–30 Two items of the Helsinki declaration are particularly pertinent to this evaluation: First, every version of the declaration has included the requirement that studies be predicated on a “thorough knowledge of the scientific literature.”24–30 In the current version, this appears in the section “Principles For All Medical Research.”30(p2) Second, every version of the Declaration since 1975 has included a standard-of-care requirement for the control arm of randomized studies.25–30 From 1975 to 2008, the requirement was for a best current care standard.25–29 The 2008 revision modified this standard to the best current proven care, and it added an evidentiary requirement.30 All versions either impute or state that if there is no best current care or, more recently, best current proven care, then other standards may be used if, and only if, they do not substantially increase the risk of harm to participants.25–30
Regarding the first item, one of the most striking findings of this systematic review is the near-total absence of background data on the contemporaneous best practices of SC presented in the preamble or discussion of these studies. In the overwhelming majority of studies that involved a SC arm, there was no evidence to suggest that the SC strategies employed were based on a thorough knowledge of the scientific literature and other relevant sources of information, as required by the Helsinki Declarations. In most of the studies, the standard of SC was manifestly ad hoc, and there was no apparent use of invoked standards.36,37,39,41,43,44,46,47,50,53,54,59,63,64,67,68,70,71,73–75 Even when standards were invoked,34,38,49,51,55,56,61,62,65,66 they were infrequently elucidated, and little evidence was presented to demonstrate the degree to which they were actually adhered.
Did contemporaneous standards for best current SC exist at the time these studies were planned and executed? Standards for best current SC have existed in various formulations before the initiation of all but the earliest of these studies. The volume and quality of these formulations have developed in number and sophistication over time. The first of many textbooks to formalize this body of knowledge was published in 1978.76 International standards for best practice in cancer pain77 and the palliative care for patients with advanced cancer78 were first published by the WHO in 1986 and 1990, respectively. Multi-author, comprehensive texts describing best practices have existed since 1993.79 Current guidelines have been derived from an extensive evidence base,80,81 and they are widely covered in major palliative care82 and oncology texts,83 monographs,84–86 and evidence-based standards of practice.87–94 Condensed from the 1990 WHO recommendations,78 10 elements of the SC practices relevant to this genre of studies are presented in Table 5.
Derived Standards for Supportive Care Relevant to Best Supportive Care Clinical Studies
No study incorporated justification for deviation from a best care standard; indeed, 18 studies used the superlative, best, to describe the SC (ie, BSC) without any justifying, evidentiary qualification.36–39,41,43,44,46,47,55,56,58,59,63–67,70 This point is illustrated additionally by the manner in which SC was elaborated, typically as a list of therapeutic approaches—such as palliative radiotherapy, analgesia, symptomatic care, and psychological care—rather than as standards for assessment, implementation, or monitoring. Only the study by Muers et al61 explicitly included more than one of the 10 recommended elements of SC derived from the 1990 WHO standards (Table 5).
Researchers may assert that the control arm in many of these studies represented usual practice, as this appears to be the case in most of these studies. On the basis of what is known about usual practice, it is reasonable to presume that patients received SC that ranged from good through mediocre to very poor. The Declaration of Helsinki, however, explicitly rejects a usual practice standard for control arms when usual practice is inferior to best practices and when it may result in significant, untoward patient consequences.25–30
Are These Studies Scientifically Valid?
“Well-designed and properly executed randomized, controlled trials (RCTs) provide the best evidence on the efficacy of health care interventions, but trials with inadequate methodologic approaches are associated with exaggerated treatment effects. Biased results from poorly designed and reported trials can mislead decision making in health care at all levels, from treatment decisions for the individual patient to formulation of national public health policies.”31(p663)
Supportive care is a complex, nonpharmacologic intervention that also involves interdisciplinary care, major commitments to assessment and monitoring, and components of pharmacologic therapies. In studies that incorporate an SC arm, the effect sizes of antitumor treatment interventions may be inflated if the comparator is substandard. The quality of the SC is influenced by many factors, including the expertise of the participating clinicians and the centers in which they work, the invoked standards for SC administration incorporated in the protocol, standardization across participating sites, and actual SC delivered.
Consolidated Standards of Reporting Trials (CONSORT), a consortium supported by the International Committee of Medical Journal Editors, provides a framework to facilitate the appraisal of research reports. The premise of this approach is that explicit reporting of specific components of trial methodology is necessary to allow readers to accurately assess trial validity.31,95 CONSORT has recently published an extension to the original 22-item checklist31; the extension addresses methodological issues specific to studies that involve complex, nonpharmacologic interventions, such as SC.32 This checklist includes specific methodological precautions to reduce bias in this genre of studies and includes a requirement to acknowledge the potential for bias. The precautions include the following: 1 Careful description of the care providers and the centers in which participants were treated, along with presentation and justification of eligibility of the care providers, including professional qualifications, expertise, or specific pretrial training (item 3); 2 detail of the treatment components that may influence the outcome effect, including provision for individual subject condition, tolerance, and clinical course (item 4a); 3 detail of how interventions were standardized across centers (item 4b); 4 assessment of protocol adherence (item 4c); and 5 acknowledgment that results may be biased by the standards of care that was actually provided (item 20).
None of the studies in this systematic review comply closely with these requirements. Few involved SC credentialing of providers or their centers (and some were carried out in countries without ready access to opioids), standardization of SC across treatment sites, or assessment of adherence to standards of SC delivery. In the overwhelming majority, the description of the supportive care was suboptimal,34,36–40,46,47,51,52,54–57,59,62–65,68,71,75 and five studies had no description.35,43,53,66,67,70 One paper detailed the planned care, but the approach was not consistent with contemporaneous standards of practice.58 No paper clearly described the SC that was actually delivered. In the discussion sections of these 32 papers, only Ganz et al51 offered a disclaimer that results may be influenced by the standard of SC and called for the incorporation of standards in future studies.
The methodological validity of the 12 studies that ostensibly randomly assigned patients between anticancer treatment and SC are particularly problematic. The design implies that patients would receive one treatment or the other and that this would serve as the basis for comparison between the approaches. Given the symptom burden of patients with advanced cancer and the clinical and ethical imperative to relieve distress, a clean comparison is implausible. Indeed, none of these studies specifically stated that symptom control was precluded from patents on the treatment arm; five studies specifically described the provision of symptom control to patients on the treatment arm,42–44,50,60 and it is unlikely that any patient participating in these nine studies was actually denied SC if it was needed.
Are These Studies Ethical?
On the basis of a review of codes for clinical research ethics, Emanuel et al33 derived seven universally applicable requirements that provide a coherent and systematic framework for addressing ethical validity (Table 6). Many of these studies have compliance issues with four of the seven requirements: social value, scientific validity, balancing of risks and benefits, and respect for enrolled participants.
Seven Universal Criteria for Ethical Research
Social value.
Social value is addressed through the inquiry as to whether the research will derive knowledge or promote ends that will improve health and well-being.33,96 The promotion of ends to improve health includes indirect benefits of research, such as capacity building96 and the educative and role-modeling value of the research.97
The converse of social value is social harm. Although we acknowledge the value of research that has proven or disproven the merit of various anticancer therapies, we believe that the lack of standards applied to the SC delivery of this body of research may have, simultaneously, contributed to social harm by modeling ad hoc care as an acceptable standard of practice. Rather than conveying the message that palliative and supportive care should be administered in accordance with best practice standards, this body of research implies that ad hoc routine care, bereft of any recognized standards, is a reasonable and acceptable standard of practice. As emphasized by Harold Varmus,97 clinical practitioners look to the medical research community to seek out standards of best contemporaneous practice. If Varmus is correct, the role modeling provided by these studies may be one of the reasons many cancer clinicians and treatment centers have not adopted recognized standards of practice for their routine administration of SC.19,98–100 The ad hoc SC modeled in most of these studies neglects important evidence that adherence to standards of SC can reduce patient and family suffering80,81,86,101–106 and is contrary to the efforts of the American Society of Clinical Oncology (ASCO)107–109 and the European Society for Medical Oncology (ESMO)110 to promote professional rigor to the delivery of this aspect of care. The negative educational impact of this modeling is illustrated by the lack of development in the standards of SC used in this body of research during 20 years; with one exception,61 all of the studies initiated in 2002 to 200523,36–40,66 replicated the ad hoc standard of care modeled in studies performed in 1986 to 1996.41,43,47,50,53,54,63,64,68,71,74,75
Scientific validity.
As previously stated, by criteria derived from the CONSORT statements,32 there are methodological shortcomings that may have introduced systematic bias in most of the studies.
Favorable risk-benefit ratio.
According to this criterion, clinical research can be justified only if three conditions are fulfilled: the potential risks to individual participants are minimized; the potential benefits to individual participants are enhanced; and the potential benefits to individual participants and society are proportionate to—or outweigh—the risks.33 To elaborate on the second point, the authors highlight that the requirement to enhance potential benefit relates only to benefits derived directly from the research plan.33 Except for a few,55,56,61 the studies that incorporated SC arms did not minimize participant risk by the incorporation of best contemporaneous SC practices; standards for SC were not reviewed and were rarely invoked, and ad hoc practices were the norm.
Respect for potential and enrolled participants.
All patients with advanced and incurable cancer are entitled to receive quality supportive and palliative care.111 Respect for the welfare of patients with advanced and incurable cancer must incorporate provision of effective palliative care to all participants in phase I, II, and III studies.111,112 Given the recognized deficiencies of ad hoc SC, respect for enrolled participants demands a level of SC that is consistent with recognized standards of practice. With few exceptions,55,56,61 this was not the case in the studies reviewed.
How Did This Happen?
These findings beg the questions: If this body of research is so flawed, why are these methodologies so widely employed, and why have these studies been so widely published and uncritically cited? We believe that a sequence of circumstances has facilitated this.
It may be that most of these researchers were genuinely unaware of standards of practice for the administration of supportive or palliative care. By self report, a major proportion of oncologists receive inadequate training in this aspect of care.19–22 With only two apparent exceptions,51,61 these studies were conducted without participation of clinicians with specific expertise in palliative care. Naiveté aside, the researchers can be fairly criticized for failing to carefully review the standards for supportive and palliative care contemporaneous with their investigations or to involve experts in supportive or palliative care in the collaborative research process.113
Institutional review boards (IRBs) seem to have overlooked many of the ethical and methodological shortcomings of these studies. This may reflect a lack of familiarity with issues specific to the SC of patients with advanced cancer, an exaggerated evaluation of the researchers' expertise in SC, or a lack of familiarity with the more complex methodological validity requirements of studies that involve nonpharmacologic interventions. This is not a new issue. Indeed, it is widely recognized that IRBs often lack the time, research capacity, deliberative mechanisms, and expertise to evaluate all of the methodological and ethical issues raised by studies under review.114,115
When research is combined with clinical care, regulations—such as the widely endorsed Good Clinical Practice (GCP) standards116—provide inadequate protection of research participants. Specifically, section 4.3 of the GCP guideline addresses standards of ancillary (ie, off-study) care only in the most general of terms, and it has no reference to recognized standards of care. For patients with advanced cancer in particular, current GCP standards are not sufficient to ensure that patients will receive quality supportive and palliative care.
Although most editors ostensibly insist on Helsinki and GCP compliance and adherence to CONSORT criteria to evaluate the validity of studies to be published, in practice they have limited mechanisms to do so. More often than not, IRB approval and informed consent are surrogates for Helsinki compliance, and judgments of methodological validity are left to peer reviewers who may or may not be familiar with the criteria to evaluate study validity.117
Limitations and Disclaimers
Published reports ostensibly provide the information necessary to facilitate the readers' abilities to assess the methodological validity of the research studies.31 Although this assumption underscores the method invoked in this study, we recognize that there is conflicting evidence regarding this assumption. Studies have been conflicting when they have evaluated the correlation between data extracted from publications to primary data provided by principal investigators. Some have indicated that faulty reporting reflected faulty methods,118 and others have indicated that sound studies may be poorly represented because of the constraints of editorial guidelines or deficient reporting.119 It is possible that some of the shortcomings highlighted in this report may have been addressed in the research protocols or in unpublished data.
The quality of the SC provided to participants in these studies is central to this analysis. In the absence of data describing the care that was actually delivered, we have made imputations of the likely quality of the care delivered on the basis of published experience with routine SC practices by oncologists. If we acknowledge that there was no direct evidence of patient harm, then the care actually received was possibly either better or worse than that which we have imputed.
We recognize that the recommended use of the CONSORT criteria is to identify shortcomings in the reporting of randomized, clinical trials.31 When we assessed the methodological validity of studies that included a SC study arm, we used criteria derived from the CONSORT statement version that specifically addressed the study of complex nonpharmacologic interventions.32 Methodological and scientific validity is compromised in research that lacks appropriate precautions to minimize systematic bias.120 The criteria used in this evaluation focus on the methodological precautions necessary to minimize systematic bias in trials that involved nonpharmacologic study arms (eg, the SC trials) and focus on the need to explicitly address the potential for systematic bias introduced by the specific methodology employed.32 Other, more commonly used criteria for bias121 not related to the methodology of SC administration in study conditions were outside of the scope of this evaluation.
Implications
This systematic review strongly suggests that there are problems in the ethical and scientific validity of many of the BSC studies published to date. Regarding scientific validity, we believe that the published results should be interpreted critically with awareness of the potential for bias that was introduced by lax methodology. Regarding ethical shortcomings, it is imperative to ensure that future SC research endeavors be more compliant with ethical standards for research and have more positive social value, and that participants are offered SC in accordance with contemporary evidence-based standards.
These findings have specific implications for researchers, ethics review boards, and the publication process. We encourage a collaborative dialogue between researchers, ethical review boards, medical editors, and their peer reviewers to underscore certain basic agreements for future trials that include SC as part of their design:
First, because appropriate control selection is critical to ethical and scientific validity, the process should incorporate systematic review of the relevant literature, formal involvement of clinical practitioners expert in that aspect of care, and circulation of the trial protocol to solicit critical appraisal.113,122 Second, research that incorporates an SC control arm must use an adequate contemporary standard of SC.87–94 Failure to do so is unethical and introduces bias that may undermine the validity of the research project. Third, participating researchers must be adequately credentialed in SC, and appropriate quality control measures must be put in place to ensure that SC provision is consistent with recognized standards.32 Fourth, in multicenter studies, SC practices should be standardized between centers.32 Fifth, study designs that preclude the provision of palliative care to a group of participants (eg, treatment v SC) are never ethical.111,112 Finally, a logistically simpler, alternative study design should be considered. A study design to randomly assign treatment versus no treatment or placebo treatment, which thus administers SC off study to all participants,123–128 limits the liability of the researchers for many of the complex standardization and validity issues invoked by inclusion of SC in the study arms31,32 but still provides SC and addresses the central question as to whether there is added benefit to the new treatment. Even in studies that use this design, however, sound clinical practice and respect for research participants demands that SC be provided in accordance with contemporaneous standards of practice.
A more challenging problem is the more than 10 BSC studies that are currently open to accrual (www.clinicaltrials.gov). We recommend that researchers and IRBs critically review BSC studies that are currently open. When problems are identified, particularly in the standard of SC being delivered, consideration should be given to protocol amendments to ensure that patients are offered an ethically appropriate standard of SC. Researchers should refer to the CONSORT statement on trials of nonpharmacologic treatments to assist them in addressing issues related to the scientific validity of these studies and the recommended disclaimers.32 Studies that cannot adequately address these issues should be discontinued.
Studies that do not meet criteria for ethical and scientific validity submitted for publication present a challenge to reviewers and editors, because most journals are committed to publish only human research reports that are compliant with Helsinki standards. In situations in which the results do include significant, generalizable knowledge, consideration should be given to limited publication (possibly as a brief report) with appropriate disclaimers.
In conclusion, lack of rigor in supportive care practices has contributed to a generation of research with ethical flaws and methodological shortcomings that may have contributed to biased outcomes. To optimize the well-being of participants in research and patients in practice, supportive and palliative care need to be addressed with the same professional rigor, professionalism, and adherence to standards as the disease-modifying aspects of cancer care. Rectification of these shortcomings in future studies and publications will demand greater vigilance to these issues by researchers, IRBs, editors, and peer reviewers.
AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST
The author(s) indicated no potential conflicts of interest.
AUTHOR CONTRIBUTIONS
Conception and design: Nathan I. Cherny, Amy P. Abernethy, Florian Strasser, David Currow, S. Yousuf Zafar
Administrative support: Nathan I. Cherny, Rama Sapir
Provision of study materials or patients: Rama Sapir
Collection and assembly of data: Nathan I. Cherny, Florian Strasser, Rama Sapir, S. Yousuf Zafar
Data analysis and interpretation: Nathan I. Cherny, Amy P. Abernethy, Florian Strasser, S. Yousuf Zafar
Manuscript writing: Nathan I. Cherny, Amy P. Abernethy, Florian Strasser, S. Yousuf Zafar
Final approval of manuscript: Nathan I. Cherny, Amy P. Abernethy, Florian Strasser, Rama Sapir, David Currow, S. Yousuf Zafar
Acknowledgment
We thank colleagues who read and constructively criticized previous drafts of this article: Christine Grady, Franklin Miller, Ridar Lie, John Zalcberg, Alan Wetheimer, Ian Olver, Raphael Catane, Max Schwarz, Alberto Gabizon, Prue Frances, Barbera Murphy, Lori Minasian, Benjamin Djulbegovic, and the fellows of the Bioethics Department in the Clinical Center of the National Institutes of Health.
Appendix
Data Extracted From Published Reports
Footnotes
-
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
- Received January 13, 2009.
- Accepted April 7, 2009.
REFERENCES
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.
- 21.
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.
- 27.
- 28.
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.↵
- 39.↵
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 62a.↵
- 63.↵
- 64.↵
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
- 80.↵
- 81.↵
- 82.↵
- 83.↵
- 84.↵
- 85.
- 86.↵
- 87.↵
- 88.
- 89.
- 90.
- 91.
- 92.
- 93.
- 94.↵
- 95.↵
- 96.↵
- 97.↵
- 98.↵
- 99.
- 100.↵
- 101.↵
- 102.
- 103.
- 104.
- 105.
- 106.↵
- 107.↵
- 108.
- 109.↵
- 110.↵
- 111.↵
- 112.↵
- 113.↵
- 114.↵
- 115.↵
- 116.↵
- 117.↵
- 118.↵
- 119.↵
- 120.↵
- 121.↵
- 122.↵
- 123.↵
- 124.
- 125.
- 126.
- 127.
- 128.↵
