- © 2007 by American Society of Clinical Oncology
Criteria for Optimizing Prognostic Risk Groups in Pediatric Cancer: Analysis of Data From the Children's Oncology Group
- From the Children's Center for Cancer and Blood Diseases, Children's Hospital Los Angeles; Department of Preventive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA; Department of Epidemiology and Health Policy Research, University of Florida, Gainesville, FL; and the Children's Oncology Group, Bethesda, MD
- Address reprint requests to Richard Sposto, PhD, Children's Center for Cancer and Blood Diseases, Children's Hospital Los Angeles, 4650 Sunset Blvd, Mail Stop #54, Los Angeles, CA 90027-6016; e-mail: rsposto{at}chla.usc.edu
Abstract
Purpose Physicians who treat cancer often attempt to identify patient subgroups that are homogeneous in their chance of recurrence or death as a way to target the more toxic and presumably more effective treatments to patients with the worst prognosis. However, to date, prognosis-based treatment assignment in pediatric cancer has not been based on a quantitative assessment of the risks and benefits of different treatment strategies or on morbidity and efficacy outcome measures that are relevant to children.
Methods We performed a quantitative analysis of the risks and benefits of prognosis-based treatment assignment in two examples from the Children's Oncology Group using a mathematical model of cancer cure and permanent treatment morbidity and defined an optimality criterion for assigning treatments to specific risk groups.
Results In stage 4 MYCN-unamplified neuroblastoma, age-based risk grouping distinguishes clearly between patients with high and low risk of recurrence. However, our analysis suggests that the optimal age cut point depends profoundly on the morbidity of the treatments being considered and agrees with current published recommendations only for treatments that add significant morbidity. In Hodgkin's lymphoma, under our model, no clearly optimal risk groupings exist, and a compelling quantitative rationale for defining risks group at all may not exist.
Conclusion Our analysis illustrates the inadequacy of naïve application of statistical criteria for defining prognostic risk groups in pediatric cancer and highlights the importance of quantifying treatment morbidity when defining risk groups or when deciding whether risk grouping is warranted.
INTRODUCTION
Cure rates of children's cancer have improved dramatically in the last several decades.1-6 High cure rates heighten awareness of the risk of severe long-lasting treatment morbidity and reduced life expectancy7-11 in cured patients. When the majority of patients are cured, aggressively augmenting treatment for all patients unnecessarily exposes all patients to added morbidity to increase the cure rate in a minority of patients. Physicians try to minimize population morbidity by defining prognostic risk groups and reserving the most toxic and presumably more active treatments for patients with the worst prognosis and the least toxic and possibly less effective treatments for patients with the best prognosis.12-14 Advances in the ability to characterize cancers molecularly15-18 and to assess minimal residual disease19,20 may improve the accuracy of prognostic prediction by replacing or supplementing traditional clinical prognostic factors with this biologically based information.
To date, risk groups have not been optimized to maximize treatment success while minimizing morbidity. Usually, risk groups are defined by minimizing the P value in a log-rank or Cox regression analysis.21-24 P value–based criteria have undesirable properties.25-27 For example, one should be more willing to give a promising treatment that minimally increases permanent morbidity to a prognostically more heterogeneous high-risk group than one would a similarly promising treatment that adds significant morbidity, which should be reserved for the highest risk patients in whom the therapeutic benefit would justify added morbidity. The minimum P value–based risk group definitions are insensitive to these considerations. Features of pediatric cancer also make common survival methods undesirable. Risk groups are not usefully different if cure rates are similar among them because the objective is to cure and not simply to postpone death, but log-rank or Cox analyses are more sensitive to differences in time to treatment failure among patients who experience treatment failure than to differences in cure rate.28 Finally, the permanent morbidity of treatment in cured patients should receive greater weight than manageable, albeit possibly severe, toxicities that impart no permanent morbidity. It is permanent morbidity that will have a profound effect on the life of a cured child.
We describe a method for defining prognostic risk groups that balances treatment effectiveness and morbidity. We demonstrate the method in a hypothetical example and apply it to outcome data from Children's Oncology Group studies in Hodgkin's disease and neuroblastoma, which represent two extremes in the usefulness of prognostic risk groups in pediatric cancer. The analyses are not considered definitive, but rather, they illustrate important features of the problem of defining risk groups in childhood cancer. We will demonstrate that the existence of highly significant prognostic factors does not guarantee that useful prognostic risk groups can be defined and that the definition of risk groups and their utility depends on the morbidity of treatments being considered.
METHODS
Consider a prognostic score the values of which identify patients with different probabilities of cure (ie, prognosis). The score can be a single prognostic variable or a combination of variables as derived from multivariate analysis29-31 or microarray analysis.16,18 The score is not necessarily related to specific biologic targets of treatment. Prognostic scores have been used to define risk groups in childhood14,21,31,32 and adult cancer29,30,33-42 by picking appropriate cut points. (The terms risk and cure will refer to failure to eliminate the cancer and the elimination of the cancer, respectively, without regard to long-lasting morbidity from the cancer or its treatment.) The prognostic score is treatment context specific; it applies to patients who are receiving the current standard of care, which may already include treatment/risk group specificity.
Risk Group–Directed Treatment
One can define two risk groups by selecting a score cut point C and calling patients high risk (HR) if the score exceeds C and low risk (LR) if the score does not exceed C. (Larger scores will imply worse prognosis.) One can define three groups with two cut points (CLR and CHR) to define LR, HR, and intermediate-risk groups. Further subdivisions are possible. The treatment strategies we consider alter standard treatment according to risk group. In HR patients, treatment is augmented and potentially will increase cure rate but with possibly more permanent morbidity. Treatment is reduced in LR patients to lessen morbidity but at the risk of lower effectiveness.
Measure of Treatment Morbidity
Treatment morbidity is reflected in the morbidity index V, with values between 0 and 1. V reflects the degree to which a cured patient is comparable to a child who has never had cancer. A patient with no lasting effects of cancer or treatment will have V = 1. A patient with some permanent morbidity will have V < 1. Patients with V near 0 experience the worst consequences of cancer or treatment despite being cured in the narrow sense. The model considers only patients who are cured, so V is undefined for patients who are not cured. How V is determined is beyond the scope of this article, but we discuss the issue later. VStd is the morbidity index of standard treatment; VAug is the morbidity index of augmented treatment, and VRed is the morbidity index of reduced treatment, where VAug < VStd < VRed. The quotients QAug = VAug/VStd and QRed = VRed/VStd are relative morbidities of augmented and reduced treatment compared with standard treatment.
Gain and Loss From Reduced or Augmented Treatments
Table 1 lists the gain or loss in morbidity with reduced or augmented treatment instead of standard treatment. A patient who would be cured with standard treatment and also with the new treatment would gain if the new treatment was a reduction because morbidity would be less (VRed − VStd > 0) but would have a loss if the new treatment were augmentation (VAug − VStd < 0) because morbidity would be greater (row 1). A patient cured with standard treatment but not with new treatment would lose VStd regardless of the type of treatment (row 2). A patient not cured with standard treatment but cured with new treatment would gain VRed for reduced treatment and VAug for augmented treatment (row 3). A patient who is not cured with either treatment neither gains nor losses (row 4) because only cure (as opposed to simple prolongation of time to failure) and permanent morbidity (as opposed to transient toxicity) are considered.
Gain and Loss in Terms of the Morbidity Indices V According to Patient Outcome and Type of Treatment Received
Average Gain From Targeted Strategy
The average gain (or loss = –gain) from augmenting treatment in the HR group and reducing treatment in the LR group is as
follows: 
AverageGain depends on the cure rates in the LR patients receiving reduced (PCureLR,Red) or standard (PCureLR,Std) treatment, cure rates of HR patients receiving augmented (PCureHR,Aug) or standard (PCureHR,Std) treatment, morbidity indices for each type of treatment (VAug, VRed, VStd), and the fraction of patients classified as LR (FLR) and HR (FHR). The equation indicates that the lower cure rate with reduced treatment (PCureLR,Red < PCureLR,Std) is offset by a better morbidity profile (VRed > VStd) in the fraction of LR patients (FLR) and that the higher cure rate with augmentation (PCureHR,Aug > PCureHR,Std) will be offset by a worse morbidity profile with augmentation (VAug < VStd) in the fraction of HR patients (FHR).
We concentrate on the two-risk group situation where only HR patients receive augmented treatment and LR and intermediate-risk
patients receive the current standard. The earlier equation simplifies to the following equation: 
after substituting QAug × VStd for VAug and dividing out VStd. QAug is constant in this model, but the other quantities, and hence AverageGain, will change as the cut point is changed. The optimum cut point is that which maximizes the average gain. Further details
of the statistical methods and modeling are provided in the online-only Appendix.
RESULTS
Example 1: A Nearly Ideal Prognostic Score
Figure 1 shows Kaplan-Meier curves from 1,000 hypothetical patients with survival times simulated from a population with 60% cure rate and with a prognostic score that is an excellent discriminator of patients who survive or not. (Simulation details are provided in the Appendix, online only). Patients can be separated clearly into LR and HR groups by any of four different cut points that correspond to dramatically different percentages of patients in the two risk groups
Kaplan-Meier overall survival curves for low- and high-risk groups defined by four different cut points of a prognostic score in a simulated example of 1,000 patients from a population with 60% overall cure rate. Numbers of patients in the low-risk and high-risk groups are (A) 422 and 578 patients, (B) 601 and 399 patients, (C) 728 and 272 patients, and (D) 872 and 128 patients, respectively.
The strong influence of the prognostic score is evident in a parametric cure model regression analysis of cure rate28 (Fig 2A, lower curve). The distributions of scores in cured and not cured patients are distinct, albeit with some overlap (Fig 2B). Receiver operating characteristic (ROC) analysis43 (Fig 2C) confirms that the prognostic score is highly sensitive and specific as a predictor of patients who will not be cured; the area under the curve (AUC) is 0.98.
Results for Example 1 including 1,000 simulated patients. (A) Cure rate versus prognostic score with standard and augmented treatment. Interval-specific estimates ± SE demonstrate fit. (B) Histogram of scores in cured and not cured patients. (C) Receiver operating characteristic (ROC) curve for identifying not cured patients. (D) Gain versus score cut point by morbidity quotient QAug. Maximum gain (dots) and percentage of low risk patients (thin line) also shown.
Assume that augmented treatment cures 30% of patients with a particular prognostic score who are not cured by standard treatment (Fig 2A, upper curve). Figure 2D shows the average (expected) gain as a function of score cut point and morbidity quotient QAug when treatment is augmented in HR patients. QAug ranges from QAug = 0.99 for augmented treatment that has only slightly more morbidity compared with standard treatment to QAug = 0.50 for treatment that adds substantially to permanent morbidity. The percentage of patients in the LR group for each cut point is also shown.
As would be expected, the optimum cut point classifies fewer patients as HR as the morbidity of augmented treatment increases (Table 2). For augmented treatments that add the least morbidity (QAug = 0.99), 58% of patients are classified as HR at the optimum cut point, with a gain of 0.11, whereas only 33% of patients are classified as HR for an extremely morbid treatment (QAug = 0.50), with a much lower gain (0.04). For less morbid treatments, there is a wider range of acceptable cut points that give at least 95% of the maximum possible gain (Table 2). For reference, the minimum log-rank P value cut point is P = .56, which is appropriate for extremely morbid treatments.
Maximum Expected Gain and Other Quantities As a Function of Morbidity Index QAug for 1,000 Simulated Patients, 780 Patients With Hodgkin's Disease, and 317 Patients With Stage 4 MYCN-Unamplified Neuroblastoma
Although cured and not cured patients can be distinguished reliably using the prognostic score, the morbidity of augmented treatment drastically affects the percentage of patients who optimally should receive augmented treatment and the potential gain from this treatment strategy.
Example 2: Hodgkin's Disease
In the Children's Cancer Group 5942 study, patients were assigned to one of three treatments depending on stage, “B” symptoms, and presence of bulk disease.44 Seven hundred eighty patients with complete data on these variables and on erythrocyte sedimentation rate were analyzed. Because many patients who experience relapse are successfully re-treated, survival was used as the end point most reflective of cure. All of these variables are significant univariate prognostic factors for survival, with B symptoms, bulk disease, and erythrocyte sedimentation rate being significant independent factors. The first principal component45 of these variables was a significant predictor of cure rate (P < .005) and is used as the prognostic score because none of the individual factors was statistically significant after adjusting for this principal component.
Nearly 100% of patients with the lowest prognostic scores are cured compared with approximately 70% of patients with the highest scores (Fig 3A, lower curve). The overall cure rate is 92%. At the median score of 0.40, 94% of patients are cured. There is greater overlap of prognostic score here compared with Example 1 (Fig 3B). The ROC analysis (Fig 3C) suggests that this prognostic score may be a good discriminator of cured or not cured patients (AUC = 0.80)
Results for Example 2 including 780 patients with Hodgkin's disease. (A) Cure rate versus prognostic score with standard and augmented treatment. Interval-specific estimates ± SE demonstrate fit. (B) Histogram of scores in cured and not cured patients. (C) Receiver operating characteristic (ROC) curve for identifying not cured patients. (D) Gain versus score cut point by morbidity quotient QAug. Maximum gain (dots) and percentage of low-risk patients (thin line) also shown.
Assuming again that augmented treatment cures 30% of patients not cured by standard treatment (Fig 3A, upper curve), there is only a small gain in augmenting treatment in HR patients and only if the additional morbidity is small (Fig 3D, Table 2). When QAug = 0.99, the optimal cut point classifies more than 60% of patients as HR, but all patients could receive augmented treatment with little reduction in gain. When augmented treatment adds non-negligible morbidity, the gain is negligibly positive and only if the HR group is small. The minimum log-rank P value cut point is P = .65, which is optimal for QAug ≈ 0.95.
Using this prognostic score, there is little apparent advantage to segregating Hodgkin's disease patients in this manner. This does not rule out the possibility that a different prognostic score (eg, a molecular classifier) could reliably identify an HR group comprising mainly the 5% to 10% of patients not currently cured who would be candidates for augmented treatment.
Example 3: Stage 4 MYCN Nonamplified Neuroblastoma
London et al21 analyzed 3,666 pediatric neuroblastoma patients to refine risk classification in this disease. They concluded that an age cut point of 1.25 to 1.58 years was better for defining risk groups than the cut point of 1.0 year that had previously been the standard. We reanalyzed the age cut point question, restricting our analysis to 317 International Neuroblastoma Staging System stage 4, MYCN-unamplified neuroblastoma patients from this cohort. These patients were selected because they were homogeneous in that age was the single predominant prognostic factor. Because a minority of patients are successfully re-treated after relapse, the occurrence of progression, recurrence, or death was used as the end point most indicative of failure to achieve cure. The overall cure rate is 38%.
Age is a strong predictor of cure rate in these patients (Fig 4A). There also is a distinctively different distribution of age in cured and not cured patients (Fig 4B). Again, ROC analysis suggests that age alone may be a good discriminator of cured and not cured patients (AUC = 0.80; Fig 4C).
Results for Example 3 including 317 neuroblastoma patients. (A) Cure rate versus prognostic score with standard and augmented treatment. Interval-specific estimates ± SE demonstrate fit. (B) Histogram of scores in cured and not cured patients. (C) Receiver operating characteristic (ROC) for identifying not cured patients. (D) Gain versus score cut point by morbidity quotient QAug. Maximum gain (dots) and percentage of low-risk patients (thin line) also shown.
In contrast to the Hodgkin's disease example and similar to Example 1, except for the most morbid of treatments, there is a positive gain to augmenting treatment in HR patients (Fig 4D, Table 2). The minimum log-rank P value occurs at 1.4 years, but similar values occur from 1.3 to 1.6 years, which is consistent with the revised age cut point range of 1.25 to 1.58 years.21 The optimum cut point is 1.8 years for treatment augmentation with significant additional morbidity (eg, QAug = 0.50), but it is much less than 1 year (in fact, close to 0 years) if the augmented treatment adds minimal morbidity to current standard treatment; nearly all stage 4 MYCN-unamplified patients could receive such augmented treatment.
DISCUSSION
Simon,17 in an excellent review of issues surrounding developing and using a prognostic molecular signature, emphasized the need to ensure that “treatment options and costs of mis-classification are such that a classifier is likely to be used.” Our method is an attempt to quantify the costs and benefits, in terms of morbidity and treatment efficacy, of classifying (or misclassifying) patients into risk groups and assigning specific treatment to these risk groups. There are numerous examples of prognostic risk grouping in adult29,30,33-42 and pediatric14,21,31,32 cancer, but generally absent is the notion of optimality based on the benefit to the patients for risk-adapted therapy. Grosfeld12 stated that “risk-based management allows [one] to weigh the risks and benefits of treatment for each patient to maximize survival, minimize long-term morbidity, and improve quality of life.” The method we propose is possibly the first attempt in pediatric cancer to create a quantitative framework within which to make these judgments.
Our method uses simplifying assumptions that should be highlighted. Because current standard treatment in many cancers is already risk adapted, the morbidity of treatment will depend on the prognostic factors that determine risk. Our derivation of AverageGain assumes that the morbidity indices (Vs) are constant. If treatment and, therefore, treatment morbidity already depend on prognostic factors, the cut points selected from the proposed method may not be optimal. Preliminary investigation of this issue suggests that our method may be relatively insensitive to such dependence (see online-only Appendix). Our analysis assumes that augmented treatment has an additive effect; it cures a fixed proportion of patients who are not cured with standard treatment. This is appropriate for augmented treatments that have the largest absolute effect in the worst prognosis patients but differs from a common (but not well justified) proportional hazards assumption where a treatment would be comparatively more effective in lower risk patients and less effective in higher risk patients. The proportional hazards assumption would result in lower gains and different cut points. We assume that treatment morbidity is identical for each patient receiving a specific treatment. A more realistic statistical model would account for individual variability in morbidity, which can be a result of differences in susceptibility to morbidity, additional morbidity from required retrieval therapy before cure is achieved, or other factors. Our method also focuses on maximizing population gain to decide on global treatment strategies rather than on determining optimal treatment for an individual patient. It should also be noted that the method requires the availability of large, mature datasets that allow the modeling of outcome and prognosis.
The least well-defined part of this method is determining appropriate values for morbidity indices and ratios. These quantities must exist in concept and must be assignable in practice, given sufficient data, because, otherwise, there would be little basis for considering risk-based treatment assignment. Data on long-term treatment morbidity in children is now becoming available for the treatments that have been used in the past.46-49 The challenge will be to relate these data to treatments that are now being considered. There are also the human factors of perception and utility and whether a physician, parent, or patient would make the same judgments about the trade-off between cure and morbidity or about the relative importance of the different types of treatment morbidity.50,51 We investigated a range of morbidity ratios that likely bracket all realistic levels of morbidity.
Despite these limitations, we have shown that the problem of defining risk groups and assigning treatments to them is not one that can be solved using purely statistical criteria, as is the common practice, but must consider the prognosis of patients with current standard treatment, the morbidity of current treatment, the strength of the prognostic predictor, and the changes in effectiveness and morbidity that result from modifying treatment within some risk groups.
AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST
The authors indicated no potential conflicts of interest.
AUTHOR CONTRIBUTIONS
Conception and design: Richard Sposto
Collection and assembly of data: Richard Sposto, Wendy B. London
Data analysis and interpretation: Richard Sposto, Wendy B. London, Todd A. Alonzo
Manuscript writing: Richard Sposto, Wendy B. London, Todd A. Alonzo
Final approval of manuscript: Richard Sposto, Wendy B. London, Todd A. Alonzo
Appendix
Simulation of Survival Times for Example 1
Example 1 in the article is an analysis of 1,000 simulated survival times from a theoretical population with 60% cure rate.
The basis for this simulation is the mixture of the two subdensity functions for the prognostic score s in cured and not cured subpopulations shown in Figure A1. The function on the left is 0.60 × fC(s), where 
, and that on the right is 0.40 × fNC(s), where 
is the standard normal density function, and fC(s) and fNC(s) are the densities of s conditional on cured status. Hence, the density function of s in the population is g(s) = 0.60 × fC(s) + 0.40 × fNC(s).
Subdensities of prognostic score s among cured and not cured patients in theoretical example with 60% of patients cured.
For each patient, a random score value was generated from the density g(s). Then, a random uncensored survival time was generated from a nonmixture parametric cure model (PCM) (Sposto R. Stat Med 21:293-312, 2002) using a Weibull kernel with scale = 0.75 and shape = 1.3. Censoring was overlaid from a uniform distribution, which mimics Poisson patient entry over a 5-year period with 2 years of additional follow-up after the end of the accrual period.
PCM Regression of Cure Probability on Prognostic Score
In all three examples, PCM regression (Sposto R. Stat Med 21:293-312, 2002) was used to model the relationship between the probability of cure and the prognostic score. The parameters of the models were estimated via maximum likelihood (Lindsey JK. Parametric Statistical Inference. Oxford, United Kingdom, Clarendon Press, 1996). The details of the specific parameterization of the models are given below.
Example 1 (n = 1,000 simulated patients).
These data were analyzed using a nonmixture PCM with Weibull kernel. The cure probability was modeled as the logistic function
, the scale as 
, and the shape as 
. The parameter estimates and SEs are listed in Table A1. The adequacy of the fit of this model for predicting the probability of cure is illustrated in Figure 2A of the article.
Estimates of Parametric Cure Model Parameters for Example 1
Example 2 (n = 780 Hodgkin's disease patients from Children's Cancer Group study 5942).
The primary end point for survival analysis was time to death. Patients who were alive at last contact were censored at that time. Patients in each of three treatment groups received a different duration or type of therapy depending on the treatment group. See Nachman et al (Nachman JB, Sposto R, Herzog P, et al. J Clin Oncol 20:3765-3771, 2002) for details. Data available as of April 2006 were used.
Indicators of the presence at diagnosis of B symptoms, large anterior mediastinal mass, large mediastinal mass plus an additional large nodal aggregate, membership in treatment group 1, membership in treatment group 3, and log10(erythrocyte sedimentation rate + 0.5) were used as prognostic variables. These were first summarized in a principle components analysis, and the first principal component was scaled to range between 0 and 1 and was used as the prognostic score.
The cure probability, scale, and shape of the PCM were parameterized identically to Example 1, except in the context of a logistic kernel function rather than a Weibull kernel. Parameter estimates are listed in Table A2. The adequacy of the fit of this model for predicting the probability of cure is illustrated in Figure 3A of the article.
Estimates of Parametric Cure Model Parameters for Example 2: Hodgkin Disease
Example 3 (n = 317 patients with International Neuroblastoma Staging System stage 4 MYCN-unamplified neuroblastoma).
The primary end point for survival analysis was the minimum time to disease relapse, disease progression, or death from any cause. This is an appropriate end point for cure analyses because a minority of patients who experience relapse or progression are successfully re-treated. Patients who were failure free at last contact were censored at that time. See London et al (London WB, Castleberry RP, Matthay KK, et al. J Clin Oncol 23:6459-6465, 2005) for further details.
Age at diagnosis alone was used as a prognostic score in a PCM with a logistic kernel. The scale and shape parameters were
parameterized identically to Example 2. However, a special model for the cure probability was required to correctly estimate
the relationship between cure probability and age. 
Parameter estimates are listed in Table A3. The adequacy of the fit of this model for predicting the probability of cure is illustrated in Figure 4A of the article.
Estimates of Parametric Cure Model Parameters for Example 3: Neuroblastoma
Effect of Sample Variability on Estimates of the Optimum Cut Point and the Maximum Gain
A simulation was performed to study the sampling variability of the estimates of the optimum cut points and maximum gain. Specifically, 500 bootstrap samples were drawn from each of the example data sets discussed in the article. A PCM of the relevant type was fit to these data, and the optimum cut point and maximum gain were determined. This was repeated for the values of QAug = 0.5, 0.75, 0.85, 0.95, and 0.99. The bootstrap sample size was the same as the sample size for the original data set. For the Hodgkin's disease and neuroblastoma simulations, 1% to 2% of repetitions that clearly represented failure of the cure model fit algorithm to converge were excluded.
The box plots in Figures A2 through ⇓⇓⇓⇓A7 summarize these results. For the Example 1 data (Figs A2 and A3), the sampling distributions of both the optimum cut point and maximum gain are, for the most part, distinct and nonoverlapping among the different values of QAug. For the Hodgkin's disease example (Figs A4 and A5), the solutions are often degenerate in the sense that, for Q values from 0.5 to 0.85, the majority of cut point values are near 1 and the majority of maximum gain values are close to 0. However, for the lower morbidity treatment situations (Q = 0.95 and 0.99), the solutions are more dispersed and, for the most part, distinct and nonoverlapping for these two situations. The neuroblastoma example (Figs A6 and A7) is similar to Example 1, where the distributions of cut points and maximum gains are, for the most part, distinct and nonoverlapping among the Q values. Note that the median values of cut point and maximum gain in these simulations agree closely with the values reported in Table 2 of the article.
Distribution of optimum cut point for simulated Example 1 based on 500 bootstrap samples from the original data set with n = 1,000.
Distribution of maximum gain for simulated Example 1 based on 500 bootstrap samples from the original data set with n = 1,000.
Distribution of optimum cut point for Hodgkin's disease Example 2 based on 500 bootstrap samples from the original data set with n = 780.
Distribution of maximum gain for Hodgkin's disease Example 2 based on 500 bootstrap samples from the original data set with n = 780.
Distribution of optimum cut point for neuroblastoma Example 3 based on 500 bootstrap samples from the original data set with n = 317.
Distribution of maximum gain for neuroblastoma Example 3 based on 500 bootstrap samples from the original data set with n = 317.
Effect of Nonconstant QAug on Optimum Cut Points and Maximum Gain
In our analyses, we assumed that the morbidity quotient QAug was a constant unrelated to prognostic score. However, in some circumstances, it would be more realistic to assume that QAug differs for patients with different prognoses; for example, patients with more extensive disseminated disease who are treated with radiation therapy have radiation exposure to more sites than patients with localized disease. We performed a preliminary investigation of the sensitivity of our results to nonconstant QAug. Specifically, we considered the situation where QAug can vary with the score by different amounts but where the average QAug value was equal to 0.85, which was in the middle of the range that we considered in each example. The value of QAug for the lowest score (intercept) was set to 0.99, 0.95, 0.90, 0.85, and 0.80, and a log-linear change from this baseline was chosen so that the average was QAug = 0.85. The intercept of 0.85 corresponded to a zero slope and, therefore, a constant QAug.
Figure A8 summarizes these results. With the exception of the situations in Example 2 where QAug increases with prognostic score and the two situations in Example 3 where the values of QAug change by more than 40% over the age range, the cut point corresponding to the maximum does not differ importantly from the constant QAug situations that are considered in this example.
Effect of nonconstant morbidity quotient (QAug) on values for maximum gain and optimum count. The rows correspond to each of the three examples in the article. The left column shows the relationships between prognostic score and QAug that correspond to an average QAug of 0.85. The right column shows the gain function and cut points that correspond to the maximum gain for each of these situations.
Maximum Log-Rank χ2 Plots
Figure A9 shows the 1 degree of freedom log-rank χ2 values for difference prognostic score cut points. The maximum χ2 (minimum P value) cut points that are reported in the article were read from these figures.
One degree of freedom log-rank χ2 statistics (Kalbfleisch J, Prentice R. The Statistical Analysis of Failure Time Data. New York, NY, John Wiley and Sons, 1980), as a function of the prognostic score cut point, for all possible cut points that define a low-risk and high-risk group of patients. (A) Example 1 (n = 1,000 simulated patients). Maximum occurs at prognostic score of 0.56. (B) Example 2 (n = 780 patients with Hodgkin's disease). Maximum occurs at prognostic score of 0.67. (C) Example 3 (n = 317 patients with stage 4 MYCN-unamplified neuroblastoma). Maximum occurs at age of 1.4 years.
Footnotes
-
Supported by Children's Cancer Group (CCG) Grant No. CA 13539 and Children's Oncology Group (COG) Grant No. CA 98543. A complete listing of grant support for research conducted by the CCG and Pediatric Oncology Group before initiation of the COG grant in 2003 is available online at http://www.childrensoncologygroup.org/admin/grantinfo.htm.
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
- Received September 14, 2006.
- Accepted February 21, 2007.
