We based this study on the National Lung Screening Trial (NLST).17 All participants from 33 medical centers underwent baseline screening (R0) and subsequently, a maximum of two rounds of repeat annual screening (R1 and R2) if no lung cancer was diagnosed. Follow-up was conducted through the end of 2009, with the longest follow-up duration >8 years.

We accessed data from the LDCT arm (delivery ID: NLST-503) and used inclusion criteria as follows: individuals aged 55–74 years at R0 with at least a 30 pack-year smoking history and smoking cessation <15 years. Exclusion criteria were lung cancer history; CT examination within 18 months before participation; and no positive findings during R0–R2, defined as ≥1 non-calcified pulmonary nodule or mass detected on LDCT.

Patient selection is depicted in Fig A.1. We included all (809) lung cancer patients who had ≥1 diameter record, which is the primary variable for planning follow-up testing. We retrospectively selected a sample (1000) of cancer-free pulmonary nodule patients to lower the burden in nodule selection, linkage, and quantification. Sample size determination is detailed in Methods A.1. Using a 2:1 ratio, we divided the 1809 selected patients into two patient cohorts, one for schema development (1206) and another for validation (603).

The study was approved by the institutional review board of Institute of Basic Medical Sciences, Chinese Academy of Medical Science. Patient consent was exempt as only publicly available data was used.

We used a joint modelling framework and considered two classes of outcome for implementing dynamic prediction18,19: time-to-event outcomes, defined as a lung cancer diagnosis and its time interval since the most recent test; and longitudinal outcomes, i.e., trajectories of nodule diameter. We applied this simple image biomarker for ease of interpretation and clinical use, as well as for meaningful comparisons of our approach with rule-based protocols that largely rely on diameter measurement.10,11

Model predictors were selected according to statistical or clinical significance. These included epidemiological information (age, obesity, family history of lung cancer, smoking pack-years) and nodule information (attenuation and margin), coded as binary variables where appropriate. Height or weight data for determining obesity were missing in 7 (0.4 %) patients; these were imputed according to the sex mean.

We developed a Cox proportional hazards model for a baseline screening scenario and joint models for a repeated screening scenario. Mathematical details are available in Methods A.3. The joint models first predicted the longitudinal outcome (diameter trajectory); this was then used, together with other predictors, to model the risk profiles regarding the time-to-event outcome. Between these sub-models, we used an association structure to account for the diameter measured at the present test and its rate of change over time; both are clinically important in determining cancer risk.20 A unique advantage of this approach is smoothing of nodule diameter measurement error, which can be as high as 25 % in LDCT screening.21

We selected two risk cut-offs to optimize accuracy in decisions about timing of the upcoming follow-up test. We based these choices on the analysis of a time-dependent receiver operating curve.22,23 Specifically, we selected one risk cut-off that allowed for sensitivity (t= 3 months) ≥0.95, and another cut-off that allowed for specificity (t= 12 months) ≥0.95. These cut-offs were then used to classify patients (per each screening round) as having high, middle, or low risk, whereupon recommendations for a follow-up test interval of 0 months (i.e., immediate work-up), 3 months, or 12 months (i.e., annual repeat screening) were made. The ≥0.95 criterion was intended to control delayed diagnosis (defined as false recommendation of annual repeat screening for those who develop lung cancer within 3 months) and over-testing (defined as false recommendation of immediate work-up for cancer-free patients) to a small probability (<0.05).

To demonstrate strengths and potential weaknesses of the proposed schema, we created a benchmark with two nodule management protocols that are in current use: the NCCN guideline (2022 V2)10 and the Lung CT Screening Reporting & Data System (Lung-RADS 2022).11 We examined delayed diagnosis and over-testing rates following these rule-based protocols versus our personalized schema in the validation cohort. We also investigated which lung cancer patient subgroups could benefit most from a personalized schema in terms of shorter delay in diagnosis.

Because of a right-skewed distribution of the nodule diameter, we conducted a natural logarithm transform before using this longitudinal outcome. We estimated parameters of the joint models using a Bayesian method, implemented with a Markov chain Monte Carlo algorithm (1 chain, 11,000 interactions with 1000 burn-ins discarded). We assessed model performance using time-dependent accuracy metrics and estimated 95 % confidence intervals (CIs) using a 1000-sample bootstrap approach.

We performed a log-rank test to examine between-group differences among high-, mid- and low-risk strata. We drew a contingency table to tabulate recommendations on the time target of follow-up testing and ground truth of the time-to-event outcome, whereupon rates of delayed diagnosis and over-testing (as defined above) were calculated. We used a paired-samples McNemar exact probability method to test for statistical significance of these rates.

We considered a two-sided p-value <0.05 to indicate statistical significance. We performed the analyses using SAS 9.4 (SAS Institute Inc., Cary, NC, USA) and R 4.1.2 with packages “JMbayes2 0.2–8”, “riskRegression 2022.09.23”, “tdROC 1.0” and “survminer 0.4.9” (R Project for Statistical Computing, Vienna, Austria).

Table 1 presents characteristics of the included patients. The mean age at R0 was 62.7 years; 58.7 % were men; 50.5 % had an associate's, bachelor's, or higher education degree; 23.7 % were obese; and 24.9 % of patients had a family history of lung cancer. Participants had a median 52.5 pack-year smoking history with a median starting age of 16 years, and half (51.4 %) had not quit smoking before participation. Median follow-up duration was 2197 days (6 years).

Of 809 patients diagnosed with lung cancer, the median time to diagnosis was 735 days (2 years); the range was as wide as 4–2499 days. High cancer heterogeneity was also demonstrated in diverse pathological types (9.6 % small cell, 49.1 % adenocarcinoma, 21.1 % squamous cell, 19.9 % other) and stages (e.g., 71.4 % stages IA-IIIA, 26.8 % stages IIIB-IV), suggesting a need for personalized optimization of diagnostic testing.

The above patient characteristics did not differ between the cohorts used for model development and schema validation, except for negligible differences in mean age (62.5 vs. 63.2 years, p = 0.0135) and median follow-up duration (2212 vs. 2142 days, p = 0.0010).

The multi-stage models are summarized in Table A.1, and were used to predict onset of lung cancer within a time interval of interest. Results of time-dependent predictive performance of the models are available in Table A.2.

Validation results: the area under the receiver operating curve (AUC) (t= 3 months) was 0.879 (95 % CI: 0.842, 0.917) at R0 and 0.845 (95 % CI: 0.801, 0.892) at R1–R2; the AUC (t= 12 months) was 0.867 (95 % CI: 0.827, 0.894) for R0 and 0.807 (0.765, 0.948) for R1–R2. These were comparable to the development cohort, thus demonstrating the validity of the model performance.

Risk cut-offs selected according to the development cohort yielded high sensitivity (t= 3 months): 0.983 (95 % CI: 0.946, 1.000) for R0; 0.957 (95 % CI: 0.901, 1.000) for R1–R2, and moderately high specificity (t= 12 months): 0.909 (95 % CI: 0.881, 0.938) for R0; 0.936 0.936 (95 % CI: 0.914, 0.958) for R1–R2 in the validation cohort.

In Fig 1, we present risk strata according to the selected cut-offs. In the development and validation cohorts, patients determined as high-, mid- or low-risk had significantly different curves for the cumulative risk of lung cancer (p < 0.0001 at each screening round).

Fig. 1.

Risk stratification effectiveness.

(0.49MB).

We compared the personalized schema with the NCCN and Lung-RADS protocols. The results obtained from the validation cohort are shown in Table 2.

In R0, the three protocols performed equally well at controlling delayed diagnosis (rates: 1.7% vs. 6.9% vs. 1.7 % following NCCN, Lung-RADS, and our schema) and over-testing (5.6% vs. 5.6% vs. 4.9 %); all p > 0.05.

In R1–R2, the personalized schema outperformed the rule-based protocols. The rate of delayed diagnosis associated with the NCCN, Lung-RADS, and our schema was 16.7 % versus 12.5 % versus 8.3 % in R1, and 18.2 % versus 18.2 % versus 0.0 % in R2; the rate of over-testing was 16.0 % versus 11.7 % versus 5.3 % in R1, and 8.3 % versus 7.3 % versus 2.6 % in R2 (statistical significance shown in Table 2).

Among 470 available decision time points for 293 patients with lung cancer in the validation cohort, 232 (49.4 %) and 207 (44.0 %) follow-up testing recommendations were consistent between NCCN and the personalized schema and between Lung-RADS and the personalized schema, respectively. Earlier test recommendation was less frequent using NCCN versus the personalized schema: 98 (20.9 %) versus 140 (29.8 %); p = 0.0065; or using Lung-RADS versus the personalized schema: 107 (22.8 %) versus 156 (33.2 %); p = 0.0025. Subgroup analyses (Fig 2) identified several subgroups of patients with lung cancer who were more likely to benefit from the personalized schema than the NCCN protocol and the Lung-RADS protocol (patients aged ≥65 years, women, former smokers, and patients with part-solid or non-solid attenuation, adenocarcinoma cancer, and stage IIIB-IV; all p < 0.05).

Fig. 2.

Subgroup analysis of patients with lung cancer.

*p < 0.05 or ** p < 0.01 indicates statistical significance in a paired-samples test.

Lung-RADS, Lung CT Screening Reporting & Data System; NCCN, National Comprehensive Cancer Network.

(0.46MB).

We provide a web application (available at http://www.biostatpumc.com:3838/pred_risk_2.Rmd) for computer or cell phone users to check and update their follow-up recommendations generated by the personalized schema. We illustrate its use in two example cases from our institute and preliminarily examine applicability in NLST-ineligible patients (Fig A.2.).

The schema can be adapted according to patient and physician preferences. Tables A.3–A.5 illustrate that decreasing the criteria of sensitivity(t) or specificity(t) (e.g., from ≥0.95 to ≥0.90) would result in more conservative recommendations (i.e., fewer recommendations for immediate work-up and more for annual screening); in contrast, increasing these criteria would mean more aggressive recommendations.