Predicting population-level rwTTD for lung cancer and advanced head and neck cancer treatment using pembrolizumab
We tested the above algorithm in the context of lung cancer treatment and head and neck cancer treatment using pembrolizumab (for cohort selection please see Methods ). rwTTD, the duration between the first dosing to the last administration are defined by the following three criteria: a. switch to a different treatment: This is an event point, and rwTTD is defined between the first dosing to the last available administration. b. death: This is also an event point, and rwTTD is defined between the first dosing to the death date. c. With a gap >= 120 days between last known administration and last known activity: This is an event point, and rwTTD is defined between the first dose to the last known available administration. If none of the above happens, the data point is considered as censored (no data after last administration date or the gap is < 120 days).
We carried out three evaluation experiments (Fig. S14 ). The first two experiments used advanced lung cancer data and examined the performance of prediction rwTTD in this homogeneous population. In the first experiment, we randomly selected the cutoff time between the first dose time and the last contact time point (let it be censoring time or termination time), and uniformly and randomly selected a time in between as the cutoff time. All information prior to the cutoff date (observation window) is used to extract feature data (seeMethods ). The time between the cutoff time and the last contact time point is the time used to calculate the rwTTD curve. Here we are evaluating the ability of predicting rwTTD given a random length of observations. In the second experiment, the cutoff date is consistently 30 days after the first dose. Thus, we are evaluating how well we can predict given 30 days of observation data. The third experiment was trained with lung cancer data with a random cutoff and tested with head and neck cancer. Under these three sceneria, we evaluated the performance of predicting the rwTTD curve.
Overall, we found strong performance for rwTTD in both homogeneous population and cross-disease prediction tasks (Fig. 5a-c, Fig. S15-17 ). We observed an average 14.12% 13.15%, 31.59% percent absolute error rate for random cutoff cross-validation, 30 day cutoff cross-validation, and cross-disease prediction, respectively. The cumulative error rates are 23.78%, 18.43%, 34.15% respectively (Fig. 5d ). Of note, cross-disease errors are expected to be higher as the patient populations are distinct and can respond to the drug differently. We further examined the performance at 6, 12, 18, and 24 months, and error rates remained stable within this range (Fig. 5e ). In Particular, we observed a very low average 50% terminated ratio date prediction, for only 82.90, 105.33, 81.90 for random cutoff cross-validation, 30 day cutoff cross-validation, and cross-disease respectively (Fig. 5f ). These results support strong performance in real world data even when the model is delivered to data derived from a different population but share certain similarities in the EMR data that was collected.