Predicting treatment response from longitudinal images using multi-task deep learning

Cheng Jin, Heng Yu, Jia Ke, Peirong Ding, Yongju Yi, Xiaofeng Jiang, Xin Duan, Jinghua Tang, Daniel T. Chang, Xiaojian Wu (), Feng Gao () and Ruijiang Li ()
Additional contact information
Cheng Jin: Stanford University School of Medicine
Heng Yu: Stanford University School of Medicine
Jia Ke: Sun Yat-sen University
Peirong Ding: Sun Yat-sen University Cancer Center
Yongju Yi: Sun Yat-sen University
Xiaofeng Jiang: Sun Yat-sen University
Xin Duan: Sun Yat-sen University
Jinghua Tang: Sun Yat-sen University Cancer Center
Daniel T. Chang: Stanford University School of Medicine
Xiaojian Wu: Sun Yat-sen University
Feng Gao: Sun Yat-sen University
Ruijiang Li: Stanford University School of Medicine

Nature Communications, 2021, vol. 12, issue 1, 1-11

Abstract: Abstract Radiographic imaging is routinely used to evaluate treatment response in solid tumors. Current imaging response metrics do not reliably predict the underlying biological response. Here, we present a multi-task deep learning approach that allows simultaneous tumor segmentation and response prediction. We design two Siamese subnetworks that are joined at multiple layers, which enables integration of multi-scale feature representations and in-depth comparison of pre-treatment and post-treatment images. The network is trained using 2568 magnetic resonance imaging scans of 321 rectal cancer patients for predicting pathologic complete response after neoadjuvant chemoradiotherapy. In multi-institution validation, the imaging-based model achieves AUC of 0.95 (95% confidence interval: 0.91–0.98) and 0.92 (0.87–0.96) in two independent cohorts of 160 and 141 patients, respectively. When combined with blood-based tumor markers, the integrated model further improves prediction accuracy with AUC 0.97 (0.93–0.99). Our approach to capturing dynamic information in longitudinal images may be broadly used for screening, treatment response evaluation, disease monitoring, and surveillance.

Date: 2021
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://www.nature.com/articles/s41467-021-22188-y Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-22188-y

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-021-22188-y

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().