The Hybrid Imaging System Laboratory (HISLab) from the Smart Medical Information Research Center published a manuscript entitled “Review of deep learning for photoacoustic imaging” based on all the research using deep learning to solve various problems in photoacoustic imaging in recent years. This article has been accepted by the top journal Photoacoustics (CAS tier-1, IF: 5.870).
Photoacoustic (PA) imaging combines the advantages of high optical imaging contrast and deep penetration depth of ultrasound imaging, providing a complementary imaging method for existing medical imaging technologies. Although PA imaging is still considered a relatively new imaging method when compared to other medical imaging modalities, the combination with deep learning (DL) shows a large potential to quickly boost this blooming field. More specifically, our review article elaborates on applications of DL at every step of the entire PA imaging workflow:
Acquisition of high-quality PA image from sensorial data; Disease segmentation, classification and detection with reconstructed PA images; Improvement of functional imaging capability without exogenous contrast (i.e. negative effects of oxygen saturation); Assistance of pre-clinical and clinical application with PA imaging.
The number of research publications using DL for those aforementioned challenges has been growing steadily in the past few years, as shown on our statistical analysis (Fig.1). A more in depth look on those papers can give us a further insight on which sub-category these papers belong to, with Image reconstruction being the most popular application. The number of papers classified by category is given in Table 1, and a further analysis of the relationship between different tasks of PA imaging and DL architecture is found in Fig. 2.
Fig. 1. The Number of papers on DL in PA imaging in recent years. For 2020 year’s data, only the quantity before September 15 is calculated.
Table 1. The Number of papers on DL in PA imaging based on tasks.
Fig. 2. A Schematic diagram describing the PA imaging tasks and DL architectures.
From the perspective of task classification, we review all the research currently retrieved under each category from six tasks: photoacoustic image reconstruction, quantitative photoacoustic imaging, photoacoustic image detection, photoacoustic image classification, photoacoustic image segmentation, and photoacoustic assisted diagnosis. Finally, we briefly summarize the available data sets for DL within PA. Although limited by the status quo whenever a PA imaging equipment is not clinically available, or when the number of open datasets is small, we believe that these can still help peers conduct high-quality research based on these datasets. Existing datasets can be divided into two categories: one is physiologically digital phantom, and the other is experimental PA imaging data.
ShanghaiTech University is the first author affiliation of this research. Changchun Yang, a 2018 master student, and Hengrong Lan, a 2019 doctoral student, are the co-first authors of the paper, and professor Fei Gao is the corresponding author. This research was supported by the National Natural Science Foundation of China, Natural Science Foundation of Shanghai, and Start-up grant of ShanghaiTech University.
Preprint link: https://arxiv.org/ftp/arxiv/papers/2008/2008.04221.pdf
In the past two years, HISLab has published much research on DL for PA. The summary is as follows:
(Changchun Yang, et al, MICCAI 2019) EDA-net: Dense aggregation of deep and shallow information achieves quantitative photoacoustic blood oxygenation imaging deep in human breast
(Hengrong Lan, et al, MICCAI 2019) Ki-GAN: Knowledge Infusion Generative Adversarial Network for Photoacoustic Image Reconstruction In Vivo
(Changchun Yang, et al, EMBC 2019）Accelerated Photoacoustic Tomography Reconstruction via Recurrent Inference Machines
（Hengrong Lan, et al, EMBC 2019）Reconstruct the Photoacoustic Image Based On Deep Learning with Multi-frequency Ring-shape Transducer Array
（Hengrong Lan, et al, EMBC 2019）Hybrid Neural Network for Photoacoustic Imaging Reconstruction
(Hengrong Lan, et al, IUS 2019) Solving Limited-view Problem by Multi-frequency Photoacoustic Characteristic Synthesis via Deep Learning
（Changchun Yang, et al, ISBI 2019）Quantitative photoacoustic blood oxygenation imaging using deep residual and recurrent neural network
（Jiayao Zhang, et al, IEEE Access 2019）Photoacoustic image classification and segmentation of breast cancer: a feasibility study
（Hengrong Lan, et al, IEEE JSTQE 2019）Photoacoustic Classification of Tumor Model Morphology Based on Support Vector Machine: A Simulation and Phantom Study
（Yaxin Ma, et al, EMBC 2020）Human Breast Numerical Model Generation Based on Deep Learning for Photoacoustic Imaging
（Hengrong Lan, et al, Photoacoustics 2020）Y-Net: Hybrid deep learning image reconstruction for photoacoustic tomography in vivo
(Changchun Yang, et al, AAAI 2021 in revision) Deep Learning Enables Robust and Precise Light Focusing on Treatment Needs
(Tengbo Lyu, et al, ISBI 2021 under review) 3D Simulation of human skin vascular for quantitative photoacoustic computed tomography
(Jiadong Zhang, et al, ISBI 2021 under review) Limited-view photoacoustic image reconstruction with dual domain inputs based on mutual information
(Hengrong Lan, Daohuai Jiang, et al, IEEE TBME under review) Real-Time Photoacoustic Tomography System via Single Data Acquisition Channel
(Hengrong Lan, Changchun Yang, et al, IEEE TMI under review) Beyond Supervised Reconstruction: Limit-view Compensation and Artifacts Removal for Photoacoustic Tomography
沪公网安备 31011502006855号
