TY - GEN
T1 - Weakly Unpaired Image Translation from Hematoxylin and Eosin Staining Image to Immunohistochemistry Staining Image
AU - Huang, Kuan
AU - Cheng, Yifei
AU - Gao, Qiang
AU - Zhang, Bing
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Histopathological imaging is one of the most critical disease diagnosis tools. Hematoxylin and eosin (H&E) staining images are routinely acquired and show all cells in the tissue. In contrast, immunohistochemistry (IHC) staining of cell type markers reveals spatial information of specific cell types of interest, and their use is limited by more complex laboratory processing and high cost. As an example, CD3 IHC images provide critical spatial information on the tumor-infiltrating lymphocytes (TILs), but for most tumor specimens, only H&E images are available. It is of high interest to predict the localization of CD 3 + cells based on H&E images without performing CD3 IHC, which may be achieved through image translation. Deep learning methods have been developed to perform image translation from exactly matched image pairs, but it is hard to acquire matched pairs of H&E and CD3 IHC images because they are typically from close but different slices of one tissue. In this paper, we propose a Registration Generative Adversarial Network (R-GAN) to translate H&E images into CD3 IHC images using weakly unpaired training samples. A registration module and a novel patch-level L1 loss function are incorporated into Pix2PixGAN to address the challenges arising from weakly unpaired training samples. We conduct benchmarking experiments on a dataset constructed with tissue microarrays (TMAs) of liver cancer tissues, which contains H&E images and the corresponding CD3 IHC images for 1,073 tissue cores. The proposed R-GAN outperforms two commonly used translation methods in terms of both image quality and style relevance metrics.
AB - Histopathological imaging is one of the most critical disease diagnosis tools. Hematoxylin and eosin (H&E) staining images are routinely acquired and show all cells in the tissue. In contrast, immunohistochemistry (IHC) staining of cell type markers reveals spatial information of specific cell types of interest, and their use is limited by more complex laboratory processing and high cost. As an example, CD3 IHC images provide critical spatial information on the tumor-infiltrating lymphocytes (TILs), but for most tumor specimens, only H&E images are available. It is of high interest to predict the localization of CD 3 + cells based on H&E images without performing CD3 IHC, which may be achieved through image translation. Deep learning methods have been developed to perform image translation from exactly matched image pairs, but it is hard to acquire matched pairs of H&E and CD3 IHC images because they are typically from close but different slices of one tissue. In this paper, we propose a Registration Generative Adversarial Network (R-GAN) to translate H&E images into CD3 IHC images using weakly unpaired training samples. A registration module and a novel patch-level L1 loss function are incorporated into Pix2PixGAN to address the challenges arising from weakly unpaired training samples. We conduct benchmarking experiments on a dataset constructed with tissue microarrays (TMAs) of liver cancer tissues, which contains H&E images and the corresponding CD3 IHC images for 1,073 tissue cores. The proposed R-GAN outperforms two commonly used translation methods in terms of both image quality and style relevance metrics.
KW - CD3 IHC image
KW - deep learning
KW - H&E image
KW - image translation
KW - weakly supervised learning
UR - http://www.scopus.com/inward/record.url?scp=85146637050&partnerID=8YFLogxK
U2 - 10.1109/BIBM55620.2022.9995520
DO - 10.1109/BIBM55620.2022.9995520
M3 - Conference contribution
AN - SCOPUS:85146637050
T3 - Proceedings - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
SP - 1013
EP - 1019
BT - Proceedings - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
A2 - Adjeroh, Donald
A2 - Long, Qi
A2 - Shi, Xinghua
A2 - Guo, Fei
A2 - Hu, Xiaohua
A2 - Aluru, Srinivas
A2 - Narasimhan, Giri
A2 - Wang, Jianxin
A2 - Kang, Mingon
A2 - Mondal, Ananda M.
A2 - Liu, Jin
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
Y2 - 6 December 2022 through 8 December 2022
ER -