Weakly Unpaired Image Translation from Hematoxylin and Eosin Staining Image to Immunohistochemistry Staining Image

Kuan Huang, Yifei Cheng, Qiang Gao, Bing Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Histopathological imaging is one of the most critical disease diagnosis tools. Hematoxylin and eosin (H&E) staining images are routinely acquired and show all cells in the tissue. In contrast, immunohistochemistry (IHC) staining of cell type markers reveals spatial information of specific cell types of interest, and their use is limited by more complex laboratory processing and high cost. As an example, CD3 IHC images provide critical spatial information on the tumor-infiltrating lymphocytes (TILs), but for most tumor specimens, only H&E images are available. It is of high interest to predict the localization of CD 3 + cells based on H&E images without performing CD3 IHC, which may be achieved through image translation. Deep learning methods have been developed to perform image translation from exactly matched image pairs, but it is hard to acquire matched pairs of H&E and CD3 IHC images because they are typically from close but different slices of one tissue. In this paper, we propose a Registration Generative Adversarial Network (R-GAN) to translate H&E images into CD3 IHC images using weakly unpaired training samples. A registration module and a novel patch-level L1 loss function are incorporated into Pix2PixGAN to address the challenges arising from weakly unpaired training samples. We conduct benchmarking experiments on a dataset constructed with tissue microarrays (TMAs) of liver cancer tissues, which contains H&E images and the corresponding CD3 IHC images for 1,073 tissue cores. The proposed R-GAN outperforms two commonly used translation methods in terms of both image quality and style relevance metrics.

Original languageEnglish
Title of host publicationProceedings - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
EditorsDonald Adjeroh, Qi Long, Xinghua Shi, Fei Guo, Xiaohua Hu, Srinivas Aluru, Giri Narasimhan, Jianxin Wang, Mingon Kang, Ananda M. Mondal, Jin Liu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1013-1019
Number of pages7
ISBN (Electronic)9781665468190
DOIs
StatePublished - 2022
Event2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022 - Las Vegas, United States
Duration: 6 Dec 20228 Dec 2022

Publication series

NameProceedings - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022

Conference

Conference2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022
Country/TerritoryUnited States
CityLas Vegas
Period6/12/228/12/22

Keywords

  • CD3 IHC image
  • deep learning
  • H&E image
  • image translation
  • weakly supervised learning

Fingerprint

Dive into the research topics of 'Weakly Unpaired Image Translation from Hematoxylin and Eosin Staining Image to Immunohistochemistry Staining Image'. Together they form a unique fingerprint.

Cite this