Preliminary Results of Applying Transformers to Geoscience and Earth Science Data

J. Delgado, U. Ebreso, Y. Kumar, J. J. Li, P. Morreale

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Transformers with neural networks (NN) have become a dominant technology in AI/ML to achieve better accuracy in classification and prediction. We experimentally compare transformer-based NN and convolutional NN using various applications such as pollen detection and weather ITCZ prediction. Using machine learning in geoscience, earth sciences, and other natural sciences is not entirely new, but applying transformers to data from these environments creates many new opportunities. We apply Facebook's detection transformer (DETR) neural network, developed in 2020, to pollen and weather data to detect forty-four types of pollen and classify earth snapshots into two categories: having vs. not having the phenomena of double Intertropical Convergence Zones (ITCZs) in them. As we conduct our trials, we observe and document how the model performs on both datasets, detect the biases present in each layer of the network, and mitigate them as we tune the model and improve classification results even further.

Original languageEnglish
Title of host publicationProceedings - 2022 International Conference on Computational Science and Computational Intelligence, CSCI 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages284-288
Number of pages5
ISBN (Electronic)9798350320282
DOIs
StatePublished - 2022
Event2022 International Conference on Computational Science and Computational Intelligence, CSCI 2022 - Las Vegas, United States
Duration: 14 Dec 202216 Dec 2022

Publication series

NameProceedings - 2022 International Conference on Computational Science and Computational Intelligence, CSCI 2022

Conference

Conference2022 International Conference on Computational Science and Computational Intelligence, CSCI 2022
Country/TerritoryUnited States
CityLas Vegas
Period14/12/2216/12/22

Keywords

  • computer science
  • deep learning
  • DETR
  • earth science
  • geoscience
  • neural networks
  • transformers

Fingerprint

Dive into the research topics of 'Preliminary Results of Applying Transformers to Geoscience and Earth Science Data'. Together they form a unique fingerprint.

Cite this