TY - JOUR
T1 - CreoleVal
T2 - Multilingual Multitask Benchmarks for Creoles
AU - Lent, Heather
AU - Tatariya, Kushal
AU - Dabre, Raj
AU - Chen, Yiyi
AU - Fekete, Marcell
AU - Ploeger, Esther
AU - Zhou, Li
AU - Armstrong, Ruth Ann
AU - Eijansantos, Abee
AU - Malau, Catriona
AU - Heje, Hans Erik
AU - Lavrinovics, Ernests
AU - Kanojia, Diptesh
AU - Belony, Paul
AU - Bollmann, Marcel
AU - Grobol, Loïc
AU - de Lhoneux, Miryam
AU - Hershcovich, Daniel
AU - Degraff, Michel
AU - Søgaard, Anders
AU - Bjerva, Johannes
N1 - Publisher Copyright:
© 2024 Association for Computational Linguistics.
PY - 2024/9/4
Y1 - 2024/9/4
N2 - Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research. While the genealogical ties between Creoles and a number of highly resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of anno-tated data. In this work we present CREOLEVAL, a collection of benchmark datasets spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an aggregate of novel development datasets for reading comprehension relation classification, and machine translation for Creoles, in addition to a practical gateway to a handful of preexisting benchmarks. For each benchmark, we conduct baseline experiments in a zero-shot setting in order to further ascertain the capabilities and limitations of transfer learning for Creoles. Ultimately, we see CREOLEVAL as an opportunity to empower research on Creoles in NLP and computational linguistics, and in general, a step towards more equitable language technology around the globe.
AB - Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research. While the genealogical ties between Creoles and a number of highly resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of anno-tated data. In this work we present CREOLEVAL, a collection of benchmark datasets spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an aggregate of novel development datasets for reading comprehension relation classification, and machine translation for Creoles, in addition to a practical gateway to a handful of preexisting benchmarks. For each benchmark, we conduct baseline experiments in a zero-shot setting in order to further ascertain the capabilities and limitations of transfer learning for Creoles. Ultimately, we see CREOLEVAL as an opportunity to empower research on Creoles in NLP and computational linguistics, and in general, a step towards more equitable language technology around the globe.
UR - http://www.scopus.com/inward/record.url?scp=85203430113&partnerID=8YFLogxK
U2 - 10.1162/tacl_a_00682
DO - 10.1162/tacl_a_00682
M3 - Article
AN - SCOPUS:85203430113
SN - 2307-387X
VL - 12
SP - 950
EP - 978
JO - Transactions of the Association for Computational Linguistics
JF - Transactions of the Association for Computational Linguistics
ER -