Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation

Bar Iluz, Yanai Elazar, Asaf Yehudai, Gabriel Stanovsky


Abstract
Most works on gender bias focus on intrinsic bias — removing traces of information about a protected group from the model’s internal representation. However, these works are often disconnected from the impact of such debiasing on downstream applications, which is the main motivation for debiasing in the first place. In this work, we systematically test how methods for intrinsic debiasing affect neural machine translation models, by measuring the extrinsic bias of such systems under different design choices. We highlight three challenges and mismatches between the debiasing techniques and their end-goal usage, including the choice of embeddings to debias, the mismatch between words and sub-word tokens debiasing, and the effect on different target languages. We find that these considerations have a significant impact on downstream performance and the success of debiasing.
Anthology ID:
2024.emnlp-main.829
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14914–14921
Language:
URL:
https://aclanthology.org/2024.emnlp-main.829/
DOI:
10.18653/v1/2024.emnlp-main.829
Bibkey:
Cite (ACL):
Bar Iluz, Yanai Elazar, Asaf Yehudai, and Gabriel Stanovsky. 2024. Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14914–14921, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation (Iluz et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.829.pdf