LLM Evaluation

Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation
A controlled pretraining study showing when and how data contamination inflates LLM evaluation scores.
Muhammed Yusuf Kocyigit, Eleftheria Briakou, Daniel Deutsch, Jiaming Luo, Colin Cherry, Markus Freitag
Evaluation Data Contamination in LLMs: How Do We Measure It and (When) Does It Matter?
ConTAM connects contamination detection metrics to downstream performance effects across models and benchmarks.
Aaditya K. Singh, Muhammed Yusuf Kocyigit, Andrew Poulton, David Esiobu, Maria Lomeli, Gergely Szilvasy, Dieuwke Hupkes
Western, Religious or Spiritual: An Evaluation of Moral Justification in Large Language Models
An evaluation of which moral traditions shape LLM justifications and where alignment can fail.
Eyup Engin Kucuk, Muhammed Yusuf Kocyigit