About
I am a research scientist at Google and a member of the Gemini large-scale pretraining core team. My work focuses on making language model evaluation more reliable: identifying and mitigating data contamination, improving evaluation signal-to-noise ratio, and understanding how training stages affect memorization and generalization. I also work on efficient, task-specific language models trained with synthetic data.
I completed my PhD in Computer Science at Boston University in 2025 under the supervision of Prof. Derry Wijaya. My doctoral research spanned LLM evaluation, machine translation, sentence representations, compositional generalization, and computational social science. I previously held research internships at Google, FAIR at Meta, and Amazon.
- LLM Evaluation
- Data Contamination and Decontamination
- Model Generalization and Memorization
- Pretraining Data Quality
- Efficient Language Models
-
PhD in Computer Science, 2020 - 2025
Boston University
-
MSc in Electrical and Electronics Engineering, 2017 - 2020
Bogazici University
-
BSc in Electrical and Electronics Engineering, 2012 - 2017
Bogazici University
Experience
Research and industry
- Analyze large-scale training runs for metric anomalies that may be caused by data contamination.
- Develop improved decontamination recipes and test-train overlap metrics to identify contamination early and increase evaluation signal-to-noise ratio.
- Study how training stages affect performance, memorization, and generalization.
- Improve task-specific, efficient small language models using synthetic data.
- Studied natural language evaluation, with an emphasis on data contamination and its effect on reported model performance.
- Improved sentence embeddings, compositional generalization evaluation, and cross-lingual representations through augmentation, multitask learning, and contrastive learning.
- Conducted temporal bias research funded by the 2021 Google Research Scholar Program.
- Led an Ottoman handwritten text recognition project that received $50K from the Scientific Research Council of Turkey.
- Quantified the effect of data contamination on LLM performance and evaluation reliability.
- Built scalable experimental frameworks to measure contamination impact and improve evaluation data selection.
- Host: Daniel Deutsch.
- Developed methods to define and detect contamination between evaluation sets and LLM pretraining data.
- Analyzed performance overestimation caused by contamination; the methodology and code were used in contamination analysis for Llama 3.1.
- Host: Dieuwke Hupkes.
- Evaluated and improved compositional generalization in semantic parsing models.
- Extended TMCD splits for more robust convergence and studied meta-learning and chain-of-thought prompting strategies.
- Host: Emilio Monti.
- Led business development, closed contracts worth more than $100K, and advised enterprise clients on AI strategy.
- Deployed customer segmentation, lifetime value, and churn models that doubled targeted-ad conversion rates for LC Waikiki.
- Improved illegal-electricity-use detection by 150% for DEDAS, with an estimated $500K business impact.
Publications
Research in language model evaluation, representation learning, and computational social science
Contact
The best way to reach me is by email.
- kocyigit@bu.edu
- Palo Alto, CA