Fairness · DIF · Measurement invariance

Does this test measure everyone the same way?

A test is fair when two people with the same level of the skill have the same chance of answering an item correctly, whatever group they belong to (AERA, APA, & NCME, 2014). When that fails for a specific item, psychometricians call it differential item functioning (DIF). When it fails for a whole scale, so that scores cannot be compared across countries, languages or genders, we say the scale lacks measurement invariance.

DIF is different from a group simply scoring lower. Groups can differ in ability. DIF means the item itself behaves differently once ability is held constant, often because of wording, cultural context or content that one group has seen more than the other.

See DIF in one item

Two-parameter IRT model
Shifts the item for group B. Equal shift at every ability level is uniform DIF.
Changes the slope for group B, so the gap grows or flips across ability. That is non-uniform DIF.
Uniform DIF
P(correct) at θ = 0, group A—
P(correct) at θ = 0, group B—
Unsigned area between curves—

Two-parameter logistic model (Birnbaum, 1968); group A has a = 1.2, b = 0. The area is the unsigned area between the two curves, computed numerically over θ = −3 to +3. Raju's (1988) area measure is defined over the whole θ range, so this value is its restricted-range version. The uniform / non-uniform distinction follows Mellenbergh (1982).

Run a Mantel–Haenszel DIF analysis on your own data →

The questions my research asks

Research lines

Which items are unfair, and how do we find them?

Comparing DIF detection methods for dichotomous and polytomous items, from logistic regression and Poly-SIBTEST to lasso-penalized IRT, machine learning and generative AI.

Can scores be compared across many groups?

Testing measurement invariance in large-scale assessments such as PISA with alignment optimization and Bayesian approximate invariance, where dozens of countries are compared at once.

Where does DIF come from?

Explaining DIF with mixture Rasch and IRT-MIMIC models, covariates and latent classes, and asking whether DIF changes across ability levels.

Publications and presentations

Evidence

Journal articles

2023

Comparison of GPCMlasso and alignment methods in detecting differential item functioning

Sirganci, G. · International Journal of Eurasian Education and Culture, 8(23), 2270–2299

Compares a lasso-penalized generalized partial credit model with alignment optimization for flagging DIF in polytomous items.

2023

A machine learning approach to assess differential item functioning of the PISA 2018 ICT Engagement Questionnaire

Sirganci, G. · International Journal of Curriculum and Instruction, 15(3), 2079–2093

Uses machine learning to examine DIF in a PISA 2018 student questionnaire on engagement with information and communication technology.

2020

Measurement invariance testing with alignment method: Many groups comparison

Sırgancı, G., Uyumaz, G., & Yandı, A. · International Journal of Assessment Tools in Education, 7(4), 657–673

Applies the alignment method to test measurement invariance when many groups are compared at the same time.

2020

Detecting differential item functioning by using sequential logistic regression and Poly-SIBTEST methods

Sırgancı, G., & Çakan, M. · Journal of the Faculty of Education, Bolu Abant İzzet Baysal University, 20(1), 705–717 · in Turkish

Compares two DIF detection methods for polytomously scored items using a PISA student questionnaire.

Under review

2026

Utility of off-the-shelf generative AIs for DIF detection

Patarapichayatham, C., Sirganci, G., & Kamata, A. · Under review, Educational Assessment · presented at NCME 2026, Los Angeles

Tests whether widely available generative AI tools can identify items that function differently across groups.

2026

Is creativity measured fairly? Ability-dependent differential item functioning by gender in the PISA 2022 Creative Thinking Assessment

Sirganci, G., & Meşe, C. · Under review, The Journal of Creative Behavior

Conference presentations

2025

Modeling variational DIF for different ability levels with a moderated mediation approach

Sirganci, G., Kara, Y., Fong, M., & Kamata, A. · NCME Annual Meeting, Denver

2025

Model-based detection of passage bias in oral reading fluency assessments

Sirganci, G., Kara, Y., Nese, J. F. T., & Kamata, A. · NCME Annual Meeting, Denver

2025

Exploring fairness of next-generation items: Differential item action analysis on process data

Kara, Y., Sirganci, G., Qiao, X., Kamata, A., Sumsong, S., & Fong, M. · NCME Annual Meeting, Denver

2021

Measurement invariance testing with many groups: A comparison of Bayesian structural equation modeling and alignment optimization

Sirganci, G., Uyumaz, G., & Kamata, A. · 7th International Congress on Measurement and Evaluation in Education and Psychology, Ankara

2020

The effect of covariate variables on the determination of differential item functioning using the mixture Rasch model

Sirganci, G., & Çokluk Bökeoğlu, Ö. · AERA Annual Meeting, San Francisco (accepted; meeting canceled) · based on my doctoral dissertation, Ankara University, 2019

Book chapters · in Turkish

2022

Measurement invariance in the scale development process

In M. Acar Güvendir & Y. Özer Özkan (Eds.), Tüm yönleriyle ölçek geliştirme süreci (pp. 155–182). Pegem Akademi

2022

New approaches to the validity of multiple-group comparisons in large-scale assessments

In S. Karabatak (Ed.), Eğitim & Bilim 2022-IV (pp. 95–116). Efe Akademik Yayıncılık

Funded projects · Principal investigator

2017

Detecting differential item functioning for groups and latent classes simultaneously based on mixture IRT and IRT-MIMIC models

TÜBİTAK International Doctoral Research Fellowship · Southern Methodist University, 2017–2018

2020

Determination of approximate measurement invariance with Bayesian alignment optimization

Yozgat Bozok University Scientific Research Projects Program, 2020–2022

References