Fairness-Aware Reinforcement Learning for Personalized Educational Assessment: Counterfactual Policy Regularization under Measurement Bias
Keywords:
adaptive assessment, reinforcement learning, algorithmic fairness, computerized adaptive testing, counterfactual consistency, educational measurementAbstract
Personalized educational assessment can reduce test burden by adapting item selection to a learner’s evolving knowledge state, but a policy that optimizes efficiency alone may convert measurement bias into systematic differences in question difficulty and test length across demographic groups. This paper formulates adaptive assessment as a constrained sequential decision problem and introduces CF-Q, a lightweight fairness-aware reinforcement-learning method that combines (i) group-wise calibration of a biased initial ability proxy, (ii) counterfactual Q-consistency that couples value estimates under protected-attribute swaps, and (iii) a dual variable that penalizes residual action disparity. The method is evaluated in a controlled three-skill item-response simulation designed to isolate algorithmic behavior under a protected-group measurement barrier. Across three independent seeds and 2,700 held-out simulated learners, CF-Q reduced the conditional difficulty-exposure gap from 0.267 to 0.073 (72.8%) relative to efficiency-only Q-learning, while maintaining similar ability-estimation error (RMSE 0.413 versus 0.407). It also reduced the conditional test-length gap from 0.467 to 0.227. The results show that fairness constraints can be integrated directly into the adaptive policy rather than applied only as a post-hoc audit, yielding a transparent accuracy–efficiency–fairness trade-off suitable for educational assessment research.
Downloads
References
[1] Bian, M., Wang, Y., & Lin, Y. (2024). Explainable supplier risk prediction for critical telecommunications infrastructure procurement: An empirical XGBoost–SHAP framework. Social Science and Management, 1(1), 64-77.
[2] Shang, W., & Chen, T. (2024). PrefixKD-Edge: Real-time locomotion-intent prediction with prefix self-distillation on lightweight temporal networks. In-novation and Technology Studies, 1(1), 30–38.
[3] Rhee, M., Zou, J., & Qin, Y. (2024). CLARITY: A closed loop framework for duplicate aware failure prediction and budgeted automated remediation in cloud infrastructure. AI and Data Science Journal, 1(1), 76-85.
[4] Lin, Y., Wang, Y., & Bian, M. (2024). XOR EW: An explainable and cost aware artificial intelligence framework for operational risk early warning in fi-nancial institutions. AI and Data Science Journal, 1(1), 94-102.
[5] Zhao, W., & Wang, B. (2022). CAPA: A prediction‑driven autoscaling framework for SLO‑aware cloud‑native machine learning inference. Journal of Computer Science and Electrical Engineering, 4(1), 8-16.
[6] Wang, Y., & Zhang, C. (2023). AERIS: Adaptive executor scoped resource management for privacy preserving data processing in Apache Spark. Eurasia Journal of Science and Technology, 5(2), 25-33.
[7] Y. Cheng, ‘When cognitive diagnosis meets computerized adaptive testing: CD-CAT,’ Psychometrika, vol. 74, pp. 619–632, 2009.
[8] A. T. Corbett and J. R. Anderson, ‘Knowledge tracing: Modeling the acqui-sition of procedural knowledge,’ User Modeling and User-Adapted Interaction, vol. 4, pp. 253–278, 1994.
[9] C. Piech et al., ‘Deep knowledge tracing,’ in Advances in Neural Information Processing Systems 28, 2015.
[10] S. Pandey and G. Karypis, ‘A self-attentive model for knowledge tracing,’ arXiv:1907.06837, 2019.
[11] A. Ghosh, N. Heffernan, and A. S. Lan, ‘Context-aware attentive knowledge tracing,’ in Proc. 26th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, 2020, pp. 2330–2339.
[12] J. Kuzilek, M. Hlosta, and Z. Zdrahal, ‘Open University Learning Analytics dataset,’ Scientific Data, vol. 4, Art. no. 170171, 2017, doi: 10.1038/sdata.2017.171.
[13] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018.
[14] V. Mnih et al., ‘Human-level control through deep reinforcement learning,’ Nature, vol. 518, pp. 529–533, 2015.
[15] J. Achiam, D. Held, A. Tamar, and P. Abbeel, ‘Constrained policy optimiza-tion,’ in Proc. 34th Int. Conf. Machine Learning, PMLR 70, 2017, pp. 22–31.
[16] S. Jabbari, M. Joseph, M. Kearns, J. Morgenstern, and A. Roth, ‘Fairness in reinforcement learning,’ in Proc. 34th Int. Conf. Machine Learning, PMLR 70, 2017, pp. 1617–1626.
[17] M. Joseph, M. Kearns, J. H. Morgenstern, and A. Roth, ‘Fairness in learning: Classic and contextual bandits,’ in Advances in Neural Information Pro-cessing Systems 29, 2016.
[18] Q. Fang and W. Liu, ‘HARLA-ED: Resolving information asymmetry and enhancing algorithmic symmetry in intelligent educational assessment via hybrid reinforcement learning,’ Symmetry, vol. 18, Art. no. 58, 2026, doi: 10.3390/sym18010058.
[19] B. Tang, S. Li, and C. Zhao, ‘Reinforcement learning framework for com-puterized adaptive testing using multi armed bandit approach,’ Scientific Re-ports, vol. 16, Art. no. 7441, 2026.
[20] J. Shin and O. Bulut, ‘Building an intelligent recommendation system for personalized test scheduling in computerized assessments: A reinforcement learning approach,’ Behavior Research Methods, vol. 54, pp. 216–232, 2022.
[21] B. Fahad Mon, A. Wasfi, M. Hayajneh, A. Slim, and N. Abu Ali, ‘Rein-forcement learning in education: A literature review,’ Informatics, vol. 10, no. 3, Art. no. 74, 2023.
[22] A. Riedmann, P. Schaper, and B. Lugrin, ‘Reinforcement learning in educa-tion: A systematic literature review,’ International Journal of Artificial Intel-ligence in Education, vol. 35, pp. 2669–2723, 2025.
[23] U.S. Department of Education, Office of Educational Technology, Artificial Intelligence and the Future of Teaching and Learning: Insights and Recom-mendations. Washington, DC, USA, 2023.
[24] Y. Gal and Z. Ghahramani, ‘Dropout as a Bayesian approximation: Repre-senting model uncertainty in deep learning,’ in Proc. 33rd Int. Conf. Machine Learning, 2016, pp. 1050–1059.
[25] C. Finn, P. Abbeel, and S. Levine, ‘Model-agnostic meta-learning for fast adaptation of deep networks,’ in Proc. 34th Int. Conf. Machine Learning, 2017, pp. 1126–1135.
[26] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, ‘Proxi-mal policy optimization algorithms,’ arXiv:1707.06347, 2017.
[27] Yue, X., & Zhu, R. (2024). An explainable hybrid machine learning frame-work for cash-flow-conditioned multi-horizon liquidity risk early warning: An SME-oriented benchmark study. World Journal of Management Science, 2(1), 63-72.
[28] Jiao, Y., & Liang, Y. (2024). An integrated process mining and explainable machine learning framework for bottleneck and internal control risk detection in the multi entity financial close. Social Science and Management, 1(3), 96-107.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Alice Morgan, James Turner, Emma Clarke
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
This article is licensed under the Creative Commons Attribution 4.0 International License. This license permits use, sharing, and adaptation of the article for non-commercial purposes, provided the original authors and source are properly credited.
Alternatively, you may choose other Creative Commons licenses based on your preferences:
-
CC Attribution-NonCommercial-NoDerivatives 4.0: This license allows others to download the works and share them with others as long as they credit the author, but they can’t change them in any way or use them commercially.
-
CC Attribution-NonCommercial 4.0: This license allows others to remix, tweak, and build upon the work non-commercially, as long as they credit the author, but they cannot use the material for commercial purposes.
-
CC Attribution-NonCommercial-ShareAlike 4.0: This allows others to remix, tweak, and build upon the work non-commercially, as long as they credit the author and license their new creations under identical terms.
These terms ensure that your article is freely accessible and usable, while respecting the rights of the original authors.
