Has Validity Really Changed?Prof. Stephen G. Sireci Explores AI's Impact on Educational Assessment
Interviewed by Amina Afif
As part of the Conversations with the AEA-Europe 2026 Keynotes interview series, Amina Afif sat down with Prof. Stephen G. Sireci, Distinguished University Professor and Executive Director of the Center for Educational Assessment at the University of Massachusetts Amherst, to discuss the ideas behind his upcoming keynote address at the AEA-Europe 2026 Conference in Rome.
During the conversation, Prof. Sireci addressed a question that is becoming increasingly relevant in the age of artificial intelligence: Has validity really changed?
While AI is transforming many aspects of assessment, from item development to automated scoring, Prof. Sireci argues that the fundamental principles of validity remain unchanged. According to him, assessments continue to require clear evidence that they measure the intended construct and that their results are used appropriately for their intended purpose.
The discussion also explores how AI is influencing assessment design. Rather than viewing artificial intelligence as a replacement for human expertise, Prof. Sireci describes it as a tool that can support established psychometric practices. However, he stresses that greater use of AI also creates greater responsibility to validate its role throughout the assessment process.
Another important topic is the potential risk of AI narrowing the way knowledge and skills are represented. If AI systems rely on limited or biased sources of information, they may unintentionally produce assessments that fail to capture the full breadth of the constructs being measured. For this reason, rigorous validation remains as important as ever.
Throughout the interview, Prof. Sireci highlights the unique role of psychometricians in evaluating AI-driven assessment methods, explaining that the same validation principles long used in educational measurement can also be applied to emerging technologies.
The conversation offers an insightful preview of the themes that will be explored in his keynote presentation and provides valuable perspectives for researchers, assessment professionals, and anyone interested in the future of educational and psychological testing.
Short Bio
Stephen Sireci
University of Massachusetts Amherst
Stephen G. Sireci, Ph.D. is Distinguished Professor and Executive Director of the Center for Educational Assessment in the College of Education, University of Massachusetts Amherst. He earned his Ph.D. in psychometrics from Fordham University and his master and bachelor degrees in psychology from Loyola College Maryland. Before UMass, he was Senior Psychometrician at GED Testing Service, Psychometrician for the CPA Exam and Research Supervisor of Testing for the Newark NJ Board of Education.
He is known for his research in validity and fairness of educational tests, and for innovations in test development. He currently serves/has served on several advisory boards including the National Board of Professional Teaching Standards, Duolingo English Test, and technical advisory committees for Florida, Maryland, New Hampshire, New York, Montana, Puerto Rico, and Texas. He is a Fellow of American Educational Research Association and of Division 5 of American Psychological Association, and a lifetime member of the National Academy of Education. He is a past President of International Test Commission, Northeastern Educational Research Association, and National Council on Measurement in Education.
His UMass honors include School of Education’s Outstanding Teacher Award, Conti Faculty Fellowship, Public Engagement Fellowship, Outstanding Accomplishments in Research and Creative Activity Award, and the Chancellor’s Medal. He also received the Messick Memorial Lecture Award from Educational Testing Service/International Language Testing Association.
He serves on several editorial boards including Applied Measurement in Education, Educational Assessment, Educational Measurement: Issues and Practice, Educational and Psychological Measurement, Practical Assessment Research and Evaluation, and Psicothema.