Abstract

Holistic assessment encompasses both cognitive and non-cognitive aspects of student development. Because non-cognitive skills are typically assessed using multi-item questionnaires, these instruments can become lengthy, increasing respondent burden and potentially reducing data quality. Developing efficient questionnaires that preserve construct validity while minimizing the number of items is therefore an important methodological challenge in educational assessment.

This webinar introduces the Semantic–Psychometric Integration Framework (SPIF), a novel methodology that reconceptualises questionnaire optimisation as an evidence integration process rather than a purely statistical exercise. The framework combines conceptual foundations (construct definitions and expert consensus), semantic analysis using natural language processing, semantic embeddings, large language models, and psychometric evaluation based on reliability analysis, confirmatory factor analysis, and Item Response Theory. Together, these complementary perspectives inform transparent decisions to retain, revise, merge, or remove questionnaire items.

The methodology is illustrated through a case study involving a 180-item questionnaire measuring 26 non-cognitive skills, based on approximately 28,000 questionnaire responses collected from secondary school students across three survey administrations. The webinar proposes a transferable methodological framework for AI-assisted educational assessment design. Participants will gain an understanding of how conceptual, semantic, and psychometric evidence can be integrated within a unified optimisation process to develop efficient educational measurement instruments while preserving construct validity and measurement quality. The webinar concludes by discussing the implications of SPIF for educational measurement.

Authors

Holistic Assessment SIG

To succeed in the modern society, including in the workplace, young people need to complete their education with more than the academic knowledge, skills and understanding assessed in traditional school subjects. They need a broad range of 21st-century competencies coupled with other personal attributes such as solid ethical values, positive learning dispositions, and high levels of personal wellbeing.