Aligning language tests to China’s Standards of English Language Ability: validity issues

Chair:

He Lianzhen (Zhejiang University)

Participants:

Wu Sha, National Education Examinations Authority, China

Richard J. Tannenbaum,Educational Testing Service, USA

Cai Hongwen, Guangdong University of Foreign Studies, China

Min Shangchao, Zhejiang University, China

Zhang Jie, Shanghai University of Finance and Economics, China

Jamie Dunlea, British Council, UK

Richard Spiby, British Council, UK

Summary

The past decade has seen emerging interests in aligning tests to external performance scales or frameworks, e.g. the CEFR. Such alignment gives more meaning to the test scores through association with the descriptions of proficiency that comprise the external framework. Test score alone is not very informative for test users to make decisions, but a test score with information locating learners at a performance level defined by “can do” descriptions can provide a clearer understanding of what that test score really means. Test alignment is to make a claim on the proper interpretation of test scores in relation to performance levels. To support such a claim, test alignment is a process of validation, requiring multiple sources of evidence, e.g. internal validity, procedural validity and external validity evidences.

Aligning an international examination to a localized framework can help make the meaning construction of the test score more relevant to the context of use. This symposium attempts to illustrate the process, discuss the challenges and explore the implications of such a process with two parallel research projects on linking IELTS and TOEFL to China’s Standards of English Language Ability (CSE), i.e. a set of comprehensive performance scales of English proficiency specifically designed for the Chinese EFL context. With the joint efforts from test providers and academic institutions from UK, USA and China, similar research methodologies are employed for these two research projects. By adapting the steps of familiarization, standardization, specification, standard setting and validation proposed in the Manual for Relating Examinations to the CEFR (Council of Europe, 2009), the two projects put special emphasis on constructing a chain of validity evidences, including internal validity, procedural validity and external validity.

This symposium includes four talks. The first talk will discuss the significance of conducting the test alignment research and the role of educational and social context in the alignment process. The second talk will focus on how to build up the concept of “just qualified candidates”, which is the most important part of standard setting. The third talk, based on the AUA framework, will discuss how to evaluate the quality of the standard setting process and the credibility of the results with internal and procedural validity evidences. The fourth talk will address the value and ways of collecting evidence of external validity.