APTIS SCORING SYSTEM
KAREN DUNN
PAGE 8
7. CEFR LEVEL ALLOCATIONS
FOR INDIVIDUAL SKILL AREAS
In testing, one of the most important decisions is where to locate the cut scores, or thresholds
between adjacent levels of achievement. Often the side of a cut score that a candidate falls can have
immediate and significant impact on the life of a test candidate.
The decision as to where the cut scores will fall in an Aptis test component is reached via a process of
standard setting (see Section 6 above), in which the decision is made regarding which score indicates
that a particular level of English language proficiency has been reached for each successive CEFR
level. Cut scores in Aptis are not the same across each of the skills components for each of the CEFR
levels (see the Appendix). This is because the content and design of the tests has been independently
evaluated and aligned with the language descriptors set out in the CEFR. In order to accurately reflect
a candidate’s level of English language ability in any given skill area, each skill is addressed
separately and the cut scores relevant for each CEFR level are based on the data and evidence
collected in the alignment process. For Speaking and Writing skills, this reflects the quality of the
language that is produced to achieve a score that meets each threshold. For Listening and Reading,
this reflects the level of understanding indicative of a given CEFR level. This means that the exact
point on the skill-specific score scale at which a person demonstrates a B2 level of proficiency, for
example, will not be the same in the Reading component as it is for Listening.
In cases where a candidate’s score for a given component falls in the borderline between two levels,
Aptis uses score information from the Core component to refine the final CEFR level allocation. In this
way, the Grammar and Vocabulary score feeds into the decision-making process for the awarding of
CEFR levels across all skills: Listening, Reading, Speaking and Writing. This increases the fairness
and accuracy of CEFR level allocation (McCray & Dunn, forthcoming).
The rationale for doing this is as follows:
§
In allocating a test score, there will always be a degree of error in the test’s reflection of the
candidate’s true ability in any given skill. This is known as “standard error of measurement” in
testing theory. The aim is to minimalise this error during test development, but it is impossible
to eradicate fully.
§
Grammar and vocabulary have been shown theoretically to be key processes in models of
receptive and productive L2 language ability (e.g., Field, 2013; Grabe & Kaplan, 1996;
Khalifa & Weir, 2009; Levelt, 1989), and empirically good predictors of language proficiency
(e.g., Shiotsu, 2010). A solid foundation in grammar and vocabulary knowledge can therefore
be viewed as relevant across all skill areas.
§
Scores achieved in the grammar and vocabulary component provide a valuable additional
source of evidence in finalising CEFR level allocation in the Aptis test for all skill areas,
especially for the crucial thresholds between B1 and B2, B2 and C (McCray & Dunn,
forthcoming).
|