Content introduction chapter-i assessing learner's writing skills according to cefr scales

Task-Difficulty Characteristics
When using a level-specific task approach, research on task-difficulty characteristics may help to shed light on information processing and cognitive demands. Influential work in this field, mainly situated in pedagogical, second language acquisition contexts, is reported by, for example, Brown, Anderson, Shillcock, and Yule (1984); Prabhu (1987); Robinson (2001); Skehan (1998); and Skehan and Foster (2001). The results of these studies point toward a progression of difficulty from simple tasks, where all information necessary to solve the task is given, to more complex ones, where the prompt requires the explanation of abstract concepts or the development of an argument; the number of elements to address also increases task difficulty, as does the amount of processing and reasoning required to solve the task. Task difficulty decreases with an increasing level of familiarity and prior knowledge of topics, tasks, and cognitive operations demanded; furthermore, difficulty decreases with the amount of information given and the degree of precision—the more precise, the easier the task.
Skehan and Foster (2001) claimed in their Limited Attention Capacity Model (LACM) that an increase in task complexity would strain the limited attention capacity of learners, who would thus shift their focus on form and produce linguistically less complex output with more errors. Robinson (2001) proposed the Triadic Componential Framework, expanding the LACM and claiming that as learners possess different attentional resource pools, an increase in task complexity would not necessarily lead to a decrease in their linguistic output.
When looking at studies that tried to empirically put these models into operation, the results are rather inconsistent. Kuiken and Vedder (2007), for example, found it difficult to operationalize the variables of the models in their study of task complexity and linguistic performance (p. 265), and they came to the conclusion that it was not possible to establish interactional effects between task complexity and proficiency level (p. 276). An example of the application of the LACM in a testing context is the study by Iwashita, McNamara, and Elder (2001) on speaking tasks in the ETS oral test. They found a “lack of consonance” between their results and SLA research findings and came to the conclusion that the “differences between testing and pedagogical contexts are so great as to alter the cognitive focus of the task” (p. 430). This seems further corroborated by Norris, Brown, Hudson, and Bonk (2002), who reported inconsistent results when operationalizing Skehan's model.
The aforementioned studies seem to indicate that it is difficult to apply task characteristics found in pedagogical contexts to test contexts; nevertheless, these characteristics can help to underpin test constructs and the level descriptors in the CEFR with empirical research. Although a direct alignment of these characteristics to specific CEFR levels is not the aim of the study reported here, it would remain a challenging desideratum for further research. In this context, issues of congruence between the level specificity of test tasks and the functional orientation of the CEFR would need to be explored in more depth. As a starting point, one could select relevant CEFR descriptors focusing on “communicative activities” (Chapter 4), extract relevant functional concepts and task characteristics and translate these concepts and characteristics into test specifications. Rating criteria and scales could also be based on relevant CEFR descriptors targeting “linguistic competences” as described in CEFR Chapter 5. This is the approach we chose, which is described in more detail next. It is worth mentioning here that the CEFR differentiates between “quality” and “quantity,” as, for example, De Jong (2004) or Hulstijn (2007) stated: Quantity refers to what learners can do in terms of functional skills as for instance demanded by a task, whereas quality refers to how well learners perform in terms of effective and efficient use of language skills. Bearing this in mind, it seems reasonable in a large-scale assessment to offer test tasks targeting a span of proficiency levels, and assessment criteria covering both, task quantity and performance quality.