An analysis of the English SEC examination in relation to the Common European framework of reference for languages

Abstract

The study was concerned with relating a national English language examination in Malta to the Common European Framework of Reference for Languages: learning, teaching, assessment" (CEFR). Using procedures described in the "Manual for Relating Language examinations to the CEFR" the Secondary Educational Certificate (SEC) examination for English Language was analysed and the corresponding CEFR level of the examination was estimated. In order to carry this out, procedures in Specification and Standardisation were carried out, preceded by Familiarisation with the existing CEFR documentation. The study made use of a sample of 600 scripts from the 2007 session and the focus was placed on the reading and writing tasks included in this session of the examination. The Specification phase included the filling in of all the relevant forms included in the Manual. Following this, Item and Content Analyses of the reading comprehension tasks were carried out using the Dutch Grid developed by ALTE, while the writing tasks were analysed using the CEFR Writing Grid. Item Analysis allowed the researcher to make an estimation of the CEFR level of each item. This also helped to identify the level of learners who would be able to comprehend the texts that accompanied these items. This stage of the analysis was complemented by calculating p-values for the number of correct answers obtained on each item included in the reading tasks. Conclusions were also made regarding the overall CEFR level of the two parallel papers of the examination, Paper 2A an<l Paper 2B. The study was carried out by only one researcher and was thus limited in resources and m the opportunity of making judgments based on a group's opinion, as indicated m the Manual. However, this was taken into consideration when reporting results.