Is TIMSS Advanced an appropriate instrument for evaluating mathematical performance at the advanced level of Norwegian upper secondary school? An analysis of curriculum documents and assessment items

Abstract

The results of international, large-scale achievement studies attract a lot of attention and may affect educational policies. With this in mind, the primary aim of the present study was to examine the appropriateness of the mathematics tests developed for the two cycles of TIMSS Advanced for the evaluation of the mathematical performance of Norwegian upper secondary school students. The study utilized the methodology for alignment studies developed by Porter (2002), which entails analyzing and describing the mathematical content of the Norwegian curriculum documents and the TIMSS Advanced assessment items and calculating the agreement between these. The analyses showed a moderate alignment between the different iterations of TIMSS Advanced and the curri-culum followed by the participating students, implying that the emphasis in the Norwegian mathematics curriculum differs somewhat from that in the TIMSS Advanced tests. This does not mean that the TIMSS Advanced tests were inappropriate instruments for assessing the mathematical performance of participating Norwegian students. However, when interpreting the results of this large-scale survey, the differences in emphasis between the different iterations of the TIMSS Advanced and the curriculum should be taken into account. Finally, as alignment is evaluated by contrasting alignment indices calculated for different combinations of the intended and assessed curriculum, additional research is needed to make more firm judgments. An additional contribution of the present study is demonstrating a powerful methodology for conducting this kind of research.