Early Grade Writing Assessment : A report on development of an instrument

Abstract

This report presents main outcomes of a project on assessment of writing skills in early grades of primary school (EGWA), implemented by UNESCO in collaboration with Professor Juan E. Jiménez and his team from the Universidad de La Laguna in the Canary Islands, Spain, with the support of the Russian Federation. It provides findings of a literature review on early literacy development from a cognitive sciences perspective and a model instrument developed to assess writing skills in the Spanish language, as well as an account of how this instrument had been developed. Literacy is an integral part of knowledge, skills and competences required in our knowledge-oriented society. While our skills to read, understand, analyze and produce written text and digital materials associated to varying contexts can be continuously developed throughout our lives, acquisition of basic literacy skills, possibly at an early stage of life, is critically important as a foundation of our lives, work and further learning. Writing skills, together with reading and mathematics, are an essential part of basic literacy skills. During the past decade, however, such skills have been less measured, compared to reading skills and mathematics. It was this concern and a related recommendation made in 2009 by an UNESCO-led International Working Group on Assessing and Improving Quality Learning that laid the ground for this project. Several years later, today, however, the need for better gauge of writing skills still exists, for which this report is expected to make a contribution. The main target audience of the report is those who are concerned with a global learning crisis, where some 617 million children and adolescents worldwide – 56 per cent of children of primary school age and 61 per cent of adolescents of lower secondary school age – are not achieving minimum proficiency levels in reading and mathematics. It is especially for teachers, curriculum developers, assessment experts, and other types of practitioners, who are keen to improve their understandings of children’s writing skills and teaching and learning practice in early grades of primary education. Due to its formative nature, the main purpose of this EGWA is not for the comparative and bench-marking purposes, while EGWA has the potential to enrich policy dialogue. This project benefited from accumulated knowledge through a literature review and peer reviews conducted by a number of experts in related fields, including those who have worked on early grade reading assessment (EGRA). The structure of the report is as follows. Chapter 1 - Introduction: explains some background, locating this project in global education agendas and providing some rationale for this work on early grade writing assessment. Chapter 2 - Conceptual framework and research foundations: examines existing knowledge regarding early literacy development and related issues, providing a conceptual foundation for developing an instrument for early grade writing assessment (EGWA). Starting with the early precursor skills that impacts on later literacy learning, it first explores issues related to a writing process and learning to write, the state of writing skills in some developed countries, written systems and language, as well as the relationships between reading and writing in English and Spanish. It then explains a developmental model of writing adapted (Berninger and Amtmann, 2003) which has the following three components, related to working memory: 1) transcription (handwriting, keyboarding and spelling); 2) executive functions (supervisory attention, planning, reviewing, revising, strategies for self-monitoring and regulations); and 3) text generation (words, sentences, discourse). In this model, the transcription process, which enables a writer to translate internal language into external written system, is the first to develop for mastering of both fluency and accuracy in handwriting, as well as a thorough knowledge of spelling. Both the transcription processes and executive processes that follow the former, support text generation to turn ideas into appropriate language. The report first elaborates the transcription processes, touching upon issues such as handwriting legibility and speed, spelling acquisition, which are of fundamental importance in the early stages of writing. It then look into text generation, illustrating major phases of development from forming individual letters and spelling words to constructing phrases, sentences, paragraphs and stories. This Chapter also analyzes key issues in teaching of writing skills. Finally, the Chapter explores writing assessment, stating that both an output and a process of writing should be assessed in order to help children become better writers ultimately. Chapter 3 - Expert Meeting on Formative Writing Assessment in Early Grades of Primary Education: provides a record of the Expert Meeting held in Canary Islands held in 2012. Following the completion of the literature review and the development of the draft instrument of EGWA, the Expert Meeting was organized jointly by UNESCO and the Universidad de La Laguna in Spain, to review these drafts and define possible next steps. The main outcomes of the meeting were: 1) constructive and useful comments provided by the experts for finalizing the drafts; 2) an agreement reached on an immediate next step, namely the piloting of the EGWA instrument in the Canary Islands with around 800 primary school children; and 3) recommendations on potential future action to build on this project on EGWA, including exploring possible adaption of EGWA into non-formal learning settings and a possibility of coordinating items of EGWA and EGRA. Chapter 4 - Early Grade Writing Assessment (EGWA): provides the details of the instrument, including description of tasks, administration and scoring procedures for each of the ten tasks, ranging from copying letters to story writing. The first four tasks consist of copying (copying letters, copying words and copying a sentence). The next four tasks consist of dictating activities (writing down dictated words and pseudowords, and dictated sentences). Tasks 9 and 10 are free-writing tasks (writing sentences and writing a story). A child is prompted to copy or write each letter, word or sentence exactly as it is shown in the test protocol or dictated by the evaluator. Chapter 5 - EGWA Pilot Study in the Canary Islands: presents how the EGWA instrument was piloted and its results. The Chapter starts with the study design. As a sampling strategy, probability sampling was selected to involve 1,653 pupils in the first, second and third grades from twelve primary schools randomly selected in the Canary Islands. It also describes the procedure of EGWA administration. Managed by the Canary Islands Department of Education in coordination with the twelve schools, the EGWA was administrated individually by trained university students from the Educational Psychology Department at the Universidad de La Laguna over three sessions (thirty minutes per session). This Chapter then presents results of the assessment in light of four main components of writing which were measured by EGWA: handwriting fluency; spelling; sentence production; and text production. 1) handwriting fluency is measured by the number of letters written correctly per minute in alphabet copying and letters written correctly per minute in allograph selection; 2) spelling is measured by the number of words spelled correctly in word and sentence dictation tasks and the number of pseudowords with correct graphical representation of the sounds in the pseudoword dictation task; 3) sentence production is measured by the number of words written, the number of words spelled correctly and spacing between words; and 4) text production is measured by the number of written words, the number of words with correct spelling, word sequence, and global structure. The results show the reliability of the EGWA measures, EGWA’s inter-rater reliability, its concurrent and predictive validity, and its ability to make multiple comparison of performance between grades. It also concludes that the standardization of writing assessment for the early grades (Grade 1, 2, and 3) is possible. Finally, the Chapter also explains how EGWA’s preliminary results have been disseminated and the short version of EGWA has been piloted with teachers. Chapter 6 - Using EGWA: implications for policy dialogue: illustrates areas where EGWA can make a difference. Recognizing the weak link between policy and research, it states that the knowledge and insights generated through writing skills assessments could inform not only teaching and learning practice in school but also policy dialogue. It then suggests possible ways of strengthening this link, notably improved communication between policymakers and researchers, and involving policy-makers and other stakeholders in joint research activities. It also reflects on experiences of early grade reading assessment (EGRA) which has generated interest of policy-makers and officials. Learning from EGRA’s experiences would help make the appropriate and wise use of EGWA in the future. Finally, the Chapter suggests multiple ways of using EGWA in school for improving the effectiveness of pedagogy, teaching and curriculum, as well as better accountability and resource allocation.