Fourth-Grade Students Ready for Computer-Based Writing

An innovative pilot study administered as part of the National Assessment of Educational Progress (NAEP) has demonstrated that fourth-grade students can meaningfully participate in a computer-based writing assessment. The study, conducted by the National Center for Education Statistics (NCES), presents lessons learned that can inform educators’ and policymakers’ efforts to develop computer assessments that can measure elementary students’ writing and composition skills. 

“NAEP has studied and captured data on fourth-grade students’ ability to write using a computer, and we are excited to report that they are capable of using computer programs to type, organize and write well enough to be assessed,” said David P. Driscoll, the chairman of the National Assessment Governing Board, which oversees NAEP. “The Governing Board plans to transition all NAEP assessments to be computer-based, and this evidence makes us confident that it is appropriate to include fourth-grade writing along with other subjects in which students must provide written answers.”

The Web-based report, “Lessons Learned From the 2012 Grade 4 Writing Computer-Based Assessment Study,” describes specific ways the design of this writing assessment facilitates the writing process and offers information about how fourth-grade students can produce written text when using a computer. 

The findings come from two separate efforts: a usability study, which allowed NCES to improve the computer program with fourth-grade students in mind, and a pilot writing assessment on the revised program.

“As the gold standard in large-scale assessment, the National Assessment of Educational Progress plays a leadership role in identifying best practices in the field,” said Peggy G. Carr, associate commissioner of the assessments division at NCES. “Through this pilot study, NAEP shares these lessons learned to inform how large-scale assessments can use computers to accurately measure the writing skills of fourth-graders.”

The usability study asked fourth-grade students to use the same NAEP computer platform that was used to measure eighth- and 12th-graders’ writing skills. Findings from the usability study showed that fourth-grade students had difficulty with elements of the platform’s design, including reading and following long sets of instructions, using drop-down menus and understanding text-to-speech options.

The study asked students about their access to computers at home and in school. Of the 60 fourth-grade participants, 100 percent of the students reported having access to a computer at school, while 93 percent reported having access at home and 92 percent reported previously taking a computer-based assessment.

In response to the usability study findings, NAEP redesigned the writing assessment to introduce shorter, sequenced directions, more icons in place of drop-down menus and other features. The new assessment system was then used in the pilot study given to 13,000 students nationwide. Though the sample is not representative of the nation, the results provide an indication of fourth-graders’ ability to show their writing skills and use editing tools on a computer-delivered assessment. Students were able to produce writing that can be evaluated against the NAEP rubric, according to the findings: They typed enough words to allow evaluators to measure their writing skill, used the computer tools to complete the exercises and showed that they could edit text using word processing tools, such as spell-check and backspace functions.

Each student, provided with a laptop and headphones, was given two scenarios. Each scenario included text; some also included pictures, or audio or video components. The scenarios were designed to encourage students to develop and organize ideas and demonstrate a specific writing skill: to persuade, to explain or to convey experience.

Students were asked to respond to writing situations that were designed to elicit one of those three modes for writing. A sample persuasion question — which required a student to change the reader’s point of view or affect the reader’s action — asked students to write a letter to their principal convincing him or her to choose the writer’s preferred school mascot, giving reasons and examples to support their position. To explain, students had to write in ways that expanded the reader’s understanding, such as describing what lunchtime is like during their school day. Conveying experience entailed writers’ bringing a real or imaginary situation to life; for example, students were asked to respond to a photo of the Eiffel Tower by writing a story about what happens next when they are somehow transported, while sleeping, to the sidewalk beneath the tower and then wake up there. 

Students’ responses were measured on a holistic scale from 1 to 6, with 1 indicating that the student had little or no writing skill and 6 signifying that the student could write effectively. The scoring rubric includes multiple criteria, including development of ideas, organization, language use and grammar. Students were given 20- and 30-minute writing tasks. Overall, students provided more complete responses when given 30 minutes, rather than 20 minutes, to write. The majority of students (about 61 percent) who were given two 30-minute prompts scored at least a 3, meaning that they wrote enough to be assessed, included ideas that were mostly on topic and used simple organizational strategies in most of their writing.

“This study offers lessons to educators who want to support elementary students in using computers to write: Make the instructions easy to read and understand, use clearly labeled icons wherever possible and give students the time they need to complete the assignment,” said Shannon Garrison, a fourth-grade teacher in Los Angeles and Governing Board member. “Anyone looking to improve online assessment tools should learn from NAEP and share these lessons widely.”