In 2012, the National Center for Education Statistics (NCES) administered the National Assessment of Educational Progress (NAEP) computer-based pilot writing assessment at grade 4, providing students with a laptop computer and asking them to complete two 30-minute writing tasks. NCES conducted the current study to answer the following two questions about the assessment:
- Can fourth-graders fully demonstrate their writing skills on the computer?
- What factors are related to fourth-graders’ writing performance on the computer?
1. Writing performance
The 2012 NAEP
computer-based pilot writing assessment proved challenging for
fourth-grade students as a group, especially low-performing students.
More than two-thirds (68 percent) of fourth-graders’ responses received
scores in the bottom half of the 6-point scoring scale used in this
study. These responses were not consistent in their use of grammar and
mechanics, provided insufficient supporting statements, and reflected
little or no awareness of audience and purpose.
While fourth-graders had similar overall average scores on the 2012 NAEP computer-based writing assessment and on a paper-based pilot writing assessment administered in 2010, an analysis of 15 writing tasks common to both assessments revealed a different story. The average score of high-performing fourth-graders was higher on the computer than on paper, whereas low- performing students did not appear to benefit from using the computer. This finding suggests that low-performing fourth-graders did not fully demonstrate their writing ability on the computer in the 2012 NAEP computer-based pilot writing assessment, and that the use of the computer may have widened the writing achievement gap.
Since typing speed is relevant to the amount of text produced, it was assessed in a usability study conducted in 2011. Fourth-graders’ average typing speed was 12 words per minute, compared to 30 words per minute for eighth-graders.
While fourth-graders had similar overall average scores on the 2012 NAEP computer-based writing assessment and on a paper-based pilot writing assessment administered in 2010, an analysis of 15 writing tasks common to both assessments revealed a different story. The average score of high-performing fourth-graders was higher on the computer than on paper, whereas low- performing students did not appear to benefit from using the computer. This finding suggests that low-performing fourth-graders did not fully demonstrate their writing ability on the computer in the 2012 NAEP computer-based pilot writing assessment, and that the use of the computer may have widened the writing achievement gap.
2. Text length
The average
number of words produced by fourth-graders, as a whole, was smaller on
the 2012 computer-based pilot assessment than on the 2010 paper-based
pilot assessment (110 vs. 159). In terms of ability levels,
low-performing fourth-graders produced fewer words (60) than
middle- and high-performing fourth graders (104 and 179, respectively)
on the computer-based assessment. To place these numbers in a larger
context, consider that on the 2011 NAEP writing assessment, eighth- and
twelfth-graders produced more words on the computer than on paper (300
vs. 155 at grade 8; 389 vs. 235 at grade 12).
Since typing speed is relevant to the amount of text produced, it was assessed in a usability study conducted in 2011. Fourth-graders’ average typing speed was 12 words per minute, compared to 30 words per minute for eighth-graders.
3. Relationship between measured outcomes
D ata from the computer-based pilot assessment demonstrated that there is a positive relationship between text length and fourth-graders’ writing performance, such that the longer a student’s response, the higher the score it received. What factors are related to fourth-graders’ writing performance on the computer?1. Use of editing tools
In the computer-based pilot assessment, students’ actions on the computer were captured and analyzed for the lowest performing 20 percent of students, the highest performing 20 percent of students, and the middle-performing 60 percent of students. Compared to the middle- and high-performing students, a higher percentage of low-performing students:- used key presses less frequently;
- did not use the spellcheck function;
- did not accept any automated spelling corrections; and
- used the backspace key less frequently to edit their work.
Overall,
students who accepted spelling corrections and used the backspace key
more often were also likely to write longer responses.
2. Prior exposure to writing on the computer
Prior
exposure to writing on the computer provides insight into fourth-grade
students’ acquisition of keyboarding and word processing skills, which
can impact writing performance. Internet access at home and using the
internet to look for information are two such types of prior exposure.
Internet access at home: The 2012 fourth-grade writing data indicate that students with access to the Internet at home were more likely than those without access to:
There is an association between each of the above student actions and characteristics and students’ writing scores, and, directly, between students’ access to the Internet at home and their writing scores. For example, in 2012, 94 percent of high-performing students had access to the Internet at home, compared to 52 percent of low-performing students. Information about other factors related to writing performance can be found in the full report.
Using the Internet to look for information to include in their writing: Students who used the Internet to look for information to include in their writing were more likely to:
When asked about their preferred mode of writing, fourth-graders with
the following types of prior exposure and demographic characteristics
were more likely to say that they would prefer to take a writing
assessment on paper:
Internet access at home: The 2012 fourth-grade writing data indicate that students with access to the Internet at home were more likely than those without access to:
- write longer responses;
- use the spellcheck tool more often;
- use the thesaurus tool more often; and
- use bold and italics for emphasis more often.
Students
with access to a computer at home were also more likely to prefer to
write on the computer than on paper, compared to those without access.
The percentage of fourth-graders without access to the Internet at home was higher for Black students, Hispanic students, students eligible for free or reduced-price lunch, English language learners, and students with a disability.
The percentage of fourth-graders without access to the Internet at home was higher for Black students, Hispanic students, students eligible for free or reduced-price lunch, English language learners, and students with a disability.
There is an association between each of the above student actions and characteristics and students’ writing scores, and, directly, between students’ access to the Internet at home and their writing scores. For example, in 2012, 94 percent of high-performing students had access to the Internet at home, compared to 52 percent of low-performing students. Information about other factors related to writing performance can be found in the full report.
Using the Internet to look for information to include in their writing: Students who used the Internet to look for information to include in their writing were more likely to:
- write longer responses;
- accept automated spelling corrections; and
- use the backspace key to edit their work.
There is a positive relationship between each of these student actions
and students’ writing scores, and, directly between students’ use of the
Internet to look for information to include in their writing and their
writing scores. For example, 25 percent of high-performing
fourth-graders reported they never looked for information on the
Internet to include in their writing, compared to 32 percent of
low-performing fourth-graders.
3. Preferred mode of writing
When asked about their preferred mode of writing, fourth-graders with
the following types of prior exposure and demographic characteristics
were more likely to say that they would prefer to take a writing
assessment on paper:
- students with no access to the Internet at home;
- students with less experience looking for information on the Internet to include in their writing;
- students with less experience writing to friends and family using the Internet;
- male students;
- English language learners; and
- students with a disability
There
is an association between each of these student characteristics and
students’ writing scores, as well as between students’ preferred mode of
writing and their writing scores. For example, in 2012, 11 percent of
high-performing fourth-graders preferred writing on paper, compared to
27 percent of low-performing fourth-graders.
Conclusion
Whether or not the results of the NAEP 2012 computer-based pilot writing assessment offer a true measure of all students’ writing abilities has yet to be determined. The data indicate that fourth-graders, as a group, produce considerably shorter texts on the computer than on paper—a measured outcome that is related to writing performance. This suggests that when a student is only capable of producing a few sentences per hour on the computer in a NAEP-like assessment, he or she is likely to have difficulties with composing a whole essay. Part of that difficulty may have to do with simply finding the letters on the keyboard and the editing tools on the menu—taking time and cognitive resources away from critical thinking processes, such as finding the right word, varying sentences, and providing strong details and supportive evidence in light of the given audience and purpose of writing. The NAEP data also indicate that limited keyboarding and word processing skills may be associated with prior exposure to writing on the computer.
Whether or not the results of the NAEP 2012 computer-based pilot writing assessment offer a true measure of all students’ writing abilities has yet to be determined. The data indicate that fourth-graders, as a group, produce considerably shorter texts on the computer than on paper—a measured outcome that is related to writing performance. This suggests that when a student is only capable of producing a few sentences per hour on the computer in a NAEP-like assessment, he or she is likely to have difficulties with composing a whole essay. Part of that difficulty may have to do with simply finding the letters on the keyboard and the editing tools on the menu—taking time and cognitive resources away from critical thinking processes, such as finding the right word, varying sentences, and providing strong details and supportive evidence in light of the given audience and purpose of writing. The NAEP data also indicate that limited keyboarding and word processing skills may be associated with prior exposure to writing on the computer.
No comments:
Post a Comment