Tuesday, April 30, 2019

2018 Technology and Engineering Literacy (TEL) assessment


In 2018, the National Assessment of Educational Progress (NAEP) administered the digitally based Technology and Engineering Literacy (TEL) assessment to better understand what U.S. students know and can do in the areas of technology and engineering.

Related article

Eighth-grade students were presented with real-world technology and engineering challenges. They were asked to respond to questions aimed at assessing their knowledge and skills in understanding technological principles, solving technology and engineering-related problems, and using technology to communicate and collaborate. Students also answered questions about their experiences learning about and engaging in technology and engineering both in and outside of school.
Below are the key findings for the NAEP TEL assessment. Results are compared to the previous assessment year in 2014.
The TEL assessment was administered on laptops between January and March 2018
  • National sample size was 15,400 eighth-graders from 600 schools.
  • Results are available for the nation.
  • Performance is reported as
    • Average scale scores: 0-300 scale
    • NAEP Achievement Levels: NAEP Basic, NAEP Proficient, NAEP Advanced
  • The assessment included 15 scenario-based tasks and 77 discrete questions.
  • Total testing time per student was 60 minutes.
  • Survey questionnaires were administered to students and school administrators.
Scores at a Glance
Increase in average overall TEL score for eighth-graders in 2018 compared to 2014
Compared to 2014, there was a 2-point increase in the average overall TEL score at grade 8 in 2018.
2014 ScoreScore Change2018 Score
150
on 0–300 scale
2ptsIcon showing an arrow pointing up
Increase from 2014 to 2018
152
on 0–300 scale
The TEL assessment measures three interconnected content areas—Technology and Society, Design and Systems, and Information and Communication Technology—and three practices that cut across the content areas—Understanding Technological Principles, Developing Solutions and Achieving Goals, and Communicating and Collaborating. When solving problems, students are expected to be able to apply each of the TEL practices within and across the content areas. Learn more about what the TEL content areas and practices measure.
In 2018, eighth-grade students also scored higher in all three TEL content areas and in all three practices compared to 2014.

2018 ScoreScore Change
TEL
Content
Areas
Technology & Society152
2 ptsIcon showing an arrow pointing up
Design & Systems153
3 ptsIcon showing an arrow pointing up
Information & Communication Technology153
3 ptsIcon showing an arrow pointing up
TEL
Practices
Understanding Technological Principles152
2 ptsIcon showing an arrow pointing up
Developing Solutions & Achieving Goals152
2 ptsIcon showing an arrow pointing up
Communicating & Collaborating153
3 ptsIcon showing an arrow pointing up
See Details Using the TEL Report Card:
Scores by Percentiles
Increases in overall TEL scores for middle- and higher-performing eighth-graders in 2018
NAEP reports scores at five selected percentiles to show the progress made by lower- (10th and 25th percentiles), middle- (50th percentile), and higher- (75th and 90th percentiles) performing students. In comparison to 2014, the 2018 overall TEL scores were higher for eighth-graders performing at the 50th, 75th, and 90th percentiles.
Percentiles chart
See Details Using the TEL Report Card:
NAEP Achievement-Level Results
Higher percentage of eighth-graders at or above NAEP Proficient compared to 2014
In 2018, forty-six percent of eighth-grade students performed at or above the NAEP Proficient level in TEL compared to 43 percent of students in 2014.
Achievement-Level Results chart

NAEP achievement-level results for eighth-grade students assessed in NAEP technology and engineering literacy (TEL): 2014 and 2018

201846201443*1009080706050403020100102030405060708090100PercentYearPercentage at or above Proficient
*
Significantly different (p < .05) from 2018.
NAEP achievement levels are to be used on a trial basis and should be interpreted and used with caution.
In 2018, the percentages of eighth-grade students who performed at or above NAEP Proficient were higher for several student groups compared to 2014.
Changes in NAEP Proficient and NAEP Advanced achievement-level results for eighth-grade students assessed in NAEP technology and engineering literacy (TEL), by student group: 2014 and 2018




Student groups
At or above NAEP Proficient
At NAEP ProficientAt NAEP Advanced
Race/ethnicity
White
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating that results were higher
Black
Icon indicating that results were higher
Icon indicating that results were higherIcon indicating no significant difference
Hispanic
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating no significant difference
Asian
Icon indicating that results were higher
Icon indicating no significant differenceIcon indicating that results were higher
Native Hawaiian/Other Pacific Islander
Icon indicating reporting standards not met
Icon indicating reporting standards not metIcon indicating reporting standards not met
American Indian/Alaska Native
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating no significant difference
Two or More Races
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating no significant difference
Gender
Male
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating that results were higher
Female
Icon indicating that results were higher
Icon indicating no significant differenceIcon indicating that results were higher
Eligibility for National School Lunch Program
Eligible
Icon indicating that results were higher
Icon indicating that results were higherIcon indicating that results were higher
Not eligible
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating that results were higher
Information not available
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating no significant difference
Type of School
Public
Icon indicating that results were higher
Icon indicating no significant differenceIcon indicating that results were higher
Private: Catholic
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating no significant difference
Charter
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating no significant difference
Public, noncharter
Icon indicating that results were higher
Icon indicating no significant differenceIcon indicating that results were higher
Other student groups with statistically significant changes
Parents did not finish high school
Icon indicating that results were higher
Icon indicating that results were higherIcon indicating no significant difference
Either parent graduated from college
Icon indicating that results were higher
Icon indicating no significant differenceIcon indicating that results were higher
Students attending schools in city
Icon indicating no significant difference
Icon indicating no significant differenceIcon indicating that results were higher
Students not identified as students with disabilities
Icon indicating that results were higher
Icon indicating no significant differenceIcon indicating that results were higher
Students not identified as English language learners
Icon indicating that results were higher
Icon indicating that results were higherIcon indicating that results were higher
Icon indicating that results were higherThe 2018 percentage is higher than the 2014 percentage.
Icon indicating no significant differenceNo significant difference between 2014 and 2018.
Icon indicating reporting standards not metReporting standards not met. Sample size insufficient to permit reliable comparisons.
NAEP achievement levels are performance standards that describe what students should know and be able to do. Results are reported as percentages of students performing at or above three achievement levels (NAEP Basic, NAEP Proficient, and NAEP Advanced). Students performing at or above the NAEP Proficient level on NAEP assessments demonstrate solid academic performance and competency over challenging subject matter. NAEP Proficient does not represent grade-level proficiency as determined by other assessment standards (e.g., state or district assessments). As provided by law, NAEP achievement levels are to be used on a trial basis and should be interpreted and used with caution. Find out more about the NAEP TEL achievement levels.
See Details Using the TEL Report Card:
Score Differences by Gender
Female students outperformed male students in TEL
In 2018, female students scored higher than their male peers in TEL overall. Female students also scored higher than their male peers in more content areas and practices in 2018 compared to 2014.

20142018
TEL
Overall
FemaleDashed lines151*
155
Female-male
Score gap
3point significant score gap
5point significant score gap
Male149

150
* Significantly different (p < .05) from 2018.Score gaps are calculated based on unrounded scores

TEL Content Areas

Technology & Society

20142018
Female151*154
Female-male
Score gap
14
Male149151
Design & Systems

20142018
Female150*154
Female-male
Score gap
#1
Male150152
Information & Communication Technology

20142018
Female153*156
Female-male
Score gap
67
Male147149

TEL Practices

Understanding Technological Principles

20142018
Female151*154
Female-male
Score gap
13
Male149151
Developing Solutions & Achieving Goals

20142018
Female151*155
Female-male
Score gap
24
Male149150
Communicating & Collaborating

20142018
Female153*157
Female-male
Score gap
57
Male147149

NOTE: Score gaps are calculated based on unrounded scores between male and female students.
# Rounds to zero.
* Significantly different (p < .05) from 2018.
Significant differenceNo significant difference
See Details Using the TEL Report Card:
Coursetaking
A higher percentage of students reported taking at least one course related to technology or engineering compared to 2014
As part of the TEL assessment, students answered survey questionnaires that included questions about their learning experiences in technology and engineering. For example, students were asked whether they had taken the following classes in school: industrial technology; engineering; any class that involves learning to use, program, or build computers; or any other technology-related class. Students could select one or more of the classes or indicate that they had not taken any of these classes.
Fifty-seven percent of eighth-graders reported taking at least one class related to technology or engineering in 2018, an increase of 5 percentage points compared to 2014.
Chart
Students who reported taking at least one technology- or engineering-related class in 2018 had a higher TEL score on average than those who reported not taking any of these classes.
Chart
See Details Using the TEL Report Card:
Explore Sample Scenario-Based Tasks
In the NAEP TEL assessment, students were tested with computer simulations of technology and engineering problem-solving tasks set in a variety of realistic scenarios. Using laptops, students interacted with these scenario-based tasks that included videos, audio clips, animations, and text.

For example, in the task Andromeda, a television network is using a website to promote a television show about the Andromeda Galaxy. Students need to select a suitable image of the Andromeda Galaxy for the website. They must secure permission to use a copyrighted image and to properly credit use of the image in accordance with fair use guidelines.
A picture of the Andromeda galaxy

Students were able to select appropriate information to communicate an idea in accordance with fair use guidelines but were less able to accurately cite the source of copyrighted material.
60%
of students correctly identified images that could be legally used on a website.
51%
of students identified the proper reference style to use for writing a citation.
29%
of students successfully wrote a citation that included all required copyright information.


No comments: