Thursday, September 29, 2016

Assessing the accuracy of evaluating principals using student test scores

States and districts need ways of measuring principal performance that correctly identify effective principals. A new report from the Institute of Education Sciences examines the accuracy of test-based measures of principal performance that could be implemented broadly.

The study assessed the predictive validity of these measures—the extent to which ratings from these measures accurately reflect principals’ contributions to student achievement in future years. Performance measures should have high predictive validity to be useful for informing personnel decisions about principals. Because such decisions determine which individuals will lead schools in subsequent years, measures should accurately identify which principals are likely to perform well in the future.

The report finds little evidence that test-based performance measures accurately predict principals’ future contributions to student achievement. Using statewide data from Pennsylvania for the 2007-08 to 2013-14 school years, the study found:

•    Performance measures that reflected students’ end-of-year achievement, without accounting for their past achievement, provided no information for predicting principals’ contributions to student achievement in the following year;

•    Performance measures that accounted for students’ past achievement by measuring their growth provided, at most, a small amount of information for predicting principals’ contributions in the following year; and

•    Averaging performance measures across multiple recent years did not improve the accuracy of these measures.

These findings suggest that states and districts should exercise caution when using these measures to make major decisions about principals. If a principal evaluation system is designed to use student test scores, measures based on student growth are preferable because they are stronger predictors than end-of-year achievement.


The study conducted two sets of analyses using Pennsylvania's statewide data on students and principals from 2007/08 to 2013/14. First, using data on 2,424 principals, the study assessed the extent to which ratings from each measure are stable by examining the association between principals' ratings from earlier and later years. Second, using data on 123 principals, the study examined the relationship between the stable part of each principal's rating and his or her contributions to student achievement in future years. Based on results from both analyses, the study simulated each measure's accuracy for predicting principals' contributions to student achievement in the following year.

The study found that the two performance measures that did not account for students' past achievement—average achievement and adjusted average achievement—provided no information for predicting principals' contributions to student achievement in the following year.

The two performance measures that accounted for students' past achievement—school value-added and adjusted school value-added—provided, at most, a small amount of information for predicting principals' contributions in the following year, with less than one-third of each difference in value-added ratings across principals reflecting differences in their future contributions.

Projections of Education Statistics to 2024

Enrollment in elementary and secondary schools rose 4 percent between 1999 and 2012 and is projected to increase an additional 5 percent by 2024, according to a new report released today (Sept. 29).

Projections of Education Statistics to 2024, provides national-level data on enrollment, teachers, high school graduates, and expenditures at the elementary and secondary level and enrollment and degrees at the postsecondary level for the past 15 years and projections to the year 2024. This is the 43nd edition of the publication, which is a product of the National Center for Education Statistics in the Institute of Education Sciences. .

Other findings include:

* Postsecondary enrollment rose by 37 percent between 1999 and 2013, and is projected to increase by another 14 percent by 2024;

* The number of high school graduates increased by 22 percent between 1999-2000 and 2011-12, and is projected to increase 3 percent by 2024-25; and,

* After adjusting for inflation, current expenditure per pupil increased by 15 percent between 1999-2000 and 2011-12, with a further increase of 13 percent is projected by 2024-25.

Implicit bias may help explain high preschool expulsion rates for black children

Preschool teachers and staff show signs of implicit bias in administering discipline, but the race of the teacher plays a big role in the outcome, according to research conducted by the Yale Child Study Center. The results help explain why black students tend to be suspended at much higher rates than white students, the authors say.

Researchers used sophisticated eye-tracking technology and found that preschool teachers “show a tendency to more closely observe black students, and especially boys, when challenging behaviors are expected,” the authors found.

But at the same time, black teachers hold black students to a higher standard of behavior than do their white counterparts, the researchers found. While the study did not explore why this difference in attitude exists, the researchers speculated that black educators may be demonstrating “a belief that black children require harsh assessment and discipline to prepare them for a harsh world.”

White educators, by contrast, may be acting on a stereotype that black preschoolers are more likely to misbehave in the first place, so they judge them against a different, more lenient standard than what they’re applying to white children.

“The tendency to base classroom observation on the gender and race of the child may explain in part why those children are more frequently identified as misbehaving and hence why there is a racial disparity in discipline,” added Walter S. Gilliam, director of The Edward Zigler Center in Child Development and Social Policy and associate professor of child psychiatry and psychology at the Yale Child Study Center. Gilliam is one of five researchers who conducted what is thought to be the first such study of its type.

Findings suggested that when the preschool teacher and child were of the same race, knowing about family stressors led to increased teacher empathy for the preschooler and decreased how severe the behaviors appeared to the teacher. But, when the teacher and child were of a different race, the same family information seemed to overwhelm the teachers and the behaviors were perceived as being more severe.

“These findings suggest that teachers need support in understanding family struggles, as they may related to child behaviors, especially when the teacher and child are of different races,” Gilliam said.

Wednesday, September 28, 2016

Impact of teacher professional development on student achievement in math

Content-intensive professional development (PD) in math boosted fourth-grade teachers’ subject knowledge and some aspects of instructional quality, but did not have a positive impact on student achievement, according to a new report.

The National Center for Education Evaluation and Regional Assistance (NCEE), in the Institute of Education Sciences, released a new evaluation report today (Sept. 28) that examines the effectiveness of a content-intensive PD program. The PD program included an 80-hour summer workshop (Intel Math) that focused on grades K–8 math, as well as 13 additional hours of collaborative meetings focused on analyzing student work and one-on-one coaching based on observations of teachers’ lessons. More than 200 4th-grade teachers from six districts in five states were randomly assigned to either a treatment group that received the study PD or a control group that did not.

Key findings include:

•    The PD had a positive impact on teacher knowledge: Average scores on a study-administered math test were 21 percentile points higher for teachers who received the study PD than for those who did not;

•    The PD had a positive impact on some aspects of instruction: Average ratings of teachers’ use and quality of math explanations in the classroom were 23 percentile points higher for teachers who received the study PD than for those who did not; and

•    The PD did not have a positive impact on student achievement: Students of teachers who received the PD scored, on average, 2 percentile points lower than control teachers’ students on both a study-administered math test and state assessment. In general, this difference was not statistically significant.

Tuesday, September 27, 2016

SAT® participation increases for test-takers in 2016

Over 6.7 million test-takers completed the SAT® or a PSAT-related assessment during the 2015-16 school year, according to the 2016 SAT Suite of Assessments results released today. The SAT Suite of Assessments, which launched in 2015, includes the new SAT, PSAT/NMSQT®, PSAT 10™, and PSAT 8/9™, and helps millions of students and educators set an earlier trajectory for college and career readiness. This is because it focuses on areas that research and evidence show matter most and reflects what students are already learning in class.

More test-takers completed the new SAT from March through June of this year than took the old SAT during the same period in 2015, which is a substantial show of support for the test’s redesign. Nearly 1.36 million test-takers took the new SAT in 2016, compared to 1.18 million who took the old SAT in 2015. This is a jump of approximately 180,000 SAT takers.

“We have transformed all of our tests to deliver greater opportunities and to clear a path for students to succeed in college and careers,” said College Board President and CEO David Coleman. “No other assessments provide students access to hundreds of millions of dollars in scholarships and free, personalized practice.”

Unprecedented numbers of students now have more and better access to opportunity because an increasing number of states, schools, and districts offer the SAT or a PSAT-related assessment to their students. The number of students who took the SAT during a school day more than doubled: there were over 458,000 SAT School Day participants in 2015-16, compared to nearly 219,500 in 2014-15. Twelve states, the District of Columbia, and 561 districts administered the new SAT or a PSAT-related assessment to their students during a school day in 2015-16, compared to eight states, the District of Columbia, and 429 districts that administered the old SAT or a PSAT-related assessment in 2014-15. It is anticipated that in 2016-17, the SAT or a PSAT-assessment will be administered during a school day in 13 states, the District of Columbia, and more than 700 districts.

“We’re seeing great momentum working with states and districts to create a pipeline for students to become college ready,” said Cyndie Schmeiser, chief of assessment for the College Board. “We are providing educators with information that better equips them to identify students who are falling behind and allows them to intervene and differentiate instructional work much earlier in high school and middle school.”

Connecting Students to Opportunities

In addition to providing more information about students’ college readiness, College Board assessments connect students to opportunities such as test and college application fee waivers, challenging course work, scholarships, and free, personalized practice.

Since the College Board’s Official SAT Practice on Khan Academy® launched in June 2015, more than 1.7 million unique users have signed up to practice for the SAT — free of charge. In nearly half of all U.S. high schools, students are logging on to the site for productive practice.

“Now, preparing for college is the same as practicing for the SAT,” said Schmeiser. “For the first time, there is equal access to world-class SAT preparation. And it’s free. Our data show more students across all ethnicities and income brackets prepare for the SAT with Khan Academy than with all commercial test-prep courses combined.”

According to our surveys of SAT takers, nearly 60% who took the new SAT this spring and practiced for the test reported using Official SAT Practice to prepare, compared to 19% who used commercial test prep, and there was a 10% drop in the number of students who paid for SAT prep resources.

This year, students taking the first administration of the redesigned PSAT/NMSQT and the new PSAT 10 had access to nearly $180 million in combined annual awards through our new scholarship partners. These new partnerships, combined with the College Board’s longstanding partnership with National Merit Scholarship Corporation, create a portfolio of scholarship opportunities that reach every major segment of students. For example, as a result of taking the PSAT/NMSQT or PSAT 10, more applicants than ever before were identified as potential recipients of Jack Kent Cooke Foundation scholarships. In 2016, 32% of the Cooke Scholars were identified because they took the PSAT/NMSQT or PSAT 10, and each received a full $40,000 a year in scholarships.

Since 2007, income-eligible students have received eight free score reports. Research shows that these students are more likely (by 10 percentage points) to send eight or more score reports to colleges than was the case before this free score report initiative was implemented.

Other participation and performance highlights from this year’s report:

Old SAT Results: Class of 2016

Because of the SAT redesign, this year’s performance data from the graduating class of 2016 includes students who took the old SAT through its last administration in January 2016. Comparisons of SAT results for the class of 2016 to those of previous graduating classes cannot be made because the number of test administrations and the characteristics of the class cohort are different from those in the past.

To provide a more accurate comparison, below are the mean scores for students in the graduating class of 2016 who took the SAT at least once through January 2016, and mean scores for students in the graduating class of 2015 who took the SAT at least once through January 2015.

  • The old SAT critical reading mean score was 494 for the class of 2016, compared to 497 for the class of 2015.
  • The old SAT math mean score was 508 for the class of 2016, compared to 512 for the class of 2015.
  • The old SAT writing mean score was 482 for the class of 2016, compared to 487 for the class of 2015.
In all three areas, the mean scores for those 2016 high school graduates who took the old SAT through January 2016 are slightly below the average scores for the 2015 high school graduates who took the old SAT through January 2015.

PSAT-Related Assessment Results

The PSAT/NMSQT was redesigned alongside the SAT and administered for the first time in fall 2015. Over 4 million students took the redesigned PSAT/NMSQT in fall 2015, more than the previous high volume of 3.8 million in the fall in 2014.

The data below report the percentage of students who met or exceeded the new College and Career Readiness Benchmarks for their particular grade and assessment taken. There are two section-level benchmarks for the SAT and the grade-level benchmarks: one for the Math section and one for the Evidence-Based Reading and Writing section. Overall college readiness is defined as achieving both of the section-level benchmarks. The below figures represent the percentage of test takers by grade and assessment taken who met or exceeded both section-level benchmarks.

Because the groups of students are those who chose to take each assessment and are not nationally representative, the percentages of college-ready students across the assessments cannot be compared.


  • In 2015, 38.5% of 10th-grade students who took the PSAT/NMSQT in the fall met the new 10th-grade College and Career Readiness Benchmark.
  • In 2015, 41.6% of 11th-grade students who took the PSAT/NMSQT met the new 11th-grade College and Career Readiness Benchmark.
  • In spring 2016, 38.2% of 10th-graders who took the PSAT 10 in the spring met the new 10th-grade College and Career Readiness Benchmark.
  • In 2015-16, 32.1% of eighth-graders who took the PSAT 8/9 met the new eighth-grade College and Career Readiness Benchmark. For ninth-graders who took the PSAT 8/9, 37.6% met the new ninth-grade College and Career Readiness Benchmark.

“The percentages we’re seeing from the PSAT-related assessments indicate that there are still far too many students who are not on target to become college ready by the time they graduate from high school,” said Schmeiser. “We still have much work to do to help these students prepare for success after high school.”

AP Potential™:

  • There was a 7% increase in the number of 11th-grade PSAT/NMSQT takers who showed high potential to succeed in at least one AP course based on their test results, from 750,524 in 2014 to 802,642 in 2015.
  • There was an 8% increase in the number of 10th-grade PSAT/NMSQT takers who showed high potential to succeed in at least one AP course based on their test results, from 519,668 in 2014 to 561,469 in 2015.

New SAT:

The College Board redesigned the SAT to make it more straightforward and connected to classroom learning. It remains a strong indicator of college readiness for all students. Some of the changes reflected in the new SAT include removing the guessing penalty, focusing on words students will use in college and careers, and making the essay optional.

Following the debut of the new SAT, the College Board surveyed approximately 70,000 test-takers and found that students not only prefer the new test to the old but support changes to the test to make it more focused and clear:

  • By a 7-to-1 margin, students said they preferred the format of the new SAT over that of the old SAT.
  • 80% of students reported feeling confident going into the new SAT.
  • 77% said the vocabulary on the new test would be useful to them later in life.
  • 75% said the Reading Test was the same as or easier than they expected.
  • 72% said the new test reflected what they’re learning in school.
This year’s new SAT data cannot be compared to that of previous years because the redesigned SAT is a different assessment from the old SAT. Moreover, the scale that has been established for the SAT Suite of Assessments is a new scale.

Monday, September 26, 2016

Public School Quality and Real Estate Values

Over the years 2000 to 2013, the Los Angeles real estate market featured a boom, a bust, and then another boom. This study uses this variation to test how the valuation of school quality varies over the business cycle. 

The study finds that the capitalization of school quality is counter-cyclical. While good schools always command a price premium, this premium grows during the bust. Possible mechanisms for these findings include consumers "trading down" from private to public schools during contractions as well as the effects of reduced household mobility during downturns in raising the value of the public school option. 

Wednesday, September 21, 2016

Identifying and describing emerging technologies that could impact learning, teaching, and creative inquiry in K-12 education

The New Media Consortium (NMC) and CoSN (Consortium for School Networking) have released the NMC/CoSN Horizon Report > 2016 K-12 Edition and Digital Toolkit.

The new edition reveals annual findings from the NMC Horizon Project, an ongoing research project, designed to identify and describe emerging technologies that could impact learning, teaching, and creative inquiry in K-12 education. Both organizations have also collaborated to produce an implementation toolkit to help educators put ideas from the report into action. The report and toolkit are made possible by Share Fair Nation under a grant from the Morgridge Family Foundation.

Six key trends, six significant challenges, and six important developments in educational technology are identified across three adoption horizons over the next one to five years, giving school leaders, educational technologists, and teachers a valuable guide for strategic technology planning. The report provides in-depth insight into how trends and challenges are accelerating and impeding the adoption of educational technology, along with their implications for policy, leadership, and practice.

“Teachers, administrators, and policymakers worldwide will use the report to inform critical conversations and develop progressive strategies to meet the needs of today’s learners,” says Samantha Becker, NMC Senior Director of Publications & Communications and co-principal investigator for the report. “This edition reveals that more schools are implementing active learning approaches, transforming pedagogies and teachers’ roles in the classroom. We’re excited by how technology is enabling more students to apply creativity and critical thinking to address global issues.”

“The flagship Horizon Report, along with the new practical toolkit, give educators the insight to lead and take advantage of emerging educational innovations,” notes Keith Krueger, CEO of CoSN and co-principal investigator for the report. “The toolkit will better allow school leaders to talk about these technologies right in the backyards of their communities. We are proud to partner with NMC on this forward-thinking effort and help school leaders make effective use of the tools needed to create and support modern, digital learning settings.”

The trends, challenges, and important developments in technology featured in the report are summarized below. New to this year’s edition and the NMC Horizon Report series in general are topics addressing digital equity, the achievement gap, and artificial intelligence, among others.

Key Trends Accelerating K-12 Educational Technology Adoption
The NMC/CoSN Horizon Report > 2016 K-12 Edition identifies “Coding as a Literacy” and “Students as Creators” as short-term trends accelerating the adoption of educational technology in K-12 education over the next one to two years. “Collaborative Learning” and “Deeper Learning Approaches” are mid-term trends expected to drive technology use in the next three to five years; meanwhile, “Redesigning Learning Spaces” and “Rethinking How Schools Work” are long-term trends anticipated to impact institutions for the next five years or more.

Significant Challenges Impeding K-12 Educational Technology Adoption
Several challenges are barriers to the mainstream use of technology in schools. “Authentic Learning Experiences” and “Rethinking the Roles of Teachers” are perceived as solvable challenges — those which we both understand and know how to solve. “Advancing Digital Equity” and “Scaling Teaching Innovations” are considered difficult challenges, which are defined and well understood but with solutions that are elusive. Described as wicked challenges are the “Achievement Gap” and “Personalizing Learning,” which are complex to define, much less to address.

Important Developments in Educational Technology for K-12 Education
Additionally, the report identifies makerspaces and online learning as digital strategies and technologies expected to enter mainstream use in the near-term horizon of one year or less. Robotics and virtual reality are seen in the mid-term horizon of two to three years; artificial intelligence and wearable technology are seen emerging in the far-term horizon of four to five years.

The report’s findings were identified through a qualitative research process designed and conducted by the NMC that engaged an international body of experts in K-12 schools, technology, business, and other fields. The subject matter collaboration focused on a set of research questions designed to surface significant trends and challenges and to identify emerging technologies with a strong likelihood of adoption in K-12 education. The NMC/CoSN Horizon Report > 2016 K-12 Edition details the areas in which these experts were in strong agreement.

The NMC/CoSN Horizon Report > 2016 K-12 Edition and Toolkit are available online, free of charge, and are released under a Creative Commons license to facilitate their widespread use, easy duplication, and broad distribution.