Truth in American Education

Fighting to stop the Common Core State Standards, their Assessments and Student Data Mining.

  • Home
  • About Us
    • TAE Advocates
    • Network Participants
    • Related Websites
  • Common Core State Standards
    • National Education Standards
    • Gates Foundation & NCEE Influence
    • State Costs for Adopting and Implementing the Common Core State Standards
    • National Curriculum
    • Common Core State Standards Content
      • Standard Algorithms in the Common Core State Standards
    • Myths Versus Facts
    • States Fighting Back Map
    • Closing the Door on Innovation
    • CCSSI Development Teams
  • Common Core Assessments
    • Opt Out Info
  • Race To The Top
    • District-Level Race to the Top–Race to the Top IV
  • Resources
    • Legislative Bills Against CCSS
    • Pioneer Institute White Papers
    • Model Resolutions
    • Parents’ & Educators’ Executive Order
    • CC = Conditions + Coercion + Conflict of Interest
  • Audio & Video
  • Privacy Issues and State Longitudinal Data Systems
    • Statewide Longitudinal Data Systems
  • ESEA/NCLB
    • Statements and Proposed Plans
    • Every Child Achieves Act July 2015
    • Student Success Act
    • Every Child Ready for College or Career Act
    • No Child Left Behind Waivers
    • ESEA Blueprint, Briefing Book, and Position Paper
  • Home School/Private School
  • Action Center
    • Parent and Community Action Plan
    • Stop CCSSI ToolKit
    • Sign Up or Contact TAE

US Education’s Dominant Research Method: Cherry Picking Evidence

April 12, 2019 By Richard Phelps

We all know about gerrymandering—the process by which politicians in a majority party sit down with a map and carve out electoral boundaries to maximize their party’s electoral advantage. Gerrymandering upends the electoral process. Rather than allow voters to choose their representatives, incumbent politicians choose their voters. More than 90 percent of US congressional elections are now noncompetitive, largely due to gerrymandering.

Some scholars employ a similar process in their research. Rather than allow all of the research “literature”—the full expanse of all relevant evidence on a topic—to lead them to a research conclusion, they reference only that part that supports their preferred conclusions. 

I call these scholars “dismissive reviewers,” because they ignore or declare nonexistent (i.e., dismiss) relevant evidence. When a group of dismissive reviewers cooperate they form a “citation cartel”—citing only each other’s research and dismissing all the rest. 

Readers of this blog already know that much—perhaps most—mainstream US education research is cherry picked. In part, that is how easily disprovable education myths persist. Education journals publish study after study that purports to consider all the relevant evidence on a topic but, in fact, references only that part of the evidence supportive of the myth. That proportion may be tiny, but it is still “evidence.”

This is why I find little reassurance in the phrase “evidence-based research.” All research is “evidence-based.” But, some is based on only part of the evidence available. Moreover, some of that is fraudulent. Education hosts a gargantuan quantity of research evidence. But, much is of poor quality. And, much more than most people realize is simply dishonest, with fabricated or doctored data, surreptitiously altered definitions of terms, selective references, and dismissive literature reviews. 

Despite its reputation as the most trustworthy of US education research sub-fields, the research conducted in education testing, or “psychometrics,” is no different. Some of it is poorly done. Some is biased. And, some cherry-picks its evidence to reach preferred conclusions. 

To my observation, honest, objective scholars still run things in the more technical realms of education testing research. In the realm of education testing policy, however, cherry pickers have run the show for over three decades. Moreover, they have managed to “capture” the testing policy research function at the National Research Council, the National Academy of Education, the World Bank, and the National Council of Measurement in Education (NCME), the primary US professional association of testing and measurement scholars in education. 

Indeed, just recently, NCME announced the names of the scholars who will write the testing policy section—”Accountability in K-12 Assessment”—of the next edition of the organization’s primary reference publication, Educational Measurement. NCME appears to have chosen a group that will assure continuity with past versions that reliably use cherry-picked evidence to advantage authors’ citation cartel. Furthermore, all four current authors and “reviewer-collaborators” have participated in Common Core promotion efforts and done work for the Gates Foundation.

By avoiding mention of rival evidence, and profusely referencing each other, citation cartel members can boost their own professional profiles, at the expense of other scholars’.

I have long been a strong advocate for education testing in general and standardized testing in particular. Yet, I would agree with many readers of this blog that US education testing policy has been sub-optimal since 2001 and remains so today. I do not share the conclusion that testing itself is responsible, however. Rather, responsibility lies with our country’s policymakers, in both major parties, who continue to rely on the advice of a relatively small group of policy analysts who limit their perspective to a pinhole of the available research evidence. 

Un-Rig the Research!

Filed Under: Education Reform Tagged With: education policy, testing and accountability

About Richard Phelps

Richard P. Phelps is the founder of the Nonpartisan Education Review. He is the author of four books and used to teach secondary school math back in the day.

Comments

  1. Niki Hayes says

    April 12, 2019 at 5:53 pm

    Thank you, thank you! I keep remembering that old saying in statistics: “Figures don’t lie, but liars do figure.”

    However, I had come to the point of accepting any education research that had been peer reviewed. Then I learned about the “old boy network” within the review circles. Now, I believe education research is like a bowl of jello. There’s plenty of shine and movement but no consistent direction. It offers little nourishment; kids like it because it’s fun to play with, teachers like it because the kids like it. And, very important, it does provide money to a vendor for providing its “substance.”

    How to determine what to believe on education issues today comes down to my gut feeling after 38 years in the field: “What are the proven results?” That means success must have happened across all identified subgroups for 3+ consecutive years in specified locations. Those on the ground involved—the schools, administrators, teachers, parents and students— must be willing to provide personal and clear evidence, if asked for such unexpectedly.

    If results have not been successful after three years, someone must pay a price for experimenting—again—with our children’s lives.

  2. Mommy says

    April 22, 2019 at 11:42 am

    There’s a lot of preschools using “evidence” to prove that children need education-enhanced daycare to succeed. Pay close attention, it’s always your child will do better in K, 1, or 2, which is often true–it can give many children an artificial boost. What they won’t tell you is that the boost wears off after 3rd grade. They will never present evidence that your child will do better in high school or college.

  • Email
  • Facebook
  • Phone
  • Twitter

States Fighting Back

https://app.box.com/s/10nl1409mkaf00zzzuyf

CCSS Opt-Out Form

  • Click here to download the CCSS Opt-Out Form

Campbell’s Law

"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

- Donald Campbell

Copyright © 2021 Truth in American Education · Developed & Hosted by 4:15 Communications, LLC.