The Assessments Are Rigged

We all know that polls can be skewed and that ‘what everybody knows’ may not be so. Similarly, assessments and assessment data can be gathered, used, and presented in various ways to feed an agenda.  Just because a child is said to be proficient on a state assessment doesn’t mean he or she actually is ‘proficient’ in the way parents want him or her to be.

When I was in school, my teachers would give us tests to help figure out how much of what they were teaching we had actually learned.  Then, the state stepped in and started giving assessments to make sure teachers were teaching what the state wanted them to teach.  And now?  We’re told the assessments are great, but we are just supposed to trust.  We can’t see the assessment questions.  The algorithms (mathematical formulas) determining which questions come next or whether you have a higher or a lower score are kept secret. The State Boards of Education or the assessment vendors, themselves, can move and change the ‘proficiency’ levels at will.

We take it on faith when a student passes a math assessment it means the student is proficient.  Is it possible to rig an assessment?  Not only is it possible, but it’s also being done all the time.  I have four examples of how the assessments are and have been manipulated to provide different results than most people expect.  This is being done without oversight, without insight into what is occurring, and certainly without permission from parents.

The first example is assessing not just what a student is supposed to know but making them do the problem in a particular way. Ask yourself, does this create a disadvantage for a child who knows the math facts but hasn’t been shown a particular way of doing things?

This problem is an example of a Common Core Math Standard from First Grade:

Add and subtract within 20, demonstrating fluency for addition and subtraction within 10. Use strategies such as counting on; making ten (e.g., 8 + 6 = 8 + 2 + 4 = 10 + 4 = 14); decomposing a number leading to a ten (e.g., 13 – 4 = 13 – 3 – 1 = 10 – 1 = 9); using the relationship between addition and subtraction (e.g., knowing that 8 + 4 = 12, one knows 12 – 8 = 4); and creating equivalent but easier or known sums (e.g., adding 6 + 7 by creating the known equivalent 6 + 6 + 1 = 12 + 1 = 13).   

This question doesn’t just assess whether a student knows how to do an addition word problem, but it assesses whether a student has been trained on the Making Ten Strategy as outlined in the standard.  Could a student solve 8+6 without knowing the Making Ten Strategy?  Yes, of course.  Does using the Making Ten Strategy indicate critical thinking?  Or does it simply indicate a student has been instructed in this strategy?  Would you be able to succeed as a mathematician without learning this Making Ten Strategy in First Grade? Have you successfully used addition in your life without thinking about the Making Ten Strategy?

Many parent complaints about Common Core Math come from having to show the various methods for getting the answer or having to explain why an answer is correct.

Parent:“When I was in school, we did it this way.”

Child: “I have to do it this other way or it will be marked wrong.”

One mother asked her child’s teacher if he could simply do the standard algorithm on all his math homework because the multiple strategies were causing him stress.  The teacher said if he didn’t learn the strategies, he wouldn’t do well on the state assessment.  Once the mother indicated her child would not be taking the assessment, the teacher readily agreed to give credit for just the standard algorithms.  The reason for the multiple methods?  To do well on the assessment.

A review written in 2011 by Dr. Stephen Wilson of Johns Hopkins University states the following about the Common Core SBAC test (then under development).  He says, “It appears that the assessments will focus on communication skills and Mathematical Practices over content knowledge.”

Furthermore, “Mathematical Practices, or what was usually called ‘process’ standards in most states, do little more than describe how someone pretty good at mathematics seems to approach mathematics problems. As stand-alone standards, they are neither teachable nor testable. Mathematics is about solving problems, and anyone who can solve a complex multi-step problem using mathematics automatically demonstrates their skill with the Mathematical Practices, (whether they can communicate well or not).”

In short, we see Dr. Wilson’s concerns demonstrated in the above example: the process of getting the answer is of greater importance than the actual mathematical abilities most people think the assessment should be assessing.

A second example comes from Utah’s SAGE (end-of-year) sample assessment for Third Grade. This question is supposed to assess a deeper understanding of division than simply asking if a child knows the answer to 12 ÷ 4. Unfortunately, in creating a more convoluted problem, the assessment question can be solved without knowing anything more than how to count and how to write a division problem. Division facts, themselves, are not necessary.

There are lots of kids who can divide things equally by putting them in different boxes without knowing 12 ÷ 4 = 3.  Supposedly, by dragging the stars and dragging the numbers, you are assessing higher-order thinking.  But what you are really assessing is the child’s familiarity with the software interface, the format of the problem, and whether they can count and relate counting to division.  But they don’t have to know 12 ÷ 4 = 3.

Would a child who knows her division facts be able to do this problem anyway?  Most likely.  However, it is also true this question doesn’t distinguish the child who does know her math facts from the one who does not.

A third example has to do with reading comprehension.  It dates back to the 1980’s but illustrates that what is on an assessment and how it is asked can be used to manipulate and ‘direct’ a student’s thought processes.  I quote Dr. Peg Luksik who worked for Pennsylvania’s Department of Education.  From her video :

‘A sample question said: “There’s a group called the Midnight Marauders and they went out at night and did vandalism. I (the child) would join the group IF…”

“…my best friend was in the group.”

“…my mother wouldn’t find out.”

There was no place to say they would not join the group. They had to say they would join the group.’

Dr. Luksik states that while this was listed as a citizenship assessment, the internal documents stated, “We’re not testing objective knowledge. We are testing and scoring for the child’s threshold for behavior change without protest.”

Additionally, Dr. Luksik discusses another state’s Reading Assessment question: “If you found a wallet with money in it, would you take it?”

She asked, ‘Do you read better if you say “yes”? Or do you read better if you say “no”? Or were they assessing a child’s honesty on a state assessment with their name on it…?’

Clearly, these are examples of assessment questions that were not assessing either citizenship or reading as you and I would define them.

And finally, before a single Utah student took the state’s SAGE assessment in 2014, the head of state assessments warned local school board members that student test scores were going to drop by 10 or 20 points.  He also stated there was no way to correlate the previous test results with the SAGE results.  So, how did he know this?  The point was they knew what the target proficiency rate was.  Utah was looking for a proficiency rate in the 40’s.  And as they went through the process of setting those proficiency scores, they did so after the first round of testing. Then they modified the scoring to make sure the result fell within that 40% range*.  So, in one year, did Utah kids lose 20 points of knowledge?  Or does it simply mean the Powers That Be decided only 40% of the kids got to be labeled ‘proficient’ regardless of what they actually knew?

The only sure way of knowing an assessment is truly measuring academic content and grading it appropriately requires transparency with the assessment questions, the assessment methodology, and independent verification procedures.

Instead of wondering how kids are doing on state assessments and whether a school is “good” based on the assessment scores, we need to be asking what are these assessments supposed to be measuring and how do we know they really are measuring what they claim?

*Alpine School Board Study Session Audio September 23, 2014, Additional Media->Study Session @ 45 minutes. http://board.alpineschools.org/2014/09/18/september-23-2014-board-meeting/

Cross-post.

9 thoughts on “The Assessments Are Rigged

  1. As a School Psychologist for over 40 years, I’m very concerned about the perversion of Psychological Assessments, which we are currently seeing with the Common Core. Whatever happened to ethics in testing?
    It used to be, that Psychological assessments were only administered to children who were of special concern, after the proper procedures were followed, including: Individual referral, Informed parental consent, privacy safeguards and complete interpretation of results.
    Now we see psychological assessment being stealthily integrated into group testing and administered nationwide to 95% of children by law. The resulting “data” is uploaded to multiple databases where it will now be available in perpetuity for “research” (and hacking) by nearly anyone.
    This generation is being stripped psychologically naked, leaving them vulnerable to abuse from whomever or whatever chooses to exploit them. This is a human rights violation on a massive scale.

    1. I have refused Naviance for my children and I have refused for them to take part in SEL assignments in school. Algebra classwork and homework assignments are Growth Mindset worksheets weekly…not one number or math fact on them! I have told the teacher and Principle that if they continue to give these assignments, I will go to a school board meeting and display these assignments to the public (I have taken photos of these assignments) to let them know what is going on in math classes. Magically, my child is excused from these assignments and it doesn’t count against his grade.

  2. BINGO! But now to get all the parents to realize that they’ve “been had”. When will the teachers start talking to the parents (instead of among themselves)? When will school administrators start taking a stand? When the people working in the war zone (public schools) start the revolt, the parents will take notice. As it stands, the powers that control this have done a very good job of convincing the general public that public schools are a failure. Parents have bought into the fact that their children are nothing more than a number on a stupid standardized test that is controlled by evil doers that want nothing more than to keep the common man down. The whole test and punish scheme is anti-democratic. REFUSE the tests. OPT OUT….deny the beast the flawed data it needs to thrive.

    1. Lisa the real challenge is not how to get the information to parents but how to get parents to care enough to act on it. I say start pulling out kids out in huge numbers and STARVE THE BEAST. We need to implode the system and start all over. It is too late to fix it. We were awakened too late. We can do. There are so many ways to make this work for all parents including working parents. It takes the communities to come together and help each other. There are a ton of retired teachers and baby boomers that could get up off the sofa and help. So many ways to make it happen. I have a website I put up about 2 years ago and I update it regularly. http://www.starvethebeastusa.com It gives some ideas. Freedom Project Academy is the best deal in town. Open a room in a church, open up a your home. FPA is affordable, teacher led, accredited on-line private school. Christian, classical education. No Common Core, No data collection or sharing, they take no state or federal money. Limited computer time. Encourage parent engagement in the process. All students are tested for appropriate grade level. $2000 per year per student. They also give scholarships to needy families. WE CAN DO THIS. Time to STARVE THE BEAST and do it now. Public school is no longer public school it is a federal institution. We can turn this around.

      1. My son will be going to private school next year as he enters High school. No Common Core, no BS tests, no data collection, limited computer time, full time librarian and school nurse, emphasis on science, history, PE and the arts. It will cost us dearly, but it will be worth it. The private schools in our area are doing a booming business (some are overcrowded!) because they are catering to the unhappy public school parents and children. MD is a garbage public education state still tied to PARCC/CC and charters. We have Finn and Smarick sitting on our appointed school board. Don’t see any good changes in the near future even with the Governor’s race coming up. The Dems are just as bad as the Repubs.

        1. Alas, Common Core is in private schools now too. Also, it’s not just PARCC (AKA CRAPP). Even Texas people have said that they know that they have Common Core in Texas, even if it doesn’t go by that name. It’s being shoved down our throats by the Every Students Succeeds Act, which applies to all 50 states plus the 7 only Obama knows about.

Comments are closed.