This is a comment from Robert Harrell a few weeks ago, published here as an article so that I can link it directly to the assessment categories. Robert makes a very important point about how data collectors seem to often have to scramble to defend their jobs, or least that is the true folly he describes below:
I read an article (unfortunately I don’t have it or the URL handy) about a study that showed that over 70% of the score on standardized tests is the result of knowing how to take the test, not the result of content knowledge. That puts all standardized tests into question as a tool for testing student content knowledge. Pearson believed that the author of the study had made an error in his calculation of the percentage and began a huge campaign to discredit him. However, in their own materials Pearson admitted that the number is at least 50% knowledge of test taking. So, to protect their financial interests, Pearson worked hard (and fairly successfully) to discredit an entire study over a perceived difference in percentage even when by their own admission the test is fully 50% unreliable as an indicator of student content knowledge.
Here’s a link to an article by Diane Ravitch about this:
http://dianeravitch.net/category/pearson/
Here’s a link to the original article in the Texas Observer:
http://www.texasobserver.org/walter-stroup-standardized-testing-pearson/