in dianeravitch.net/2014/10/05/bob-shepherd-should-test-questions-have-only-one-right-answer/comment-page-1/#comment-2243253 NYS Teacher comments: There seems to be some confusion about the required use of “plausible distractors” when constructing MC test items. Only one of the three sets of response options below uses plausible distractors correctly. On set are a good example of what we are seeing in the Pearson ELA assessmensts. ________________________________________________ Shown below is a sentence using the word “flabbergasted”. Read the sentence carefully and use all the information in the sentence to help answer test question number one (1). Sean was flabbergasted to learn that he was ranked third in his graduating class. 1) Which word best describes the meaning of the word “flabbergasted” as it appears in the sentence provided? a) surprised b) astonished c) stunned d) amazed e) disappointed f) relieved g) delighted h) amazed i) pushed j) pulled k) twisted l) amazed The last set of response options ( i – l) shows why distractors need to plausible. In my early days, before I was trained as an item writer, I would commonly use silly or stupid, or cutesy, “implausible” terms in if I couldn’t think of a good distractor. That 3rd distractor is often difficult to produce. Not an uncommon practice with teacher created tests. The first set (a – d) is closer to what many of the Pearson items were like on their ELA test. When seasoned, veteran ELA teachers cannot agree on the correct response on multiple items, something is deeply wrong with the test. Ideally the terms “best” or “most likely” should not even be used in an item stem of an objective MC test. Pearson’s exams are littered with these – setting off alarm bells. If I submitted items like these to Measure Progress they wouldn’t have made it past the circular file. makes one wonder if this was just a hack job done on the fly or possibly intentional- in order to help produce the failure rate they wanted? As you have stated before, using an MC [multiple choice] test to measure the proficiency of subjective and abstract standards is simply the wrong tool for the job. Yet teachers careers and reputations now hinge on these poorly constructed tests. NYS should get its money back from Pearson, all test scores should be invalidated, and AIS assignments revoked. This should be a MAJOR SCANDAL. I can’t begin to imagine the blowback once the PARCC and SBAC tests hit the fan this spring.Teachers and parents need to be relentless advocates regarding test transparency. False claims of test security will be used to avoid test scrutiny. If academic “rigor” is so important to this reform movement, you’d think that they would apply some to their test development. Bob Shepherd responds: Here’s what I think should happen: some Congressperson should subpoena these testing companies and consortia, demanding copies of the ELA tests and of any reliability and validity “studies” performed on those, and then the tests should be subjected to item-by-item analysis and the studies should be subjected to competent peer review. Of course, the resulting reports would reveal that the tests in ELA are a complete scam being perpetrated on the taxpayers. The tests do not measure what they purport to measure and cannot as they are currently conceived.
Posted on: Tue, 07 Oct 2014 09:30:44 +0000
Trending Topics
Recently Viewed Topics
© 2015