I want to preface my criticism by acknowledging that College Board and ETS took what was a horrible situation — and that was their own doing, mind you — and made it better than it might have been. At the end of the day, the same cannot be said of any number of corporate mistakes.
And yet…
Most well-advised companies have learned a PR protocol for dealing with “disasters” — tainted food, product recalls, CEO meltdowns. College Board apparently missed many of those lessons.
1) Take Responsibility
When reports first started developing, College Board only put a small blurb on its website that it was investigating the problem and, essentially, putting all of the blame on ETS. On Monday, the release was updated to glibly minimize the problem and act as if College Board had been prepared for this all along. And it still kept the finger pointed at ETS. Students don’t care about blame; they care about scores and college applications. Emphasizing the role of ETS served to minimize College Board’s control over the SAT — not a good thing when it is making the most dramatic overhaul in 75 years. Pretending that the SAT was constructed to handle just this sort of doomsday scenario was tone deaf and implied that College Board had been over-testing students by a third for decades.
2) Put Senior Executives Out Front
Unsigned web posts and grudging statements from communication directors imply that the College Board did not take the issue seriously — an issue that impacts almost 500,000 testers! If College Board President David Coleman can’t be bothered to address the problem, what more important tasks is he working on?
3) Solutions, not Spin
Explain why the mistake happened, what steps have been taken to prevent it in the future, and what solutions are being offered. To its credit, College Board did come out with a partial solution on Monday. It did not handle everything well and did not think through the obvious follow-up questions from students — Will these scores hurt me? Can I cancel the scores? Can I retake the test for free? Have you surveyed your college membership to make sure that they’ll accept my scores? College Board has yet to provide any hard data to back up its claims.
Spin also created an obvious dissonance. College Board was quick to trumpet the fact that the multi-section structure of the SAT saved the day. One can almost hear the movie trailer voice in the statement that “test administrations can be fragile, so our assessments are not.” College Board was probably a little too quick to trumpet that fact. The redesigned SAT — the one that they have been heavily promoting and that will arrive in March 2016 — is not based on a multi-section structure, and cross-domain scores mean that sections are inter-dependent. One printing error on the new SAT and everything has to be wiped clean. It could be argued that the simpler structure — also in use by the ACT — will be less confusing and less prone to printing errors.
The idea that the College Board had an extra section in reserve is unsupported by any evidence. Nowhere in the original construction documents, decades of research, multiple technical manuals, and specifications for several test re-workings can I find any mention of superfluous questions added to make the test resilient to “fire drills and power outages to mistiming and disruptive behavior.” Creating hundreds of extra questions is expensive. Printing and shipping costs would go up. And proctoring and other administrative costs would be much higher. In short, College Board and ETS would have been spending millions of dollars each year to protect against a once-in-a-lifetime event.
4) Don’t let rumors and opponents take the lead. Get ahead of the story.
Rumors and fear swept through online forums and social media while the College Board remained mum. It has been in catch-up mode throughout the incident. College Board is compounding its pain by responding in bits and pieces. The error impacted 1 section. The error impacted 2 sections. Students might receive a free October test. They might not. They will. College Board kept referring back to a changing web post rather than explaining things fully to their audience. Not once has College Board tweeted any information about the situation. Every delay or change in the story hands a megaphone to SAT opponents such as Bob Schaeffer at FairTest. It must have left some folks at the ACT headquarters in Iowa City sniggering.
Earlier this spring ACT put out a piece extolling its own long-term consistency and making a not-even-thinly-veiled attack on the new SAT’s radical redo and the test’s previous revisions and score recentering. The title of the ACT piece sounds prophetic in retrospect: Why Scores on the ACT Test Are Scores You Can Trust. The fact is that this sort of printing error could have happened to ACT or Pearson or ETS, and no testing company is setup to deal with the scope of what developed on June 6. It happened to the SAT, though, and College Board needs to be the one addressing it fully.
Want to learn how students should respond to mishaps on test day? We have come up with six strategies that every student should know.
Dear Art,
Hello, my name is Heather, and I am a 46-year-old, master’s-degreed woman who really never gave a flip about her standardized test scores in high school. I recently discovered my SAT report (from November of 1991!) in a box in the attic. In particular, I noticed something strange about the report of percentiles (national vs. state): I think the college board may have made a mistake back in 1991, and while this error does not affect me personally, I wonder if it adversely affected anyone else.
In the columns for my verbal score, my percentiles were as follows:
National: 80
State: 65
Art, I took the test in Alabama, and it just doesn’t make sense that Alabama’s state average would be fifteen points higher than the national average. Scores on Math and TSWE follow the same pattern. Each category lists that my scores in Alabama were far worse than the national percentile. Does this sound odd to you as well? Like I said, this does not affect me in any way, shape, or form. My trajectory in life would not have changed at all. Yet, as a secondary, language arts teacher of nineteen years, I feel it necessary that the College Board admit it/they made a mistake. eeing as how standardized testing and subsequent textbook adoptions are such lucrative businesses.
What do you think?
Heather,
College Board likely reported the numbers correctly in your case. The quirk you are noticing has to do with the low number of students taking the SAT in Alabama, where the ACT is the dominant test. Recently, only 7% of Alabama high school seniors took the SAT. That figure was likely even lower in 1991. Alabama’s SAT scores are higher than the national average, because the students taking the SAT tend to be higher performers than their peers taking the ACT. Put another way, if a student is just taking an admission test because their high school expects them to, they’re likely taking the test everyone else does. Those taking the SAT are making a more conscious decision about attending a competitive 4-year college. We see this pattern repeated in other states, such as Iowa, with few SAT takers. In the other direction, NY seems to have an abnormally high ACT average because the students taking the ACT in NY tend to be those looking to get an edge at elite colleges by taking both the SAT and ACT.