ACT a Runaway Favorite over New SAT. Why?

Nearly three years ago, the College Board announced plans for a major redesign of the SAT. In less than three months, the new SAT will finally make its debut.

Tickets to the premiere are not selling well. Compass families — typically thoughtful and thorough when assessing their testing options — are overwhelmingly saying “No thanks!” and booking a seat for the ACT instead. A late surge in demand for the new SAT is unlikely at this point. By December of 11th grade, most Compass clients have solidified plans for the important spring testing window. The February ACT is out-polling the March SAT 20 to 1. By comparison, when the SAT last changed in 2005, the SAT was preferred 10 to 1.

Why the shift? As you’d expect from a group of savvy decision-makers, the rationales are both practical and insightful. A common theme is not fear but caution. The content of the new exam and the effort required to boost performance on the skills it assesses are well understood. Rather, a combination of logistical complications, inconvenient delays, and unfinished scaling of the tests are to blame for the new SAT’s poor reception. Families looking to remove stress from the standardized testing process tend to shy away from uncertainty.

The timing headache of the new SAT rollout in early March is the easiest problem to explain and appreciate. March has normally been a very popular test date among juniors. Students could receive PSAT scores in December, prepare for 2-3 months, receive their first SAT scores by the end of March, and plan the rest of the school year accordingly. In 2016, March scores are expected no sooner than mid-May. Waiting ten weeks for scores is awkward for 11th graders trying to make plans for re-testing, select a Subject Test date, firm up their college lists, and prepare for APs.

Further, a mid-May release of March scores is probably a best-case scenario. The College Board has struggled to meet their announced deadlines throughout this major transition; indeed, we learned today that the online release of PSAT scores — long slated for mid-December — has been delayed until January 7 due to technical problems. Students will not receive paper reports until January 29.

The turnaround of May and June scores is expected to improve to six weeks. This delayed reporting is still less than ideal, though, as most students will be into their summer break and will have less access to their school advisors when they finally receive their scores.

ACT’s menu of spring test dates – February 6, April 9, and June 11 – offers students a range of timing options, and ACT scores are expected to be released within the customary 2-3 week window. One caveat is that ACT had its own scoring fiasco in September because of problems associated with its new essay.

Delayed reporting is a nuisance but it alone is not enough to drive so many students from the SAT to the ACT. For many students, the more important factor is the respective portfolios of available practice tests and the integrity of the scales (scores) attached to the tests. College Board has more than a century of experience in grading college admission tests, so it is expected that they will “get it right” with the new SAT. However, it is difficult for families to trust a score that lacks context. Please watch your step as you follow me into the weeds of test construction.

Standardized tests like the SAT and ACT are intended to provide useful comparisons between students without regard to the particular form taken by a group of students on a particular test date. While it’s true that the raw difficulty of tests and the pool of testers varies slightly from test date to test date, the equating and norming processes employed in test development and scoring smooth away any differences. This is why you’ll never hear colleges saying “the March test was harder so scores from that date get more weight” or “the May test had a weaker pool of testers so we’re discounting those scores.” (You will occasionally hear this silliness from so-called test prep experts…but I digress.)

For the new SAT, the finalized scale cannot be published until the first administrations of the official test are complete and scores are released. This means that until mid-May, nobody (not even the College Board) really knows how a raw score on a particular form for the new SAT maps to a scaled score on the 200-800 range. We also can’t be sure yet that the distribution of scores will closely match the familiar bell-shaped historical distribution, as depicted below.

We will be shocked (and college admission officers would be dismayed) if the distribution shifts dramatically and 400 or 600 or 700 becomes the new mean. 500’s on the new test will probably be as commonplace as 500’s on the old test. The concern for students has less to do with an inability to perfectly compare the new SAT to the old SAT but rather how that uncertainty flows through to comparisons between the new SAT and the ACT.

Students should choose between tests based on a careful assessment of which test suits them better. This assessment is informed by a number of factors – some qualitative and some indirectly related to the test itself – but most crucially by a quantitative evaluation of practice test results. Ongoing practice tests then become a critical component of the test prep regimen.

For current 12th graders, this assessment was straightforward. They took well-established and reliable practice tests for the ACT and the old SAT and then compared their performance using the concordance table that was developed years ago through collaboration by the ACT and College Board. A student in the sample scenario depicted below had a solid, reliable indicator that the ACT suited her better.

Notice that the column for the new SAT is empty until May 2016. And even in May, the relationship between new SAT scores and ACT scores will have to be derived by comparing each to the most closely corresponding old SAT score. This synthetic form of concordance is the best we’ll have until roughly 2019 when the first studies are completed that are actually based on a cohort of college freshmen who took both the ACT and the new SAT while in high school.

This means that the only way right now to even try to compare a score on a new SAT practice test to an ACT score is to first make a questionable assumption that a new SAT score of, say, 530 is the same as a 530 on the old SAT, and then use the old concordance table above to compare that 530 to the ACT.

Such a comparison is further flawed by the speculative nature of the 530 from the new SAT practice test in the first place. The College Board released four practice tests last spring for the new SAT. These were beta tests, and the rough scales (score conversion tables) provided were preliminary, as well. The mock tests and tentative scales from the College Board are certainly more useful than the junk (I’m being kind) that’s been put out by third party publishers, but as test prep resources, the College Board materials are simply not as robust or reliable as the live exams released by ACT over the last two decades.

A student choosing the new SAT over the ACT is relying on a series of steps with interrelated and shaky assumptions:

1. Take a “beta” practice test for new SAT and a released live exam for ACT
2. Estimate score for new SAT based on preliminary practice test scales
3. Take a leap-of-faith that new SAT score aligns with score on old SAT
4. Use a concordance table designed for the old SAT to compare the faith-based score with ACT
5. Decide that estimated new SAT score is so much stronger than ACT score that it’s worth accepting the uncertainties and compromises around new SAT

Having read this far, you can probably appreciate why fewer than 1 in 20 Compass students have run these numbers and concluded that the new SAT is the better choice. Some uncertainty will be addressed by the release of the scores for the PSAT that juniors took back in October. PSAT scores were supposed to be released by mid-December but have been delayed until Jan 7. Return to this blog then for a detailed assessment of what is revealed — the distribution of scores in particular. It’s likely that the pendulum will start to swing back slightly towards the SAT after students see their PSAT scores and start to feel more comfortable with what they’re facing.

Meanwhile, who are the students who are already choosing the new SAT over the ACT in spite of the drawbacks? The rationales vary widely and tend to be very specific.

• The student with learning differences who received approval of special accommodations by the College Board but was denied them by the ACT.
• The student with unavoidable conflicts with the ACT dates.
• The student with overwhelmingly strong evidence from practice tests that the new SAT is a better fit.
• The student who is toxicologically allergic to the ACT Science.
• The student for whom reading and comprehension speed is such an overriding factor that the slightly reduced pacing of the new SAT swings the decision.

These types of individual concerns can be identified and examined and may tip the scales, but they are the exceptions. Students taking the ACT are, in most cases, making the tactically correct decision. We are happy to help you make sure you are making the right choice too.