Did ACT Takers Get a Fair Shake at Georgetown?

Recent quotes by Georgetown’s Dean of Undergraduate Admissions Charles Deacon about class of 2021 applications and acceptances unwittingly setoff a firestorm in counseling and testing circles. Deacon seemed to say that Georgetown had intentionally or unintentionally favored new SAT takers over ACT takers because of the higher scores produced by the redesigned exam. He implied that many students had been led astray and should have stayed with the SAT.

Nancy Griesemer wrote an interesting piece about the topic on her admissionintel.com blog. While some of the statements made by Deacon imply a misunderstanding of the new test, I’m inclined to see his opinions in the context of how many admission offices struggled with the new SAT and the difficulties they face when reporting old and new SAT scores or when making comparisons to the ACT.

University of Lake Wobegone

If we believe admission office press releases, all colleges are above average. No dean ever complains of a falloff in quality. The “inflation” of new SAT scores means that it is is going to be easy to put a good face on SAT averages this year. Virtually every college can claim its “highest scores ever.” Behind the scenes, there is more stomach churning among admission deans. When adjusted by the concordance, most colleges will see a decrease in SAT scores. To no one’s surprise, this is not how colleges appear to be reporting scores. The problem is that they are providing little clarity about what the scores mean.

In some cases, it appears that colleges are simply reporting the 400-1600 scores of the new test (EBRW + Math) as if they were the 400-1600s of the old exam (Critical Reading + Math). This does not mean that the admission offices did the same thing when evaluating applicants. In fact, I think it unlikely that any college treated the raw scores interchangeably during the admission process. Deacon bragged about the 16 point improvement in SAT scores within the applicant pool. Dartmouth touts a record high average of 1495 (up 17 points). They have glossed over the fact that the SAT definitions have changed. What percentage of the scores were old SAT, what percentage were new SATs, and how were the two combined? In Dartmouth’s case, the 1495 is clearly being compared to the old SAT total of 1478 for the class of 2020.

University of Virginia, which can usually be counted on for clarity on testing, has specifically said that the reported scores for the class of 2021 are for the new SAT, since they saw relatively few students with old SATs. The average new SAT score of accepted students is 1419. UVA’s accepted students for the class of 2020 had an average old SAT score of 1400. Students and parents need to be careful when comparing the scores of accepted students — used in the press releases that come out this time of year — with the guidebook or Common Data Set figures that are for enrolled students. The latter figures are always lower. For example, UVA enrolled students had a mean SAT score of 1346 (25th-75th percentile scores of 620-720 CR and 620-740 M).

Defending the ACT

The original article in The Hoya, Georgetown’s student newspaper, shows the danger of extemporizing on a topic adjacent to one’s area of real expertise. Many counselors were rightly offended at the implication that they had provided bad advice when recommending students take the ACT. The offense was heightened by the fact that Deacon’s criticism seemed to reveal his own misunderstandings or his dependence on the historian’s fallacy (Monday morning quarterbacking).

In keeping with sports metaphors, there may be a hint of best-defense-is-a-good-offense. If Georgetown was not able to properly sift ACT and SAT takers, the fault should not fall at the feet of counselors or students. If Georgetown’s average applicant only improved by 16 points on the new scale whereas overall scores went up 50-60 points, it impairs the citius, altius, fortius narrative.

I don’t believe Georgetown created an intentional SAT bias. At no point does Deacon say that Georgetown preferred the SAT over the ACT.  He also speculated — based on the scant evidence of an uptick in College applications — that “maybe people are getting less worried about getting a job.” Those are the sorts of sound bites expected in a piece such as this — statements that would never be expected to pass academic muster in a different context and that we should not take too seriously.

Distant Cousins, Not Twins

It is difficult, however, to square his statements in the The Hoya with a proper understanding of the relationship between the old SAT and the new SAT. We’ve been pleading with anyone who would listen over the last 2 years not to think of the new SAT as just another SAT. Other than the 3 famous letters in the name and the 3 famous digits in the scores, the old SAT and new SAT are psychometrically distant cousins, at best. They are built with different structures, different item types, different scoring, different scaling (the new SAT was not normalized), and, most important, very different academic standards. It also seems odd to trust an organization with the total overhaul of the test, yet not trust it when it tells colleges how best to interpret scores.

Working Without a Concordance

A common misunderstanding is that a college’s rejection of “official” concordances means that they are treating old SAT and new SAT scores as identical. Colleges can still make reasonable decisions without a concordance — after all, they compare GPAs from thousands of high schools without such a tool. In some cases, colleges simply judge candidates versus similarly situated candidates. An “inflated” new SAT score doesn’t matter as long as the student is being evaluated versus other new SAT scores. Georgetown had earlier said that, although it would not be using the concordance, it would view old and new SAT scores with “context.” With thousands of students applying with all manner of scores — and with thousands having multiple types of scores — most colleges can make fair comparisons with or without a concordance. We’ve talked with other admission offices going through some of the same pains. Some used the concordances literally. Some have looked at students in separate pools. Some have supplemented decisions with other figures such as percentile ranks (although those have their own flaws).

This One Has Higher Numbers

Deacon seems to have a Nigel Tufnel problem in the interview.

A lot of people are advised by their high school counselors maybe to take the ACT, which we don’t think is great advice, because the ACT was unchanged but the SAT came out higher, so that could have had an impact,” Deacon said.

In other words, the new SAT is louder, because it goes to 11.

When followed by other questionable conclusions, though, it tends to make me think that much of the brouhaha boils down to no more than sloppy speculations. Regarding the higher SAT scores (from the score inflation of the new test), the Hoya paraphrased and quoted Deacon.

Deacon said this ended up making the applicant pool significantly more competitive.

“Almost all of the increase was from people who are in the top 10 percent of their high school class. What you see going on there is that as you become more competitive, people begin to weed themselves out. People with lower scores say, ‘Why bother?’ If they are high, they say, ‘Maybe this is a school I should be applying for,’” Deacon said.

If students have an inflated sense of their scores, most behavioral economists would state that more unqualified applicants would apply.

ACT’s Growing Importance at Georgetown

Despite Deacon’s statements, I think we will see a familiar class makeup at Georgetown. Looking back over recent figures for Georgetown shows that the percentage of enrolled students who submitted ACT scores rose to an all-time high of 47% for the class of 2019 (up from 40% for the class of 2018). Since 78% also submitted SAT scores, we can say that approximately 22% submitted ACT scores only, about 56% submitted SAT only, and 25% submitted both. It is not as though Georgetown is uncomfortable using ACT scores. From the figures mentioned in the article, it sounds like the university will see another record for the class of 2021 — “the number of students submitting ACT scores compared to SAT scores was split about evenly.” It will be interesting to see if the accepted or enrolled figures follow a similar split.

We are seeing the same large swings in the percentage of ACT versus SAT takers at virtually every competitive college. The class of 2017 felt deeply ambivalent about taking an untried test that lacked the extensive history and preparation resources of the ACT. No one likes being a guinea pig — except, maybe, guinea pigs.

ACT Scores Have Remained Consistent

So far, I have not seen reports of colleges showing higher average ACT scores. If admission decisions were truly biased against ACT students, we would see more demanding scores required for admission. This is particularly true given the significant increase in applicants with ACT scores. If there were an “ACT quota” or an SAT bias, the highest ACT scores should have risen to the top.

I think it unwise to draw any conclusions from a single article. I find it equally unwise to draw any universally applicable conclusions from Georgetown alone. Georgetown has always been the eccentric uncle of admission testing. The college that required the SAT II: Writing a decade ago refused to adopt Writing when it became part of the regular SAT. Despite the disappearance of the most popular Subject Test, the university kept the required (“strongly recommended”) number of Subject Tests at three. So they threw away one-third of the testing that students did while having higher Subject Test expectations than any other college in the country.

The Lingering Impact of the New SAT

The problems created by the new SAT will not end with the class of 2017. Average scores for colleges will be an anomalous mix of old SAT, new SAT, and ACT scores. Unless colleges report figures consistently — such as in the Common Data Set — students may not be able to make fair comparisons based on SAT scores. Even with accurate reporting, we can’t make absolute conclusions. The exact mix of tests will not be repeated — if for no other reason than that the old SAT no longer exists. Students will have a harder than usual time of judging where their test scores stack up at a college. Will SAT scores for admits in the class of 2022 be even higher? Will ACT averages stay where they are? Will there be continued — mostly unnecessary — hand-wringing in deciding between the SAT and ACT? My educated guesses are: yes, yes, and yes. We will be following the topic as more scores and information are released.