Placement Tests, HS GPA, and Multiple Measures … “Just the Facts”

We know that repeated statements are often treated as proven statements, even if the original version of the statement was not accurate. In other words, if you want people to accept your point of view … don’t worry about whether it is accurate, just make sure that your statement is repeated by lots of people over a period of time. Like “HS GPA is a better predictor than placement tests”.

The original message seems to have been based upon a CCRC report (“High Stakes Placement … “, 2012 by Clayton). https://ccrc.tc.columbia.edu/publications/high-stakes-placement-exams-predict.html The conclusion about tests versus HS GPA is this:

First, focusing on the first or second columns, which examine the predictive value of placement scores alone for slightly different samples, one can see that exam scores are much better predictors of math outcomes than English outcomes. The overall proportion of variation explained is 13 percent for a continuous measure of math grades, compared with only 2 percent for a continuous measure of English grades. This is consistent with the findings from previous research.

The date being referenced is this:

Not only is the ‘HS GPA is better’ not accurate for the original research (for mathematics), people never mention a fundamental issue:

The data for the 2012 study came from ONE “large urban community college system” (LUCCS)

Now, I don’t doubt the basic premise that including more variables can improve a decision (such as a test plus HS GPA). The problem is that the message “HS GPA is better” has been repeated so often, by so many people, that decision makers accept it as truth. The truthfullness depends a great deal on the decision being made — placement in English, or placement in Mathematics. The situation looks pretty clear (in the LUCCS data) for English, where using the HS GPA only seems a better thing. In Mathematics … not so much!

Researchers have developed models for placement in mathematics based on HS transcript data, though I’ve never seen a proven model using just HS GPA. The variables connected to these research models involve:

Specific mathematics courses completed in high school (especially grades 11 and 12)
Specific grades received in those mathematics courses
As a minor factor, the overall HS GPA

A good prototype of this scheme is the California “MMAP” work; see http://rpgroup.org/Portals/0/Documents/Projects/MultipleMeasures/DecisionRulesandAnalysisCode/Statewide-Decision-Rules-5_18_16_1.pdf . Rather than the ‘drive-off-the-cliff’ approach (North Carolina, Florida, etc), this is a scientific approach to a complicated problem. Few of our colleges, and few states, are willing to invest the resources necessary for this truth-in-multiple measures approach. [The fact that California can do this seems to have been a consequence of decisions about higher education in that state 50 and 60 years ago. We probably won’t see that again.]

Some additional truth about HS GPA:

High School GPA transmits inequity

Here is some data from the US Department of Education transcript study (2009):

The issue is not that HS GPA transmits inequity while placement tests do not. In the case of SAT Math (and ACT Math) the gaps are known to exist. The issue is that HS GPA transmits the inequity without regard to the student’s abilities in a subject domain.

The race/ethnicity ‘gaps’ for HS GPA are just one way to establish that it transmits inequity. Economic and geographical inequities are also apparent in the HS GPA data. At least the test developers strive to minimize their inequity; items which show a significant differential impact are removed from the tests.

Placement tests are less harmful to students than HS GPA.

The truth about multiple measures is that it will only help students when implemented in a scientific manner in the location or region involved. HS GPA by itself will harm students.

Developmental Mathematics Revival!

Bringing a new vitality to college mathematics 2011 to 2019