Multiple Measures: How Consistent are ACT Math and Accuplacer

Like many institutions, mine allows students to place into a math course via a variety of methods. The most common methods are the ACT Math score and the Accuplacer College Level Math (CLM) test. I ran into a reference to a university which concluded that the ACT Math score was not a reliable predictor.

So, I’m posting a quick summary of how those two instruments agree (or not). As part of our normal program improvement and curricular work, I have gathered information on about 800 students who were enrolled in our pre-calculus course. Obviously, this is not a random sample of all ACT Math and all CLM scores. However, given the selection, the two instruments should have a reasonable amount of consistency.

There were 122 students with both ACT Math and CLM scores. Of these:

74 had scores on both that produce the same course placement (61%)
48 had scores such that different course placements result (39%)

The vast majority of the ‘disagreement’ involved a higher ACT Math placement than CLM placement. A quick comparison shows that students placing based on ACT Math have a lower pass rate than those who place based on the CLM. I’ve got some more work to do in analyzing the data before identifying a hypotheses about that pattern.

For that sample of 122 students with both scores, there is a significant correlation (about 0.32). That correlation is somewhat limited by the sample, which tends to emphasize relatively high scores (skewed distribution). Even with that limitation, I was concerned about the small size of the correlation … I’d expect a ‘native’ correlation (all data) of about 0.7, and a reduction to 0.5 would be reasonable given the skewed sample. That 0.32 is pretty small for these two measures.

Most of us use “alternate measures” (this method OR that method); low consistency between methods means our error rates will increase with the ‘or’. If the low consistency holds up in further analysis, we should either use the most reliable predictor … or true multiple measures where we use some combination of data to determine a course placement.

I began looking at our data because I could not find studies looking at the correlation and relative placement strength of our two measures. If you are aware of a study that provides that type of information, I’d appreciate hearing about it.

Developmental Mathematics Revival!

Bringing a new vitality to college mathematics 2011 to 2019