The Placement Test Disaster ?

For an internal project at my institution, I’ve been looking at the relationships between Accuplacer test scores, ACT Math scores, and performance in both developmental and college-level courses. Most of the results are intended for my colleagues here at LCC. However, some patterns in those relationships are important for us to explore together.

So, the first pattern that is troubling is this:

Students who place into a pre-calculus course based on their ACT Math score have lower outcomes than those who place based on the Accuplacer “College Level Math” test … and lower than those who needed to take intermediate algebra before pre-calculus.

We use the ‘college readiness’ standard on the ACT Math test of 22 (see https://www.act.org/content/act/en/education-and-career-planning/college-and-career-readiness-standards/benchmarks.html ). The pattern in our data for the ACT Math is similar to some references found at other institutions … though we tend not to talk about this.

Of course, the use of an admissions test (ACT or SAT) for course placement is “off label” — the admissions tests were not designed for this purpose. We tend to use the ACT option for placement in response to political pressure from administrators (internally) and from stakeholders (externally), and sometimes under the guise of “multiple measures”. The patterns in our data suggest that the ACT Math score is only valid for placement when used in a true multiple measures system … where two or more data sources are combined to create a placement. However, most of us operate under ‘alternative measures’, where there are different options … and a student can select the highest outcome if they wish; alternative measures is guaranteed to maximize the error rate in placement, with a single measure placement test almost always providing better results.

The second pattern reflecting areas of concern:

The correlations are low between (A) the ACT Math and Accuplacer College Level Math test, and (B) the Accuplacer Algebra and Accuplacer Arithmetic tests.

The second combination is understandable, in itself; the content of the Algebra and Arithmetic tests have low levels of overlap. The problem deals with our mythology around a sequence of math courses … that the prerequisite to algebra is ‘passing’ basic math. Decades of our research on algebra success provide strong evidence that there is little connection between measures of arithmetic mastery and passing a first algebra course. In spite of this, we continue to test students on arithmetic when there curricular needs are algebraic: that is a disaster, and a tragedy.

The first ‘low correlation’ (ACT Math, College Level Math) is not what we would expect. The content domains for the two tests have considerable overlap, and both tests measure aspects of ‘college readiness’. As an interesting ‘tidbit’, we find that a higher proportion of minorities (African American in particular) place into pre-calculus based on the more reliable College Level Math test compared to majority (white, who have a higher proportion placed based on the ACT Math) — creating a bit of a role reversal (whites placed at a disadvantage).

Placement testing can add considerable value … and placement testing can create extreme problems. For example, students with an average high school background will frequently earn a ‘college ready’ ACT Math score when they have too many gaps in their preparation for pre-calculus. A larger problem (in terms of number of students) comes from the group of students a bit ‘below average’ … who tend to do okay on a basic algebra test but not-so-good on arithmetic, which results in thousands of students taking an arithmetic-based course when they could have succeeded in a first algebra course (or Math Literacy).

Those two problems are symptoms of a non-multiple-measures use of multiple measures, where alternative measures allow students to select the ‘maximum placement’ while other measures (with higher reliability) suggest a placement better matched for a success situation.

As a profession, we are under considerable pressure to avoid the use of placement tests. Policy makers have been attacking remediation for several years now, and more reasonable advocates suggest using other measures. The professional response is to insist on the best outcomes for students — which is true multiple measures; if that is not viable, a single-measure placement test is better than either a college-admission test or a global measure of high school (like HS GPA).

And, all of us should deal with this challenge:

Why would we require any college student to take a placement test on Arithmetic, when their college program does not specifically require proficiency in the content of that type of test?

At my institution, I don’t think that there are any programs (degrees or certificates) that require basic arithmetic. We used to have several … back in 1980! Technology in the workplace has shifted the quantitative needs, while our curriculum and placement have tended to remain fixated on an obsolete view.

Join Dev Math Revival on Facebook:

placement | Jack Rotman | February 16, 2017 9:25 am

5 Comments

By schremmer, February 16, 2017 @ 3:08 pm

Re. “I don’t think that there are any programs (degrees or certificates) that require basic arithmetic. We used to have several … back in 1980! Technology in the workplace has shifted the quantitative needs, while our curriculum and placement have tended to remain fixated on an obsolete view.”

Indeed. But that is because we were told to teach arithmetic as a how to course to “enable shoppers to compute unit prices” etc.

On the other hand, here again, arithmetic is an excellent place where to take things apart to see what is the logic that drives them. In other words, to get students to ask and discuss “Why?”
By Susan Jones, February 19, 2017 @ 2:21 pm

Could there be a better assessment? In my experience, many of the people who struggle with arithmetic really aren’t ready for Math Literacy.
By Jack Rotman, February 20, 2017 @ 9:11 am

Of course, there could be a better assessment. No instrument is perfect. Some people are using “Aleks PPL”, and there is also a “Next Generation” Accuplacer set of tests.
Your comment is fascinating, and I might just write a post based on that thought.
By schremmer, February 20, 2017 @ 1:29 pm

@ Susan Jones What do you mean by “struggle with arithmetic”? Struggle with being able to trust one’s memory to perform multiplication of signed numbers? Struggle with understanding why, for instance, – times – should be +?
By Jack Rotman, February 20, 2017 @ 6:12 pm

Perhaps she’ll see it, and answer.

Developmental Mathematics Revival!

Bringing a new vitality to college mathematics 2011 to 2019