Prerequisites

Prerequisites are placed on courses for various reasons, from convenience to supporting student success.  Few prerequisites are placed on courses based on validation studies … some prerequisites are used based on professional validity, while others have even less of a scientific basis.

Let’s say that you are working for the department of education in a state such as mine (Michigan), and you have been getting more concerned about the possibility that academic standards are not consistent across regions or levels of education.  You look at mathematics, and notice that college courses use a different organizing system than high schools … and you do not want students to get credit for a college math course that is really a high-school level course.  An easy, and somewhat logical, approach is to enact a rule (or law) that says any college level math course needs to have at least an intermediate algebra prerequisite.

What’s so bad about that arrangement?  If intermediate algebra is at the level of 1oth to 11th grade high school, this seems like a pretty low standard for ‘college’; when challenged, you might add that the really logical rule would be that a college level math course needs to have at least a college-algebra prerequisite … and you’d like to do this, if WE would just make up our minds about what ‘college algebra’ really is.

However, the problems with this approach are too basic to be resolved by this framework.  First, it assumes that all mathematics builds on the stuff in an intermediate algebra course; several basic areas, most notably statistics, do not have any relationship to the concepts and skills of intermediate algebra.  By requiring intermediate algebra as the minimal prerequisite, we mislead students and cause them to take unneccessary courses … both problems are non-trivial.

Second, this approach assumes that a subtle concept like ‘rigor’ can be measured by the prerequisite.  This is not one of the valid uses of prerequisites; rigor is measured by properties of the course in question (the content, concepts, assessment and practices) … which do not necessarily change just because we list “IA” (intermediate algebra) as a prerequisite.  If we want sufficient rigor in college level mathematics classes (and I hope we do), we need to measure those courses — not a prerequisite to those courses.

Third, prerequisites tend to disproportionately affect underrepresented groups.  At my institution, it is not unusual to have 30% of a pre-algebra class be minority; the courses which immediately follow intermediate algebra are often 90% majority.  Sadly, our curriculum is still not a pump … more filtering happens, so any unvalidated prerequisite can lead to wasteful reductions in minority completion.

I’m pleased that my own institution has 5 college-credit math classes that do not have an intermediate algebra prerequisite.  Two of these 5 courses transfer to several institutions in the state.  However, students are still advised to take intermediate algebra “just in case” … they don’t really need it, but they might change their mind later.

If this topic is of interest to you, you will want to follow a position statement being worked on in the Developmental Mathematics Committee (DMC) of AMATYC.  The DMC web site is http://groups.google.com/group/amatyc-dmc?hl=en   The motivation for this position statement is to help institutions and states use appropriate prerequisites, based on validation — not prerequisites to enforce an abstract policy on ‘rigor’.

In the long term, we will replace “IA” with a more reasonable course like the New Life Transitions course — at least for the majority of students who do not need a pre-calculus type course later.  We will also replace beginning algebra with something like Mathematical Literacy for College Students, and this kind of course could really serve all students.  Until this change happens, we can work on better prerequisites relative to IA … and for all courses (including Transitions).

 
Join Dev Math Revival on Facebook:

Placement Tests – Valid?

Almost all community colleges use a placement test to identify students who need a developmental course.  Are these tests sufficiently valid for this high-stakes usage?

A recent publication from the Community College Research Center (CCRC) at Columbia College reports on a research study to examine this validity; the report is”Predicting Success in College: The Importance of Placement Tests and High School Transcripts” (CCRC Working Paper No. 42) and is available at  http://ccrc.tc.columbia.edu/DefaultFiles/SendFileToPublic.asp?ft=pdf&FilePath=c:\Websites\ccrc_tc_columbia_edu_documents\332_1030.pdf&fid=332_1030&aid=47&RID=1030&pf=Publication.asp?UID=1030

I’ve spent a little time looking through this study.  One data bit is creating quite a bit of interest … a statement that the two major tests (Compass, Accuplacer) have ‘severe error rates’ of 15% to 28%.  By severe error, they mean either of these situations:  (1) The placement test directs a student to a developmental course when the prediction is actually that they would pass the college level course or (2) The placement test directs a student to the college level course when the prediction is that they would fail.

The methodology in the study begins with the assumption that the placement test score measures degrees of readiness, not just a ‘yes’ or ‘no’ (binary) result.  Using data from a state-wide community college system, the authors correlate the placement test scores with whether students actually passed the math course (either a developmental course or college level) to create a probability value.  Since the colleges involved did not generally allow students with scores below a cutoff to take the college level course, they extrapolated to estimate the probability below the cutoff; a similar approach was done for a probability of passing the developmental course for scores above the cutoff.  For each placement test, the study includes between 300 and 800 students.

Using these models for the probabilities, the authors then calculate the severe error rate cited above.  The values shown were for mathematics — the ‘english’ rates for severe errors were slightly higher (27% to 33%).

Separate from that severe error rate, the study showed the ‘accuracy rate’ for each test for the courses; these accuracy rates reflect the pass rates for those above the cutoff and the failure rate for those below the cutoff.  These values range from 50% to 60% in math (for receiving a C grade or better).

The research also examined the relationship of high school performance to both this placement question and to general college success, and they conclude that the high school GPA is the single best predictor — even for predicting who needs a developmental course.

Several things occur to me relative to this study.  First of all, any measurement has a standard error; in the case of Accuplacer, this standard error varies with the score — for middle value scores (like 60 to 80), the standard error is about 10.  If a student scores 69 when the cutoff is 76, there is some chance that the score is ‘on the wrong side’ of the cutoff just due to the standard error of the measure.  In my experience, this standard error results in something like 10% of what the authors call ‘severe error’.  The main methodology to minimize this source of error is to have repeated measures — like having students take the placement test twice. 

Another thought … the report does not identify the math courses involved, nor the cutoff used.  Most results are given for “math 1” and “math 2”; the predictability of readiness is not uniformly distributed, and is more difficult when there are different levels of expectation (reasoning, abstraction, problem solving, etc) in two levels of courses.   Since the report does not identify which type of severe error is contributing the most to the rate, it is possible that the cutoff itself is what contributes to the severe error rate that is beyond the standard error

Though I doubt if many of us would, the use of high school GPA as a placement measure seems both awkward and risky.  We would need to replicate this study in other settings — other states and regions — to see if the same pattern exists.   Even if that result is validated, the use of a composite measure of prior learning raises issues of equity and fairness; applying this to individual students may produce varying results by student characteristics (even more than the placement tests).

The other thought is that a hidden benefit in this report is a comparison of the two primary tests (Accuplacer, Compass) for various measures of validity.  For example, the Accuplacer accuracy rates in math were somewhat higher than those for Compass.

Overall, I do not see this study raising basic questions about our use of placement tests. 

 
Join Dev Math Revival on Facebook:

Friendly Placement Testing

Like most community colleges, my college requires students to take a standard math placement test to determine their math level.  Like many, my college enforces the resulting score — students can not enroll for any course higher than what their placement test score qualifies them for.  How can this standard approach be done so that is fair to students and allows them to begin at the highest reasonable level of mathematics (for their knowledge)?

Let’s agree right at the start … the content of the standard placement tests is not aligned with the best mathematics in those areas of knowledge; the items tend to be basic procedures and basic concepts in a fairly narrow range of topics.  However, changing a placement test is a long term (and commercial) process.  As much as I would like to see (drastic) changes in the tests, that is not under our control and any changes to those tests will not be seen by students for a while (like 2 or 3 years).

Here are some observations about typical math placement testing systems that affect how friendly it is to students:

• Upon admission, community college students usually do not know what will be on the test.
• Community college students often do not understand how important the results will be.
• Our students usually take a math test without any review.
• Options for re-testing (challenging) are often limited, and we tend to not provide information on ‘what to do before retesting’.

You might not agree with all of these observations.  I hope you see enough truth in them to agree with this statement: “The advising for students prior to taking a math placement test is not currently adequate in most community colleges”.  In fact, many colleges are like mine … the first advising a new student receives is done at an orientation; students are required to complete placement testing BEFORE orientation (and advising).  There is a logical reason for this — advising tends to deal with specific questions about enrollment, and this means the results of testing are needed.  However, I would suggest that this approach is not student friendly.

If I could do so, I would make the initial advising a two-step process:

1. An orientation & advising (done in groups) which would cover information on math placement testing, followed by taking the placement test (different day)
2. Individual advising after placement testing, where possible re-testing is discussed (based on how the student sees their initial results aligns with their background).

Alas, I am not in charge of advising … as I suspect math faculty would not be in charge of advising in general.

In the meantime, here is one specific thing we could consider doing to make the process a little more friendly for our students: Make use of an online homework system for the review prior to retesting. 

At my college, some students who want to re-test for math are referred to the math department where they speak with an administrator (usually).  For many students, this results in them receiving access to a “MyMathTest” program that provides specific preparation for the placement test they want to retake.  We are able to do this because of the cooperation of the publisher, so there is no cost for the student; we are somewhat limited to a total number of users for this program, but the limit is high enough to accommodate the students talking to us about retesting.

We do not have specific results to share about how this is working (gains in math level or not).  I hope there are gains there.  However, I think this is a good thing to do just because it is student friendly.

If you are interested in using an online homework system as part of the review process for a placement test, start by talking to your book company representatives.  We have found the representatives to be helpful, and willing to look at doing something extra that would help students.

 
Join Dev Math Revival on Facebook: