Where Dreams go to Thrive … Part III (More Evidence)

The leading cause of bad policy decisions is the phrase “Research clearly shows … ” which suggests that all of us should accept one interpretation of some unnamed set of ‘research’ (most of which is not research at all).  Understanding the needs of students not prepared for college mathematics is a long-term process, involving prolonged conversations among professionals as we attempt to understand what the data and the research say about our work and our students.

My goal is to present another scientific research article on the impacts of developmental education — remedial mathematics in particular.  This article is by Bettinger & Long called “Addressing The Needs Of Under-Prepared Students In Higher Education: Does College Remediation Work?” which you can download at http://www.nber.org/papers/w11325.pdf

This research is based on a large sample of students in Ohio.  The strategy is to adjust for selection bias that is so strong in all studies on remediation — Students referred to remediation tend to have both lower specific skills (math) and more academic challenges.  The authors define a series of variables for this purpose, and eventually calculate a ‘local area treatment effect’ (LATE) which is partially based on the fact that cutoffs for remediation vary significantly among the 45 institutions of higher education in the data.  The analysis of “LATE” involved a restriction on the sample — towards the middle, where the cutoffs have more impact; this analysis excludes the weakest (roughly 10%) of the overall sample.

Key Finding #1: Equal Outcomes for those in Remediation
For outcomes such as dropping out and degree completion (bachelor’s) students who had remediation achieved similar outcomes to those who did not, once the selection bias was accounted for.

Key Finding #2: For those most impacted by remediation cutoffs, outcomes are improved
The “LATE” analysis showed that remedial students had a lower rate of dropping out and a higher rate of degree completion compared to similar students without remediation.  The authors attribute this as an accurate (perhaps even conservative) estimate of the benefits of remediation.

Here is a nice quote from their summary:

We estimate that students in remediation have better educational outcomes in comparison to students with similar backgrounds and preparation who were not required to take the courses.  [pg 19]

The research also explored the impact of remediation on student interest (as measured by type of major); you might find that discussion interesting, though it is not directly related to the question of ‘thrive’ in remedial math.  I say that because the initial major data was taken from the survey attached to the ACT exam — usually completed long before a student examines their actual choices at the college they enroll at.  The authors do find an interaction between remediation and changing type of majors (specifically, changing out of math-related majors).

This study, as the others I’ve listed lately, provide a different picture of developmental mathematics than we hear in the loud conversations by policy makers (Complete College America, for example) and proponents of ‘co-requisite remediation’.  Those external forces almost always refer to ‘research’ that is simple (few variables) and aggregated; they have not dealt with the selection bias problem at all.  If you read the pronouncements carefully, you’ll notice that the biggest evidence of our failure in remedial mathematics is the large group of students who never attempt their remedial math course(s); this ‘damning conclusion’ is presented without any evidence that the nature of the remedial math courses had any causative connection to that lack of attempt.

As professionals, it is our job to both learn about the valid research on our work (the good and not-so-good) and to inform others about what this research says.

Evidence exists which truly does indicate that remedial mathematics is where dreams go to thrive.

 Join Dev Math Revival on Facebook:

Dev Math: Where Dreams go to Thrive … Part II (Evidence)

Developmental mathematics is where dreams go to thrive; we have evidence that even the traditional courses help students succeed in college.  The narrative suggested by external political forces is often based on a simplistic view of students which is out of touch with reality.  Let’s help by spreading the word on a more complete understanding.

Students who need to take developmental math courses have a wide range of remediation needs.  Peter Bahr’s study on pathways with single or multiple domains of deficiency (https://www.devmathrevival.net/?p=2458) concluded that the basic college outcomes (such as earning a degree) show equivalent outcomes for groups of students (needed remediation versus not).

A totally different analysis by Attewell et al 2006 (see http://knowledgecenter.completionbydesign.org/sites/default/files/16%20Attewell%20JHE%20final%202006.pdf) also reaches a conclusion of equal results between groups in many ways.  Many studies of remediation are simple summaries of enrollment and grades over a short period of time.  The Attewell research was based on a longitudinal study begun on 8th graders in 1988 (thus, the acronym “NELS: 88”) done by the National Center for Educational Statistics.  Over an 12 year period, the study collected high school and college information as well as additional tests and surveys on this sample.

A key methodology in this research is ‘propensity matching’ — using other variables to predict the probability of an event and then using this probability to analyze key data.  For example, high school courses and grades, along with tests, were used to calculate the probability of needing remediation in college … where a sample of students with given probabilities did not take any remediation while another sample did.  An interesting curiosity in the results is the finding that low SES and high SES students have equal enrollment rates in remedial math when ‘propensity matched’.

Thrive: Key Result #1
Students taking remedial courses have a higher rate of earning a 2-year degree than students who do not take remedial courses with similar propensity scores for needing remedial courses.  Instead of comparing students who take remediation with the entire population, this study compared students taking remediation with similar students who did not take remediation.  The results favor remediation (34% versus 31%)

In the bachelor degree setting, the results are the other direction — which the authors analyze in a variety of ways.  One factor is the very different approach to remediation in the two sectors (4-year colleges over-avoid remediation, 2-year colleges slightly over-take remediation).   However, the time-to-degree between the two groups is very similar (4.97 years with remediation, 4.76 years without).

Thrive: Key Result #2
Students taking three or more remedial courses have just slightly reduced results.  This study shows a small decline for students needing multiple remedial courses: 23.5% earn 2-year degree, versus 27.5% of similar students without multiple courses.  The Bahr study, using a local sample, produced equivalent results in this same type of analysis.

It’s worth noting that the results for multiple remedial courses are pretty good even before we use propensity matching:  25.9% complete 2-year degree with multiple remedial courses versus 33.1% without.  This clearly shows that dreams thrive in developmental mathematics, even among students with the largest need.

Thrive: Key Result #3
Students taking 2 or more remedial math courses have results almost equivalent to other students.  The predicted probabilities for students with multiple remedial math courses is 23.8%, compared to similar students without multiple remedial math (26.7%).

Note that this study was based on data from prior to the reform movements in developmental mathematics.  Even then, the results were reasonably good and indicate that the remediation was effective at leveling the playing field.

Thrive: Key Result #4
This is the best of all:  Students who complete all of their math remediation have statistically equivalent degree completion (2-year) compared to similar students (34.0% vs 34.7%)

This result negates the common myth that taking multiple remedial math courses spells doom for students.  The data shows that this is not true, that completing math remediation does what it is meant to do — help students complete their degree.

I encourage you to take a look at this research; it’s likely that you will spot something important to you.  More than that, we should all begin to present a thrive narrative about developmental mathematics — because that is what the data is showing.

 
Join Dev Math Revival on Facebook:

Dev Math: Where Dreams go to Thrive

In response to data showing the exponential attrition of long sequences of developmental mathematics courses, some people are using the quote “developmental mathematics is where dreams go to die”.  This phrase has been one of the most influential statements in our field over the past 5 years — not because it is true but because people (especially policy makers) believe that it is true.

This is a normal political strategy: frame an argument in a way that there is only one answer (the one that ‘you’ want).  I’ve seen leaders at my own college use this method, often successfully. … and I imagine that you’ve encountered it as well.  As teachers at heart, this style of communication is not natural for us; we respond by reasoned arguments and academic research with a goal of getting everybody to understand the problem.

The difficulty is that leaders who use the “where dreams go to die” phrase have little interest in understanding the problem.  Their goal is to remove developmental mathematics as a barrier to student success.  The next phrase after “where dreams go to die” is often “co-requisite remediation”, with claims that this solution is a proven success because of all of the data.  Of course, our view of this data is a bit more restrained than the leaders and policy makers; this is not a problem for them, as they have the answer in mind — all we have to do is agree with it.

We must do two basic things so that we can really help our students succeed:

1. Shorten and modernize our mathematics curriculum, both developmental and college level.
2. Consistently use our narrative:  “Developmental mathematics is where dreams go to thrive!”

Much of the material on this blog, as well as the wiki (dm-live.wikispaces.com)is meant to help faculty with the first goal.  The new courses, Mathematical Literacy and Algebraic Literacy, allow us to provide great preparation for college level courses within an efficient structure which minimizes exponential attrition.

“Developmental mathematics is where dreams go to thrive”:  We need to articulate this accurate view of our work, which is valid even within the old-fashioned traditional curriculum with too many courses.  I’ve posted about some of the research with a ‘thrive’ conclusion:

• The Case for Remediation  (https://www.devmathrevival.net/?p=2461)
• Research Trends in Developmental Mathematics  (https://www.devmathrevival.net/?p=1176)

Also, a great project at CUNY called “ASAP” gets a glowing external evaluation:  http://www.mdrc.org/project/evaluation-accelerated-study-associate-programs-asap-developmental-education-students#overview  The ASAP model is currently being validated at other institutions.  Please let me know of other research showing that dreams thrive in developmental mathematics.

We should add our own ‘thrive’ stories and data.  For example, at my institution, we had 6 students start in pre-algebra and the proceed up to Calculus I in a four year period … 5 of them passed Calculus I on their first attempt.  If we believe the ‘die’ narrative, you would expect zero or 1 of these to exist; I am sure that most institutions have similar results to mine where the data shows more of a ‘thrive’ result.

Our traditional courses must go; we must do the exciting work of renewing the curriculum based on modern thinking about mathematics combined with more sophisticated approaches to instruction and learning.

However, that work will generally be wasted unless we establish a ‘thrive’ attitude.    The two conditions existing together create a new system that serves students well.  Developmental mathematics is where all dreams go to thrive.

Join Dev Math Revival on Facebook:

Student Success & Retention: Key Ideas

I’m working on a project which involves a search for strong research articles and summaries, and that included some work on ‘retention in STEM’.  I have some references on that, later; however, I wanted to present some key ideas about how to keep students in class so they succeed and how to retain them across semesters.

Rather than look at certain teaching methods as ‘the answer’, let’s look at some key ideas with surface validity and examine their implications for teaching.

• Students need to be working with the content over an extended period in order to be successful.

We know that learning is the result of effort, usually intentional.  Attendance is easily measured, but is not sufficient by itself.  The class needs to establish environments where students want to work with the material, and we know that grades are insufficient motivation for many students.

• Non-trivial ‘success’ (positive feedback) based on effort is strong motivation for most people.

If success seems impossible regardless of effort, it is easy to see why students would stop working.  However, success regardless of effort is also likely to result in drastic reductions in effort.  As in most human endeavors, people need to see a connection between effort and reward.

• A teacher’s attitudes are more important than specific methods.

A few years ago, I was trying some very different things in a class; in fact, I was not very proficient with some key parts of that plan.  However, my students responding to my attitude more than those methods.  As one student said, “Mr. Rotman would not give up on me!”  An honest belief that almost all students are able to succeed is strong motivation.

We need to see our classes as a human system, a community with a shared purpose.  Most people need relationships with a purpose … connections that help them deal with challenges.  I am not trying to be a friend to my students, but we do form a community which can support all members.

• Every student contributes to the success of the class.

Not all students will pass a math class.  Some of those who do not pass are able to provide help to those who do pass.  This past semester, I had a student who did very poorly on written assessments who routinely helped the class understand concepts and procedures.  The contributions of a student are valued independently of their grade, and independently of any other measure or category (ethnicity, social standing, mastery of formal language, etc).

I have not mentioned any teaching methods; pedagogy does matter … but the pedagogy follows from other ideas.  I can not use the key ideas above if all I do is ‘lecture’ (though I do a fair amount of that).  My class must provide a variety of interactions in order for my attitudes to be clear … and for all students to have opportunities to contribute.  Establishing a community is social navigation, so students need times to talk with each other in smaller groups as well as the entire class.

Here are some good articles and summaries of retention in mathematics and other STEM fields; these studies focus on retention in programs as opposed to courses … though there are obvious connections between the two.

1. Teaching For Retention In Science, Engineering, and Math Disciplines: A Guide For Faculty http://www.crlt.umich.edu/op25
2. Increasing Persistence of College Students in STEM  http://www.fgcu.edu/STEM/files/Increasing_Persistence_of_College_Students_in_STEM.pdf
3. Retaining Students in Science,Technology, Engineering, and Mathematics (STEM) Majors
   http://mazur.harvard.edu/sentFiles/Mazur_399966.pdf
4. Should We Still be Talking About Leaving? A Comparative Examination of Social Inequality in Undergraduate Patterns of Switching Majors http://wcer-web.ad.education.wisc.edu/docs/working-papers/Working_Paper_No_2014_05.pdf
5. Gender and Belonging in Undergraduate Computer Science: A Comparative Case Study of Student Experiences in Gateway Courses http://wcer-web.ad.education.wisc.edu/docs/working-papers/Working_Paper_No_2016_02.pdf

Success and retention starts with us, and depends upon both our attitudes and our professional knowledge.

 Join Dev Math Revival on Facebook: