Evidence Based Decisions

How do we prevent ‘evidence based decisions’ from becoming ‘evidence constrained decisions’?

First, let’s get clear on what ‘evidence based decision making’ is about.  Primarily, the idea is to apply evidence from the scientific method to decision making.  This is the definition given at http://en.wikipedia.org/wiki/Evidence-based_medicine ; much of the current push to use this method in education comes from improvements in medicine due to using scientific evidence as a basic methodology.

The idea is to base decisions on the evidence, when appropriate evidence is available.  Remember that we are talking about scientific evidence — which is a stronger standard than ‘data’.  The scientific evidence provides a connection between a practice or treatment with the outcomes (usually stated as a probability or odds).  Sounds good, doesn’t it?

Well, in education, there are difficulties in getting scientific evidence.  We have tons of data, which are raw measurements organized in some manner; however, this has little to do with scientific evidence.  Most commonly, we have either before and after data relative to some change; sometimes, we have data from two groups under different treatments … data on the outcomes, without data on other variables that we suspect have an impact on the outcomes. 

Scientific evidence does not come from one set of data.  After one set of data suggests, scientifically, that we have reason to believe that this treatment results in a change in the outcomes, this hypotheses gets tested by replication — done by different practitioners.  The idea of scientific evidence is that we achieve something close to an empirical proof that we have a cause and effect relationship — not just a one-time correlation.

I can not resist bringing up one of my favorite oxymorons — “data based decision making”.  Data is simply organized measurements; no decisions can be made based on data, because data is not evidence of anything.  I use brand X gasoline one week, and the next week I use brand Y — and get 10% better mileage … which means nothing; this data just means that I get slightly different outcomes, nothing else.  I normally find the phrase ‘data based decisions’ to be used as a cover for a hidden agenda.

Back to evidence based decisions … as mathematicians, we are all scientists; we understand the power of research — and it’s limitations.  The presence of evidence (in the scientific sense) suggests better courses of action (decisions) to the extent that the probable outcomes are ‘likely’.  The presence of evidence does not determine the best decision … wise people still need to evaluate the current situation and apply their understanding of the evidence.

What do we do when there is no scientific evidence relevent to our decision?  Are we constrained by the evidence available?  Even in medicine, with its superior collection of evidence, decisions are not constrained by evidence.  We should be guided by the evidence we have, and use our wisdom combined with our understanding of the outcomes desired to determine the best available decision.

Relative to mathematics education in colleges, I would present these observations:  We have large bodies of evidence about learning which can (and are) being applied to our courses.  We often mistake data for being evidence, and mistake reporting data for research, and this has led to some dramatic failures (and some less dramatic).   When we do remember the distinction between data and research, we tend to skip the step of ‘replication’ before announcing a conclusion; this has led to cynical colleagues and a skeptical public.

If we do not understand what the word ‘evidence’ means, who will?  Certainly not external forces such as politicians.  We need to be much better at articulating what we are basing a decision on, and clearer at describing results.  We need to focus on our shared values, and use them to describe the desired outcomes.  We need to focus on our wisdom, to provide guidance in the absence of evidence.

Join Dev Math Revival on Facebook:

No Comments

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a comment

You must be logged in to post a comment.