:::: MENU ::::

Evidence based practice

Image by Kuebi

As a science teacher, I’m a big fan of Dr Ben Goldacre. I knew from the contents page that I would love his book ‘Bad Science‘, and I did. And, I’m currently reading (and enjoying) ‘Bad Pharma‘. So, it is with great trepidation that I dare to write a post which doesn’t entirely agree with his position on evidence based practice in education.

Firstly though, I’m so glad that the debate has been raised around evidence based practice and it’s fantastic to see this already transform into action in the form of the planned researchED conference. One of the biggest learning outcomes from completing my MEd has been an increased appreciation for the role of research in the teaching and learning process and a much stronger need to have a clearer ‘why’ underpinning my practice. I also agree with Dr Goldacre’s point around the need for teachers to engage in the research process. I also made the point in my first MEd essay that we as teachers operate very differently to doctors in terms of the control we have over our own practice, and the enquiries I’ve undertaken since have demonstrated to me the potential teacher research has in addressing this. I also of course agree with the potential of the web to bring researching teachers together and see the potential overlap with the Pedagoo idea.

So what don’t I agree with? It’s not that I disagree as such, I’m more concerned about the emphasis. In his paper Dr Goldacre rightly states that we should use the right research method for the right question, and even acknowledges the important role of qualitative research in education, but the majority of the piece focuses on the need for more quantitative randomised trials. I find this sort of evidence useful as a teacher, and I think Geoff Petty‘s work has been particularly successful at taking this sort of evidence and making it accessible to busy teachers. Whilst I wouldn’t argue against the need for more of these sorts of studies to gather more of this form of evidence, I don’t particularly think that should be our primary focus in this evidence based revolution.

Firstly, Dr Goldacre rightly points out that qualitative research can tell you why and how something works but then quickly moves back to the importance of quantitative trials. Surely as teachers the why and how are crucial. Do we want a profession that carries out quantitative studies that generates statistics which “proves” a technique? Or do we want a profession which can carry out much richer enquiries into their practice which provides them with a wider range of evidence which then allows them to tailor their approaches to different learners in different contexts? Yes, it won’t make for easy extrapolation, but it should make for much richer educational experiences for both the teachers and their students.

But also, education is not medicine. Placing the emphasis on quantitative research makes a number of assumptions about what it is we’re trying to achieve and how we’re trying to achieve it. Of course we want to use good strategies which improve attainment, which is what this sort of evidence is ideal for, but is that all we want as the outcome of our educational process? What about confidence? Happiness? Ability to manage relationships? Independence? Resilience? Do these educational outcomes matter too? I know these qualities will be at least as important in my own children’s lives as exam results. In fact, aren’t these qualities often crucial in order to achieve those attainment results? If we have happy, confident, independent and resilient students wouldn’t we expect those sorts of young people to cope rather well with exams? So, how do we measure these outcomes in randomised trials? My suggestion is that we don’t. We use qualitative research methods and we give this evidence the status it deserves. After all, education is a social enterprise, social science methodologies make perfect sense in this field and shouldn’t be down played or written off as ‘soft’ as they so often are.

Basically, I think we’re at risk of getting ahead of ourselves a little. We can all agree that we want more of an evidence basis to our practice and that teachers and learners would benefit from participating in the research process to a greater extent, but we must first agree what it is we want from our educational system and ensure that the research methodologies we use, and therefore the evidence gathered, are valid for those outcomes.


  • Reply behrfacts |

    Good that you’ve spoken out on this as clearly teachers need to express a view. I also have some concerns about RCTs in education (given that we are experimenting on young people’s future careers) and yesterday did some Googling for my own info. This led me to research into medical research methodology which highlighted many issues with the way RCTs have been run in the past and the biases that they may have within them – they are not a panacea for all problems. Saying that I agree that there needs to be more rigour in education research, and this can mean for example tackling the maths behind multi-variate analysis etc. But like you I am a fan of case studies as well. Most people end up using mixed methods in their research. When I do desk analysis of policy evidence I will tend to triangulate sources which implies a similar approach.

    • Reply fearghal |

      Thank you so much Nick. I was actually quite nervous about publishing that post and it’s great to know that I’m not alone.

  • Reply Stephen Day |

    Hi Fearghal, Great post. I think that while RCTs are fine, in order to really understand a phenomena we also need to be aware of the cultural variations that act as confounding variables in these types of trials. I like using mixed methods in my research for that reason. But I worry about the application of a positivist research paradigm to what is essentially a complex adaptive system that is culturally bound and as such, is extremely difficult to reduce and control under lab type conditions. In my humble opinion, education is not capable of being reduced in this way. I value RCT studies and large scale quantitative studies for highlighting a potential issue but I also value the qualitative studies which, while small scale maintain the complexity and richness of the classroom dynamic. As you say it’s a case of using the right tool to answer the research question posed. However, this type of discussion is just what is needed to invigorate educational research that is more useful to classroom practice. I know of too many educational researchers who undervalue classroom practice research which is a shame and in my view short sighted.

    • Reply fearghal |

      Thanks Stephen,

      You’ve said it so well! I especially like “I worry about the application of a positivist research paradigm to what is essentially a complex adaptive system that is culturally bound“.

      Essentially, more of an evidence basis would be good but we shouldn’t be predetermining the methodologies before we’ve considered the questions we want too ask. If RCTs are the only approach we’re interested in then it will restrict us to only asking a very narrow form of question.

      I quite often find that the stuff that really matters in education is hard to measure and is therefore ignored due to the positivist mindset which seems to pervade the profession. When I’ve presented some of my own more interpretivist research into outcomes such as engagement to colleagues I’ve often been asked “ok, but did the results go up?”

      • Reply Stephen Day |

        Hi Feargal, I feel your pain. I have had a lot of difficulty in the past from fellow science teachers who struggle to see the value in interpretive work perhaps as a product of their scientific view of research. I think that It is important that educational researchers engage with more teachers perhaps by growing networks of teachers willing to critically engage with practice-based educational research. This would allow for the complexity of the reality of teaching practice to be accounted for and also build up the evidence base required to convince policy makers and practitioners that the new pedagogy emergent from CfE is effective and worthwhile persivering with. It is difficult but anything worthwhile and new is difficult.

        I believe that there is a critical mass of teachers out there willing to engage in such work but that they are perhaps unable to tap into a network that can help them. I think educational researchers need to work more on this type of engagement. I would place myself in this category as well. I feel that I should be doing more but get side tracked by other things that the University see as priority. That’s no excuse, at least I am willing to try. I know that I am perhaps preaching to the converted.

So, what do you think ?