{"id":500,"date":"2017-06-12T09:02:54","date_gmt":"2017-06-12T09:02:54","guid":{"rendered":"https:\/\/staffblogs.le.ac.uk\/lli\/?p=500"},"modified":"2025-02-26T13:27:44","modified_gmt":"2025-02-26T13:27:44","slug":"500","status":"publish","type":"post","link":"https:\/\/staffblogs.le.ac.uk\/lli\/2017\/06\/12\/500\/","title":{"rendered":"Educational policy and practice: how \u2018evidence-based\u2019 can or should it be?"},"content":{"rendered":"

\u2018On the research side, evidence-based education seems to favour a technocratic model in which it is assumed that the only relevant research questions are questions about the effectiveness of educational means and techniques, forgetting, among other things, that what counts as \u2018\u2018effective\u2019\u2019 crucially depends on judgements about what is educationally desirable. On the practice side, evidence-based education seems to limit severely the opportunities for educational practitioners to make such judgements in a way that is sensitive to and relevant for their own contextualized settings. The focus on \u2018\u2018what works\u2019\u2019 makes it difficult if not impossible to ask the questions of what it should work for and who should have a say in determining the latter.\u2019<\/p>\n

Biesta (2007)<\/p>\n

\u2018In short, social interventions are complex systems thrust amidst complex systems.\u2019<\/p>\n

Pawson et al<\/em>. (2004)<\/p>\n

 <\/p>\n

The Learning Development team, and the LLI as a whole, have been discussing how we might devise meaningful and workable approaches to evaluating the work we do. As well as considering practical considerations, models, frameworks etc. this process also inevitably invites critical reflection on the nature of what we\u2019re studying (student learning within complex social and institutional contexts), the possibilities for knowledge about this, and what, if anything, should be done in light of our knowledge. This, in turn, has prompted me to return to some of the literature I\u2019ve encountered in recent years (thanks, in large part,\u00a0to completing the University of Leicester’s\u00a0excellent MA International Education<\/a>) concerning the limitations and potential harms of taking too mechanistic an approach to the relationships between research, evidence, policy and practice.<\/p>\n

 <\/p>\n

Some problems with \u2018evidence-based\u2019 policy and practice<\/strong><\/p>\n

The belief that the knowledge we claim and the decisions that we make should be grounded in legitimate and credible evidence carries an understandable, and justified, power in HE. It explains, for example, why such methodologically flaky instruments as the NSS are met with so much ire and frustration. However, as numerous authors have pointed out (for a sample, see below), the common insistence that educational and other social policies and practices should be \u2018evidence-based<\/em>\u2019, or more specifically that evidential bases should be geared towards telling us \u2018what works\u2019 in order that we may be \u2018led by the evidence\u2019 in deciding what to do, is frequently far less straightforward, and far less benign, that it might at first appear. For a number of important reasons, we should be more cautious and critical when deciding how much credence we give, and practical utility we afford, to so-called \u2018what works\u2019 approaches to developing policy and practice in light of research evidence. Some of the work listed below targets specifically the claims, assumptions and agendas of those advocating \u2018evidence-based policy and practice\u2019 and the \u2018what works\u2019 agenda, whilst some is more generally critical of certain research paradigms and assumptions. The varied critiques deal with the following broad themes:<\/p>\n

    \n
  1. The na\u00efve ontological and epistemological assumptions often underpinning much \u2018what works\u2019-oriented empirical research, and the relationships it\u2019s believed this research can and should have to matters of policy and practice.<\/li>\n
  2. The similarly na\u00efve and empiricist theories of causation the \u2018what works\u2019 movement seem often to base their arguments on.<\/li>\n
  3. The limitations of (necessarily and unavoidably) complexity-reducing and historical research when it comes to informing the decisions and future actions of practitioners and policy-makers operating in ineluctably complex, unpredictable, open and values-laden social settings.<\/li>\n
  4. The apparent belief (sometimes tacit, sometimes more explicitly stated) that questions of \u2018what works\u2019 can somehow be disaggregated and therefore considered in isolation from questions of ethics, values and politics \u2013 what we might call those questions concerning \u2018what matters<\/em>\u2019 in education (Smyers and Smith, 2014).<\/li>\n<\/ol>\n

     <\/p>\n

    Importantly, these critiques reach well beyond technical discussions about ‘sampling procedures’, ‘effect sizes’ etc. (important though these are, too) to pose more fundamental questions concerning the ontological, epistemological and axiological assumptions that inform so much social research. I take from this and similar scholarship the important reminder that we need to work with<\/em> the complexity and dynamism of social research settings and to understand that questions of policy and practice are always moral and political ones, framed and responded to in arenas of struggle, characterised by numerous multi-layered and intersecting power relations. Learning and teaching do not take place in neutral, ahistorical, values-free spaces, in which we can simply implement what the evidence \u2018instructs\u2019 us are the most technically efficacious interventions. It would be self-deluded and dangerous to pretend otherwise.<\/p>\n

     <\/p>\n

    So, where does this leave us?<\/strong><\/p>\n

    None of this is to suggest that scholarly enquiry (of many varieties, including, but not limited to, the kinds of empirical studies usually favoured by proponents of evidence-based policy and practice) shouldn\u2019t play a central role in helping us make context-aware, ethically-informed, judgements<\/em> about what can and should be done. Nor is it to argue that developing educational policies and practices should henceforth become an exercise in evidence-free guess work! Quite the reverse: it\u2019s precisely because evidence matters so much that the complexities of its production and interpretation need to be acknowledged and taken seriously, in order that we might reflect more critically on the research we encounter. Work such as that cited below helps empower us to ground our critical reflections in: a) more philosophically literate understandings of the assumptions that underpin different types of research; b) a recognition of the necessary limitations of all complexity-reducing research when it comes to informing practical, value-laden judgements about what we can and should actually<\/em> do<\/em> in complex and unpredictable social settings like schools and universities; and c) an alertness to the ethical and political dangers inherent in pretending we can suspend normativity and develop policies and practices from the purely instrumental, and mythically neutral, perspective of \u2018just tell me what works<\/em>\u2019. Engaging in this fashion should \u2013 far from marginalising the role of scholarship in helping us to make our (always fallible, always provisional) decisions \u2013 actually help to re-affirm scholarship\u2019s true value as a resource to help stimulate and enable critical praxis. Indeed, I would argue that such approaches \u2013 rooted as they are in better understandings of what evidence actually is (and isn\u2019t), how and why it\u2019s produced, and what it can and can\u2019t tell us about what we should do \u2013 are more authentically evidence-engaged<\/em> than those often favoured by the more reductionist voices within the \u2018what works\u2019 movement.<\/p>\n

     <\/p>\n

     <\/p>\n

    References and suggested further readings:<\/strong><\/p>\n

    Biesta, G.J. (2007) Why \u201cwhat works\u201d won\u2019t work: Evidence\u2010based practice and the democratic deficit in educational research.\u00a0Educational theory<\/em>,\u00a057(1), 1-22.<\/p>\n

    Biesta, G.J. (2010) Why \u2018what works\u2019 still won\u2019t work: From evidence-based education to value-based education.\u00a0Studies in Philosophy and Education<\/em>,\u00a029(5), 491-503.<\/p>\n

    Clegg, S. (2005) Evidence\u2010based practice in educational research: a critical realist critique of systematic review, British Journal of Sociology of Education<\/em>, 26(3), 415-42.<\/p>\n

    Clegg, S., Stevenson, J. and Burke, P.J. (2016) Translating close-up research into action: a critical reflection. Reflective Practice<\/i>, 17(3), 233-244.<\/p>\n

    Bronwyn. D. (2003) Death to Critique and Dissent? The Policies and Practices of New Managerialism and of ‘Evidence-based Practice’, Gender and Education<\/em>, 15(1), 91-103.<\/p>\n

    Parkhurst, J. (2017) The Politics of Evidence: From evidence-based policy to the good governance of evidence<\/em>. Abingdon, UK: Routledge. OPEN ACCESS VERSION AVAILABLE<\/a><\/p>\n

    Hammersley, M. (2005) The Myth of Research\u2010based Practice: The Critical Case of Educational Inquiry, International Journal of Social Research Methodology<\/em>, 8(4), 317-330.<\/p>\n

    Pawson, R., Greenhalgh, T., Harvey, G. & Walshe, K. (2004) Realist synthesis: an introduction.\u00a0Manchester: ESRC Research Methods Programme, University of Manchester<\/em>.<\/p>\n

    Smyers, P. (2008) On the Epistemological Basis of Large\u2010Scale Population Studies and their Educational Use.\u00a0Journal of Philosophy of Education<\/em>,\u00a042(1), 63-86.<\/p>\n

    Smyers, P. and Smith, R. (2014) Understanding Education and Educational Research<\/em>. Cambridge: Cambridge University Press.<\/p>\n","protected":false},"excerpt":{"rendered":"

    \u2018On the research side, evidence-based education seems to favour a technocratic model in which it is assumed that the only relevant research questions are questions about the effectiveness of educational means and techniques, forgetting, among other things, that what counts as \u2018\u2018effective\u2019\u2019 crucially depends on judgements about what is educationally desirable. On the practice side, […]<\/p>\n","protected":false},"author":261,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,1],"tags":[69,71,70,68],"class_list":["post-500","post","type-post","status-publish","format-standard","hentry","category-scholarly-inspiration","category-uncategorized","tag-evidence","tag-practice","tag-research","tag-student-learning"],"_links":{"self":[{"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/posts\/500","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/users\/261"}],"replies":[{"embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/comments?post=500"}],"version-history":[{"count":13,"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/posts\/500\/revisions"}],"predecessor-version":[{"id":1308,"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/posts\/500\/revisions\/1308"}],"wp:attachment":[{"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/media?parent=500"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/categories?post=500"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/staffblogs.le.ac.uk\/lli\/wp-json\/wp\/v2\/tags?post=500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}