7 responses to “How do you win the research game? Hide the results you don’t like!”

  1. Peter Armstrong

    You could argue that the distorting effect of excluding lower-rated research portfolios from REF submissions is rather greater that the arithmetic effect on the percentage of 3* and 4* work. Institutions which have gamed their submissions in this manner seem also be those who have boosted their research portfolios by recruiting ‘star performers’ at inflated salaries and with nominal teaching commitments. This has only been made possible by loading the teaching onto more junior staff whose consequent difficulties in finding time to develop their research has led to their exclusion from the REF. Where this is the case, these exclusions subsidise the research profiles of their departments in a manner which is additional to the reduction in the denominator of the fractions of 3* and 4* work, and which is invisible to the REF.

    Particularly in the case of business and management studies, it could be argued that the work of non-teaching research stars should be excluded from the REF altogether. Very little research in this area is consequential in the sense that it directly influences practice (normally the vector of influence is the other way, from practice to research) and the little that does so is already allowed for in principle by measures of ‘impact’. This being the case, it could be maintained that the major influence of MOS research occurs through teaching, probably on MBA programmes. On this argument a proper measure of ‘research intensity’ would only count research active staff when they also make a full contribution to teaching programmes.

    Given the already-demonstrated willingness of certain university managements to game the system to its limits, one can’t be optimistic that this suggestion will ever be taken up. What Vice-Chancellor would regard an audit of teaching timetables as anything other than an unmitigated insult?

  2. Bill Cooke

    How would we feel about research that is deemed of:

    ‘Quality that is recognised internationally in terms of originality, significance and rigour.’

    Work like this is pretty good, right, something institutions and individuals should be proud of.

    Yet, this is the REF criterion for work which is ‘only’ 2*, and which many universities penalize staff for producing, and, cf this blog, go out of their way to deny. It’s a mad world.

  3. Robert Cluley

    There’s a difference with your analogy though. It was not possible to only submit students who got As in the REF – because you don’t know who got As.

    The current rumour is that only 30% of papers in 4* ABS List journals were ranked at 4* by the REF panel for Business and Management and a higher proportion of ABS List 3* papers were eventually ranked at 4*. If that’s roughly true, if you submitted only 4* researchers using the ABS List as your metric, chances are you’d be worse off!

    It is also important to emphasise that the REF is not a ranking exercise. It’s meant to be a measurement to distribute funds. Those funds are effected by the amount of research submitted. So, again, gaming the system could be counter productive.

    I’d argue that rather than employ expensive research leaders who won’t/can’t teach and can only offer 4×4* papers, a better strategy would have been to employ lots of innovative, energetic and cheap early career researchers, spread teaching thinly among them and aim for 3* outlets across the board. This would provide individuals time to create an excellence ‘environment’ and develop routes to ‘impact’.

  4. Martin Parker

    The majority of academic papers are chip paper tomorrow. Whether 4* or any other star. History is the only judge really. But, if we are having to measure, then at least let’s measure the same things. The exercise should be predicated on the assumption that all staff of certain HESA categories are included. Anything else compounds the stupidity of measurement with a stupid measurement.

  5. The Tyranny of Excellence is Hurting Research

    […] recent article by two professors at the University of Leicester drew attention to game-playing by business schools […]

  6. The Interviewer becomes the Interviewed

    […] studies published in longer books, rather than a slavish adherence to the ABS journal rankings and playing REF games, that will allow for a true representation of the contemporary […]

  7. The ABS list. | Researching Academia

Leave a Reply

Network-wide options by YD - Freelance Wordpress Developer