The Danger of Assessing Research by Economic Impact

• Campaign for the Public University http://publicuniversity.org.uk/
• Campaign for vote of no confidence in the policies of the Minister for Universities and Science http://www.noconfidence.org.uk/
• REF postponed because Willets is ``yet to be convinced by the efficacy of the impact measure''. Read about it in THES here
• The Campaign for Science and Engineering (CaSE) wrote to the leaders of the main political parties asking them to set out their party's policies on science and engineering. On 20 April, the published letters from David Cameron and Nick Clegg. You can read them here
• Parliament Science and Technology Committee enquiry into the impact of spending cuts on science and scientific research The report is now available from the link above (this is the 6th report, released on 23 March 2010). Note Item 5 of the conclusions ``We do not believe that the consideration of pathways to potential impacts should be used as a tie-breaker in grant applications.''
• Lasers would never have shone if Mandelson had been in charge John Naughton, The Observer, 17 Jan 2010
• Tory government would postpone REF over fears over impact agenda THE, 14 Jan 2010
• Nobel laureates: don't put money before science The Independent, 7 Jan 2010.
• Penalties for 'non-productive research' may prompt academic brain drain, says survey The Times, 7 Jan 2010.
• Warning over academic 'brain drain' The Telegraph, 7 Jan 2010.

• CaSE Science Policy Debate. 13th January in London. Participants: the Science Minister Lord Drayson, Adam Afriyie MP (Conservative Shadow Minister for Science and Innovation) and Dr Evan Harris MP (Lib Dem Science Spokesman)
• Blue skies ahead? The prospects for UK science Chaired by Professor Brian Cox, panel including Lord Drayson (the science minister). Available to view on demand from 1 December. See here for a write-up by Zoe Corbyn in THE and here for a write-up by Roger Highfeld, editor of the New Scientist. Follow the discussion on twitter by searching for #sciblue. (See Brian Cox debate the then chief scientific advisor Sir David King on Newsnight, 12 Sept 2008 here)

• HEFCE's proposal for the REF
• The Educators for Reform have written an excellent response to the REF consultation.
• UCU Stand Up for Research Campaign This is a campaign against the 25% `economic and social impact' proposal in the REF. The petition was signed by over 18,000 people. See also an article about this in the Times 23 Oct , 2009. See here for Philip Moriarty's "The Impact Factor" which appeared in UCU magazine, Nov 2009 See Sally Hunt's article about it in THE 3 Dec 2009 here
• Petition to allocate funds for academic research solely on the basis of academic excellence This petition is written by James Ladyman. It is written from the perspective of arts and humanities but it applies to all research. Please sign the petition!!! (Closing date: 16 October 2010.) See also an article about this in the New Statesman, 19 Oct 2009 and an article written by Ladyman in Oxford Magazine, December 2009.
• petition against the REF impact agenda on behalf of early career academics and postgraduate students.
• Nobelists protest `economic impact' clause Nature 22 Oct, 2009. Hear Nobel Laureate Venki Ramakrishanan's talk about solving the atomic structure of the ribosome, about winning the Nobel prize in chemistry and about basic research vs translational research. ``Fundamental research pays for itself many many times over in times over in terms of eventual discoveries and technology and if you support that foundation, the rest will follow.'' See also the THES story and subsequent debate (in the comments section).
• HEPI statement The Higher Education Policy Institute (HEPI) ``calls for a re-think'' on the `impact' requirement in the REF
• The Royal Astronomical Society's statement No to REF proposals. ``The Council of the RAS has stated many times that it is counterproductive to make funding for the best research conditional on its perceived economic and social benefits...''
• Poisonous Impact Felipe Fernandez-Armesto in THE, 3 December 2009
• Impact on humanities: Researchers must take a stand now or be judged and rewarded as salesmen article by Stefan Collini in the Times Literary Supplement, 13 Nov 2009 Collini's arguments are specifically about research in the humanities, but they also apply to scientific research. (This link has now broken. Stefan Collini's article was so nice that I will leave this here for now, hoping that they fix it.)
• Only scholarly freedom delivers real `impact' 2. The wise words of the philosophy panel. Letter to THES 5 Nov 2009 signed by members of the RAE 2008 philosophy panel.
• Discussion on this issue published in Oxford Magazine, 2009
• Some paragraphs written by Leslie Ann Goldberg and Mike Wooldridge in response to HEFCE's proposal.
• Humanities research threatened by demands for `economic impact' by Jessica Shepherd in the Guardian on 13 Oct 2009
• REF should stay out of the game by Andrew Oswald in The Independent on 26 Nov 2009
• Clive James on impact
• Goodbye to blue skies research? by Francis O'Gorman in the Guardian on 19 Dec 2009

• Modest revolt to save research from red tape Letter (and front-page story) in the Times Higher Education on 12 February 2009 by Don Braben and others. This letter addresses the issue of impact in the context of research council grant proposals. See also the discussion in David Colquhoun's blog here and on Steven Hill's blog here.
• A new study suggests certain types of funding - which provide more freedum and focus less on near-term results - lead to more innovative and influential research. MIT news, 9 Dec 2009
• These Men would've stopped Darwin George Monblot's excellent article in the Guardian, 11 May 2009.
• Only scholarly freedom delivers real impact 1 letter in THES 5 Nov 2009 - signed by 48 people including ten Nobel laureates.
• The economic-impact fallacy Philip Moriarty's comment in Physics World June 2009
• Response to the RCUK Consultation The feedback from UK universities on RCUK's consultation to introduce economic impact criteria in peer review. The feedback was very negative, but this happened anyway. Note particularly the Russell Group's opinion on the Warry report's proposal, "that an individual competent in the economic impact of research should be accommodated on each Panel": "There is no evidence to date of any rigorous way of measuring economic impact other than in the very broadest of terms and outputs. It is therefore extremely difficult to see how such Panel members could be identified or the basis upon which they would be expected to make their observations. Without such a rigorous and accepted methodology, this proposal could do more harm than good."
• Petition to promote discovery in UK science This petition, written by John Allen at QMUL, attracted 2294 signatures before it closed on 3 Oct, 2009.
• They're not unreasonable A rather disappointing statement published by Dave Delpy in THES on 26 November 2009, along with many reader comments pointing out the problems with Delpy's reasoning. The fundamental issue is simple: requiring researchers to focus on short-term socioeconomic impact leads to an emphasis on short-term incremental research, stifling the fundamental research which provides more significant, and more important "impact" in the long run. See also Mandelson fails to understand how science is done (Philip Moriarty making this point in the Independent on 23 Nov 2009)

Most (all?) important scientific discoveries were driven by a desire to determine what is true, rather than by a desire to achieve economic benefit in the short-term. Indeed, at the time of discovery, it is typically impossible to predict any impact. But these are the discoveries that have impact. There are endless examples of fundamental discoveries which have led to long term ``economic impact'', but for which this would have been impossible to see at the time, or during the years following the discovery. Here are some examples

• Discoveries that would not survive the REF (a nice list compiled by UCU)
• Can science save the world? by Nobel-Prize winner Sheldon Lee Glashow. The second part of his talk discusses several pieces of technology which directly resulted from unfettered pure research including the discovery of the x-ray and radioactivity and the understanding of electomagnetic induction. Some of his ``defenses of science'' seem a little beside the point to me, but his examples speak for themselves. Another example, suggested by John Dainton, is the discovery of calculus by Newton and Leibniz.
• Mathematics and its interfaces with science, technology and society by Ari Leptev in the Newsletter of the European Mathematical Society 2009 This gives numerous examples of useful mathematics which was originally developed as a result of the curiosity of scientists rather than being driven by applications (the applications could not have been predicted at the time). See also Peter Cameron's comments on these examples (and on this general issue).
• Harry Kroto's Story About an Almost Extinct Species - Left Field Science This is the story of the discovery of C60. ``History has shown time-and-again that the basics of the above story are very often the way that important breakthroughs occur. Indeed it is blindingly obvious that the really unexpected and unpredictable discoveries are invariably more important than those that are the result of targeted initiatives.'' Note the time-scale. His discoveries will obviously have economic impact, but they were not driven by considerations of economic impact, nor would any such impact have been evident in the short (15-year) timeframe proposed by REF.
• Wikipedia's description of the accidental discovery of liquid crystals by Reinitzer in 1888. Liquid crystal displays came much later.
• More examples of major scientific discoveries that would not survive the `economic impact' regime (despite having a huge impact!)
• Roentgen's discovery of x-rays Note the text at the end about exploitation: ``a representative of a well known German company, was sent to Roentgen to negotiate a contract for the industrial exploitation of his current and future discoveries. Mr. Levy recalled Roentgen's answer: `He declared, however, that according to the good tradition of German University professors he was of the opinion that his discovery and inventions belonged to humanity and that they should not in any way be hampered by patents, licences, contracts or be controlled by one group'.''
• Chance favors the prepared mind - from serendipity to rational drug design, Hugo Kubinyi in Journal of Receptor and Signal Transduction Research 19 (1-4):15-39 (1999)

• Against the Grain: `I didn't become a scientist to help companies profit' Philip Moriarty in the Independent, 28 February 2008. I really like this short interview because this is exactly what I believe, but Philip says it better.
• Negative Impact, Peter Coles's excellent blog post on the impact regime, 2 December 2009.
• Research and how to promote it in a university John Allen in Future Med. Chem. (2010) 2(1).
• Reclaiming academia from post-academia Philip Moriarty's commentary in Nature Nanotechnology Vol 3, February 2008.
• Inquiry into the Setting of Science and Technology Research Funding Priorities Don Braben's recent submission to the House of Lords Science and Technology Committee on research funding
• Don Braben's Blue Skies Research Web Page
• Blue skies or no sky at all Kevin Fong's comment in THES March 2009
• Donald Braben's book ``Pioneering Research: A Risk Worth Taking I just bought my copy - looks like I should have read it a long time ago!
• Nobel Prize Winner Harry Kroto's Autobiography contains some useful comments on research funding.
• Managers and scholars divided as resistance grows to impact agenda Article in THES 5 Nov 2009 giving annoying quotations from ``25 [anonymous] senior sector figures''. I included the article because of the quality of the readers' comments.
• Science Matters: It pays to fund research Mark Henderson in the Times December 3, 2009
• report of the Centre for Business Research (Cambridge) on University Industry Knowledge Exchange: Demand Pull, Supply Push and the Public Space Role of Higher Education Institutions in the UK Regions. Note their concluding remarks, that ``basic research can ultimately lead, often with a long time lag, to a range of important applications'' and that the pressure for eceonomic impact ``could undermine some of the core strengths of many universities''.
• Science: exploration and exploitation John Pethica in Nanotechnology Perceptions 4 (2008) 94-97

• 600 million cut hits British science New Scientist science news blog, 9 Dec 2009

• The Theory of Computing: A Scientific Perspective by Oded Goldreich and Avi Wigderson. This is specifically about theoretical computer science, but it provides some useful perspective on the nature of scientific progress and the dangers of confusing technology with science.