Discussion: View Thread

Alex's posting has certainly produced a lively debate!

A couple of contributors have mentioned in passing the UK research assessment exercises that have been carried out over recent years, most recently REF2014. In the latter, entrepreneurship and small business research were assessed as part of business management. Included on the panel were Prof Richard Thorpe from Leeds University, and Prof Robert Blackburn from Kingston University, who may well subscribe to this list and wish to comment. At the conclusion of the exercise the panel gave an overall report on the state of business management research, including that on entrepreneurship. Colleagues will note that the panel expressed reservations similar to those aired in the current debate, although finding areas of good work. Below are two key paragraphs from the report, #88 and 90, together with a link to the full report.

There have also been comments about the ABS journals list, published by what is now the Chartered Association of Business Schools. (the recent award of 'Chartered' status may be an indication of increasing political and economic influence by UK business schools.) The ABS list has some common DNA with the assessment exercises. One of the key inputs to the original list was a reverse engineering exercise which identified journals in which the work submitted to a previous panel had been rated most highly. Personally, I don't have any problems with that. Way back in the 90s a colleague and I simulated an earlier research assessment exercise in business and management using what was then known as the Social Science Citation index, now part of Web of Science, using techniques which I would now regard as crude. (Thomas, P. R. and D. S. Watkins (1998). "Institutional Research Rankings Via Bibliometric Analysis And Direct Peer Review: A Comparative Case Study With Policy Implications." Scientometrics 41(3): 335-355).

Nevertheless, our conclusion was that it was unnecessary to conduct a peer review exercise specifically for periodic research assessment, since in business and management at least one could get the same rankings through desk research using simple ISI Impact Factors (or derivatives thereof) of claimed publications as a proxy of academic 'value'. Some bibliometric data is now available to different assessment panels, but not all use it, nor are obliged to. UK government has still turned its face against the use of bibliometrics instead of separate peer review in future research assessment exercises, although of course metrics like the IF are essentially summations of the whole academic community's (favourable?) peer review through citation of the work in question. (Using journal of publication as a proxy for the value of individual items if used in this way: other metrics are available!).

Another contributor noted that 25% of the overall assessment in REF2014 was based on 'Impact@. While this is true, and is to be greatly welcomed, it is important to note that Impact is a tightly bounded concept which has to be traced back to specific 'REFable' outputs. That of course means that the research which is referenced in an impact claim has to be published under the same conditions, and with the same constraints, as any other work – which the current debate has shown has argued is likely to reduce innovation. If true, as I tend to agree, potential Impact is also by definition constrained.

Research assessment practices in Australia have greatly influenced the current situation in the UK, as I understand it. But when I speak to science policy analysts and bibliometricians from Australia I get the strong impression that impact is construed in a broader sense (perhaps meta level might be a better phrase) - which may well influence the nature of the research which entrepreneurship researchers do Down Under: let's hope so for their sake.

Warm wishes

David

Professor David Watkins

Research Professor | Graduate Studies

Southampton Solent University | East Park Terrace | Southampton SO14 0YN

T: 023 8201 3608 | E: david.watkins@solent.ac.uk | www.solent.ac.uk

"89. The topics examined in this field were diverse, offering both continuity with previous studies and insights into new areas, adding to the accumulation of knowledge and contributing to theory. These spanned analyses of people, that is the characteristics of entrepreneurs and their staff (e.g. demographics, gender, ethnicity, age); organisational studies (e.g. start-ups, family businesses, corporate entrepreneurship, SMEs); effects on the economy and society (small business performance, internationalisation); interactions with their wider environments (e.g. public policy interventions; finance, regulation); and raising awareness of business-ownership and entrepreneurship through research-based, entrepreneurship education.

90. Although showing a growing diversity in the methodologies used, the majority of outputs continue to be empirical, using positivist approaches. The outputs showed an increase in the use of large-scale data sources, utilising advanced quantitative techniques, but on occasion these were felt to be rather unimaginative with little to show for the sophistication in technique in terms of contribution to theory and/or practice. However, there was also a growing body of outputs that were more reflective, taking a critical approach to the phenomenon of entrepreneurship and associated concepts. Weaker outputs tended to be studies with insufficient contextualisation or theoretical contribution. Submissions that utilised longitudinal or ethnographic data were rare and although an ongoing challenge, may justify attention in the future. The outputs submitted to the panel used a variety of routes for dissemination, including refereed conference proceedings, specialist book series, and subject field and general management peer-reviewed journals. As with other subject areas reviewed by the panel, the overwhelming majority of outputs were disseminated through North American and European journals, with only a few books or research monographs presented for evaluation. Almost a fifth of outputs in the field were assessed as world-leading and a half as internationally excellent."

The full report can be found at: http://www.ref.ac.uk/media/ref/content/expanel/member/Main%20Panel%20C%20overview%20report.pdf

    Disclaimer