From the Leiden Manifesto to the evaluation reforms: challenges towards a responsible use of bibliometrics
DOI:
https://doi.org/10.20350/digitalCSIC/15376Keywords:
evaluation, indicators, bibliometrics, reformAbstract
This commentary reviews the principles of the Leiden Manifesto and its focus on the need of experts for contextualizing indicators to specific evaluation processes. This vision about the importance of contextualization and pluralization and the responsible use of indicators is central in the reforms of evaluation promoted by the European Commission and an international coalition of research institutions (CoARA). I argue that the implementation of these reforms in Spain will not be possible without a governance reform that devolves many evaluation processes and give more autonomy to research centres to manage recruitment and promotion of lecturers and researchers.
Downloads
References
Burrows, R. (2012). Living with the H-Index? Metric Assemblages in the Contemporary Academy. The Sociological Review, 60(2), 355–372. https://doi.org/10.1111/j.1467-954X.2012.02077.x
Delgado-López-Cózar, E., Ràfols, I., & Abadal, E. (2021). Carta: Por un cambio radical en la evaluación de la investigación en España. El profesional de la información, 30(3), e300309. https://doi.org/10.3145/epi.2021.may.09
Gläser, J., & Laudel, G. (2007). The Social Construction Of Bibliometric Evaluations. In R. Whitley & J. Gläser (Eds.), The Changing Governance of the Sciences (Vol. 26, pp. 101–123). Springer Netherlands. http://dx.doi.org/10.1007/978-1-4020-6746-4_5
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a
Martin, B. R., & Irvine, J. (1983). Assessing basic research: Some partial indicators of scientific progress in radio astronomy. Research Policy, 12(2), 61–90. https://doi.org/10.1016/0048-7333(83)90005-7
Moher, D., Bouter, L., Kleinert, S., Glasziou, P., Sham, M. H., Barbour, V., Coriat, A.-M., Foeger, N., & Dirnagl, U. (2020). The Hong Kong Principles for assessing researchers: Fostering research integrity. PLOS Biology, 18(7), e3000737. https://doi.org/10.1371/journal.pbio.3000737
Molas-Gallart, J. (2012). Research Governance and the Role of Evaluation: A Comparative Study. American Journal of Evaluation, 33(4), 583–598. https://doi.org/10.1177/1098214012450938
Molas-Gallart, J., & Rafols, I. (2018). Why bibliometric indicators break down: Unstable parameters, incorrect models and irrelevant prope. BiD: Textos Universitaris de Biblioteconomia i Documentació, (40). https://doi.org/10.1344/BiD2018.40.23
Ràfols, I. (2019). S&T indicators in the wild: Contextualization and participation for responsible metrics. Research Evaluation, 28(1), 7–22. https://doi.org/10.1093/reseval/rvy030
Ràfols, I., & Molas-Gallart, J. (2022). How to reform research evaluation in Spain. Institutional accreditation as a response to the European Agreement on research assessment. Letter. El Profesional de la información, 31(6), e310601. https://doi.org/10.3145/epi.2022.nov.01
Rafols, I., Molas-Gallart, J., Chavarro, D. A., & Robinson-Garcia, N. (2016). On the Dominance of Quantitative Evaluation in ‘Peripheral’ Countries: Auditing Research with Technologies of Distance (SSRN Scholarly Paper ID 2818335). Social Science Research Network. https://papers.ssrn.com/abstract=2818335
Rijcke, S. D., Wouters, P. F., Rushforth, A. D., Franssen, T. P., & Hammarfelt, B. (2016). Evaluation practices and effects of indicator use—A literature review. Research Evaluation, 25(2), 161–169. https://doi.org/10.1093/reseval/rvv038
Saenen, B., Morais, R., Gaillard, V., & Borrell-Damián, L. (2019). Research Assessment in the Transition to Open Science. European University Association (EUA). https://eua.eu/resources/publications/888:research-assessment-in-the-transition-to-open-science.html
Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117–131. https://doi.org/10.1007/s11192-005-0007-7
Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Jones, R., Kain, R., Kerridge, S., Thelwall, M., Tinkler, J., Viney, I., Wouters, P., Hill, J., & Johnson, B. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. https://doi.org/10.13140/RG.2.1.4929.1363
Wouters, P. (2013, October 3). Bibliometrics of individual researchers – the debate in Berlin. https://citationculture.wordpress.com/2013/10/03/bibliometrics-of-individual-researchers-the-debate-in-berlin/
Wouters, P., Glänzel, W., Gläser, J., & Rafols, I. (2013). The dilemmas of performance indicators of individual researchers–An urgent debate in bibliometrics. ISSI Newsletter, 9(3), 48–53. https://www.issi-society.org/media/1139/newsletter35.pdf
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2023 Ismael Ràfols
This work is licensed under a Creative Commons Attribution 4.0 International License.
