This paper studies the production of dissertations in eight research fields in thenatural sciences, the social sciences and the humanities. In using doctoral dissertations itbuilds on De Solla Prices seminal study which used PhD dissertations as one of severalindicators of scientific growth (Price, Little science, big science, 1963). Data from theProQuest: Dissertations and Theses database covering the years 1950–2007 are used todepict historical trends, and the Gompertz function was used for analysing the data. Adecline in the growth of dissertations can be seen in all fields in the mid-eighties andseveral fields show only a modest growth during the entire period. The growth profiles ofspecific disciplines could not be explained by traditional dichotomies such as pure/appliedor soft/hard, but rather it seems that the age of the discipline appears to be an importantfactor. Thus, it is obvious that the growth of dissertations must be explained using severalfactors emerging both inside and outside academia. Consequently, we propose that theoutput of dissertations can be used as an indicator of growth, especially in fields like thehumanities, where journal or article counts are less applicable.

This study explores how policy discourses on academic career are articulated in Swedish higher education. Discourses on academic career are often expressing meritocracy and the necessity of competition, but also include demands for flexibil-ity and global participation. Recent decades of higher education policy have also stressed the importance of gender equity, which is particularly evident in the Nordic countries. Yet, how these discourses interact and impact on contemporary ideas on academic career remains unclear. We analyse a selection of Swedish government bills to explore present policy discourses on academic career mobility, and how these discourses express and create tensions for different staff groups. The findings shows that the notion, and promotion of career mobility in Swedish higher education features tensions between career advancement, transnational mobility and work life stability. It is also clear that some scholars are defined as more career mobile and successful than others. Hence, discourses on career mobility tend to give legitimacy to already existing work divisions and hierarchies partly undermining gender equity. In conclusion, our findings show tensions and contradictions in these policies, which give base for further nuanced and critical discussions on the current conditions and possibilities in Swedish higher education and academic career.

Patent systems rely on information infrastructures that enable searchers, examiners, and other specialists not only to consider legal requirements but also to gather market intelligence, competitor analysis, and other strategic business information. These resources are today considered fundamental to the assessment of a patent system's performance in terms of its reliability and legitimacy. However, this potential was constrained historically by the multiplicity of formats, languages, and time frames in which patents in different jurisdictions were published and issued. This essay traces how a secondary market for patent information materialised from a distinct commercial engagement with these peculiarities of patents as documents. In doing so, the essay explores how patent literature was abstracted, centralised, and filtered through private information providers such as Derwent Publications Ltd that began offering customised patent information products and services in the post-war decades.

Purpose

There are concerns in the academic publishing community that it is becoming more difficult to secure reviews for scientific manuscripts. This study examines trends in editorial and peer review processes in an ophthalmological journal over the last decade.

Methods

A retrospective analysis was performed of editorial data from the journal Acta Ophthalmologica containing all manuscript submissions between 2010 and 2020.

Results

The number of yearly submissions grew between 2010 and 2019 from 1014 to 1623, and in 2020, the number of submissions increased to 2449. In total, the number of submissions increased by 142% between 2010 and 2020. Similarly, the proportion of desk-rejected manuscripts increased from 48% to 67% during the period 2010–2020. The number of invitations needed to obtain one review showed an increase from 1.9 to 2.6 between 2010 and 2019, but remained stable between 2019 and 2020. However, the number of reviewers per reviewed manuscript, reviewed manuscripts per reviewer and time from invitation to completed review assignment remained almost constant between 2010 and 2020. Researchers based in North American were disproportionally often invited to review (18%) compared to their share of published articles (7%), and they also declined review invitation more frequently compared to scholars in other parts of the world.

Conclusions

The study revealed an increase in submitted manuscripts to an ophthalmological journal over the last decade, with a further increase during the COVID-19 pandemic. The number of reviewer invitations needed to obtain one review grew during the study period but remained constant between 2019 and 2020, despite a vast increase in submitted manuscripts. Hence, the burden for unique reviewers did not increase. Instead, the proportion of desk-rejected manuscripts grew, and the reviewer pool expanded, which allowed the annual average number of reviews by individual reviewers to remain stable. 

This review of the international literature on evaluation systems, evaluation practices, and metrics (mis)uses was written as part of a larger review commissioned by the Higher Education Funding Council for England (HEFCE) to inform their independent assessment of the role of metrics in research evaluation (2014–5). The literature on evaluation systems, practices, and effects of indicator uses is extremely heterogeneous: it comprises hundreds of sources published in different media, spread over disciplines, and with considerable variation in the nature of the evidence. A condensation of the state-of-the-art in relevant research is therefore highly timely. Our reviewpresents the main strands in the literature, with a focus on empirical materials about possible effects of evaluation exercises, ‘gaming’ of indicators, and strategic responses by scientific communities and others to requirements in research assessments. In order to increase visibility and availability, an adapted and updated review is presented here as a stand-alone—after authorizationby HEFCE.

Over the past decade, academic social networking sites, such as ResearchGate and Academia.edu, have become a common tool in academia for accessing publications and displaying metrics for research evaluation and self-monitoring. In this conceptual article, we discuss how these academic social networking sites, as devices of evaluation that build on both traditional values, objects, and metrics in academic publishing and on social media logics and algorithmic metrics, come to fulfil a need in the current academic (publishing) ecosystem. We approach this issue by identifying key affordances that arise in the interaction between platform and user. We then position these affordances in relation to potential needs of academics in today’s publishing landscape by drawing on Hafermalz’s metaphor of the ‘fear of exile’, which provides an alternative way of understanding the importance of visibility in the networked world, as a combination of competitive exposure and existential recognition. We end by considering the grounds on which the platforms may be attributed some level of legitimacy. This is done in order to understand the inherent contradiction between the broad use of the platforms and the fact that their integrity has been questioned repeatedly. We seek an answer to a legitimacy for the platforms in the fact that a pragmatic, mutual benefit exists between them and the research community; a benefit that is enhanced by the audit society influencing current academia. 

Measures of research productivity and quality are key components of academic life, and a successful academic career is heavily dependent on meeting quantified performance standards. For many years citation-based measures like the Impact Factor dominated the metrics landscape, but in the last two decades a swathe of new evaluation tools have emerged, including the h-index, ranked journal lists, and altmetrics. While the effectiveness of these metrics is debatable across many disciplines, their use in the social sciences and humanities has attracted most criticism. This chapter is concerned with how early career academics are using and responding to evaluative metrics; their strategies and ambitions for the future, and their perceptions of how evaluative metrics influence their work. In-depth interviews with Australian academics in the social sciences and humanities allowed us to explore these questions and we are particularly interested in how competing ‘orders of worth’ come to the fore in these accounts, and how researchers negotiate rivalling demands and expectations. Drawing on Brandtner’s concept of ‘evaluative landscapes’, we suggest that metrics and indicators can be seen as signposts which are used to assess achievement and to navigate a pathway to an ‘idealised sense of self’.

The use of indicators and metrics for research evaluation purposes is well‐documented; however, less is known about their use by individual scholars. With a focus on the social sciences, this article contributes to the existing literature on indicators and metrics use in fields with diverse publication practices. Scholars in Australia and Sweden were asked about their use and reasons for using metrics. A total of 581 completed surveys were analyzed to generate descriptive statistics, with textual analysis performed on comments provided to open questions. While just under half of the participant group had used metrics, the Australians reported use in twice the proportion of their Swedish peers. Institutional policies and processes were frequently associated with use, and the scholars' comments suggest a high level of awareness of some metrics as well as strategic behavior in demonstrating research performance. There is also evidence of tensions between scholars' research evaluation environment and their disciplinary values and publication practices.

Purpose The article introduces selected theoretical approaches to time and temporality relevant to the field of library and information science, and it briefly introduces the papers gathered in this special issue. A number of issues that could potentially be followed in future research are presented. Design/methodology/approach The authors review a selection of theoretical and empirical approaches to the study of time that originate in or are of particular relevance to library and information science. Four main themes are identified: (1) information as object in temporal perspectives; (2) time and information as tools of power and control; (3) time in society; and (4) experiencing and practicing time. Findings The paper advocates a thorough engagement with how time and temporality shape notions of information more broadly. This includes, for example, paying attention to how various dimensions of the late-modern time regime of acceleration feed into the ways in which information is operationalised, how information work is commodified, and how hierarchies of information are established; paying attention to the changing temporal dynamics that networked information systems imply for our understanding of documents or of memory institutions; or how external events such as social and natural crises quickly alter modes, speed, and forms of data production and use, in areas as diverse as information practices, policy, management, representation, and organisation, amongst others. Originality/value By foregrounding temporal perspectives in library and information science, the authors advocate dialogue with important perspectives on time that come from other fields. Rather than just including such perspectives in library and information science, however, the authors find that the focus on information and documents that the library and information science field contributes has great potential to advance the understanding of how notions and experiences of time shape late-modern societies and individuals.

The advantages of altmetrics — the diversity of dissemination channels analysed, the speed of getting data, the openness of methods, and the ability to measure impact beyond the ‘scholarly realm’— could be seen as especially promising for fields that currently are difficult to study using established bibliometric methods and data sources. This paper reviews the benefits of using altmetric methods to analyse the impact of research in the humanities and tests some of the most common altmetric tools on a small sample of publications and authors. The findings indicate that many of the problems identified in research on the use of bibliometrics on the humanities are also relevant for altmetric approaches. The importance of non-journal publications, the reliance on print as well the limited availability of open access publishers are characteristics that hinder altmetric analysis. However, this study provides only a few examples and further studies are needed in order to examine the possibilities that altmetric methods offer.

 In this chapter, the possibility of using bibliometric measures for

evaluating research in the humanities is pondered. A review of recent attempts to

develop bibliometric methods for studying the humanities shows that organizational,

epistemological differences as well as distinct research practices in research fields

ought to be considered. The dependence on colleagues, interdisciplinarity and the

‘rural’ nature of research in many humanistic disciplines are identified as factors

that influence the possibilities of applying bibliometric methods. A few particularly

promising approaches are highlighted, and the possibility of developing a ‘bibliometrics

for the humanities’ is examined. Finally, the intellectual characteristics of

specific disciplines should be considered when quality indicators are constructed, and

the importance of including scholars from the humanities in the process is stressed.

This article employs citation analysis on a micro level—the level of the cited document; in this case, WalterBenjamin’s Illuminations (1968/2007). The study showshow this frequently cited publication—more than 4,000citations in Web of Science—has been received. Thegrowth of citations and interdisciplinary citing is studied,and a novel approach—page citation analysis—is appliedto study how different parts of Illuminations have beencited.The article demonstrates howbibliometric methodscan be used together with qualitative accounts to mapthe impact and dissemination of a particular publication.Furthermore, it shows how bibliometric methods can beutilized to study intellectual structures in the humanities,and highlights the influence of the humanities onthe social sciences and sciences.

“Discipline” is commonly used to denote particular areas of knowledge, research and education. Yet, the concept is often not very well defined or even explicitly discussed when used in knowledge organisation and related fields. The aim of this article is to encourage and facilitate further reflections on academic disciplines, while at the same time offering insights on how this elusive concept might be understood. An overarching argument is that “discipline” should foremost be understood in relation to institutional and organisational features, and this is what distinguishes it from related terms such as, field, domain or topic. The etymology and history of the concept are reviewed along with a discussion of attempts to define and conceptualise disciplines. Insights are offered on how disciplines might be studied. Regardless of our views of disciplines, either as inherently out-dated constructs or as important features of a well-functioning academia, it is concluded that further precision or care in explicating the concept is needed.

Discipline is commonly used to denote particular areas of knowledge, research and education. Yet, the concept is often not very well defined or even explicitly discussed when used in knowledge organization and related fields. The aim of this article is to encourage and facilitate further reflections on academic disciplines, while at the same time offering insights on how this elusive concept might be understood. An overarching argument is that discipline should foremost be understood in relation to institutional and organizational features and this is what distinguishes it from related terms such as, field, domain or topic. The paper reviews the etymology and history of the concept, it discusses attempts to define and conceptualise disciplines and it offers insights on how disciplines can be studied. Regardless of our views of disciplines, either as inherently out-dated constructs or as important features of a well-functioning academia, it is concluded that further precision or care in explicating the concept is needed.

Academic disciplines demarcate, organise, and structure specific branches of knowledge. From a sociological perspective, they regulate reward structures and labour markets through the allocation of prestige and status. Disciplines emerged simultaneously with the modern research university and the establishment of a formalised system for scholarly communication. The establishment of modern disciplines as the principal unit for organising knowledge in academia is largely a history of formalisation, stabilisation, and bureaucratisation. The importance of disciplines as areas for knowledge generation has weakened over time, but their role in guiding how knowledge is disseminated, stabilised, and transferred remains. While many disciplines are rather stable in their organisation, they are also continuously challenged as new fields emerge, while others disappear. Their importance for how academic knowledge is conducted, taught, organised, and evaluated remains however strong, and an understanding of how disciplines function is essential for anyone aiming to study the production of academic knowledge.

Introduction Patent citations are an important indicator for studying science and technology interaction. Yet, empirical and conceptual studies of referencing practices in patents are largely lacking, which has resulted in an under-theorised understanding of patent citations.

Theoretical approach This study analyses previous research on patent citations and discusses these in relation to citation theories developed within science studies and bibliometrics. The ambition is, however, to advance beyond a comparative approach by adding an additional layer of analysis in which the overarching rhetorical function of references in patents and papers is considered.

Conceptual Analysis The analysis shows how referencing in patents have distinguishing characteristics in terms of who cites, the temporality of referencing, national and international context, quantity of references, and incentives for citing. These characteristics are discussed using the concept of fencing and stacking. Fencing illustrates how references in patents are used to demarcate claims, while the stacking of references in scientific papers is an act of cumulative persuasion.

Conclusions Substantial rhetorical and epistemological differences exist in how references are used in patents and papers, and these differences have implications for how citations patterns can be interpreted. The conceptualisation of referencing as fencing and stacking is suggested as a possible framework for interpreting empirical studies of patent citations. Overall, it is argued that a further theoretical, and empirical, understanding of referencing practices in patents are need in order to contextualise and problematise patent citations, and their current use in research and policy.

This thesis provides an in-depth study of the possibilities of applying bibliometric methods to the research field of literary studies. The four articles that constitute the backbone of this thesis focus on different aspects of references and citations in literary studies: from the use of references in the text to citation patterns among 34 literature journals. The analysis covers both an Anglo-Saxon context as well as research in Swedish literary studies, and the materials used include Web of Science data, references in the Swedish literature journal TFL (Tidskrift för Litteraturvetenskap) and applications to the Swedish Research Council (Vetenskapsrådet). A study is also made of the influence of one single publication—Walter Benjamin’s Illuminations—and its impact in literary studies and in wider academia.

The results from the four articles are elaborated upon using a theoretical framework that focuses on differences in the social and intellectual organization of research fields. According to these theories literary studies can be described as a fragmented, heterogenic, interdisciplinary and ‘rural’ field with a diverse audience. The fragmented and rural organization of the field is reflected in low citation frequencies as well as in the difficulties in discerning research specialities in co-citation mappings, while the analysis of the intellectual base (highly cited authors) is an example of the heterogenic and interdisciplinary character of the field, as it includes authors from many fields across the humanities and the social sciences.

The thesis emphasizes that bibliometric studies of research fields in the humanities need to incorporate non-English and non-journal publications in order to produce valid and fair results. Moreover, bibliometric methods must be modified in accordance with the organization of research in a particular field, and differences in referencing practices and citation patterns ought to be considered. Consequently, it is advised that bibliometric measures for evaluating research in these fields should, if used at all, be applied with great caution.

Hammarfelt, B. 2012. Following the Footnotes: A Bibliometric Analysis of Citation Patterns in Literary Studies. Department of ALM. Skrifter utgivna av Institutionen för ABM vid Uppsala universitet 5. 193 pp. Uppsala. ISBN 978-91-506-2279-9.

This thesis provides an in-depth study of the possibilities of applying bibliometric methods to the research field of literary studies. The four articles that constitute the backbone of this thesis focus on different aspects of references and citations in literary studies: from the use of references in the text to citation patterns among 34 literature journals. The analysis covers both an Anglo-Saxon context as well as research in Swedish literary studies, and the materials used include Web of Science data, references in the Swedish literature journal TFL (Tidskrift för Litteraturvetenskap) and applications to the Swedish Research Council (Vetenskapsrådet). A study is also made of the influence of one single publication—Walter Benjamin’s Illuminations—and its impact in literary studies and in wider academia. The results from the four articles are elaborated upon using a theoretical framework that focuses on differences in the social and intellectual organization of research fields. According to these theories literary studies can be described as a fragmented, heterogenic, interdisciplinary and ‘rural’ field with a diverse audience. The fragmented and rural organization of the field is reflected in low citation frequencies as well as in the difficulties in discerning research specialities in co-citation mappings, while the analysis of the intellectual base (highly cited authors) is an example of the heterogenic and interdisciplinary character of the field, as it includes authors from many fields across the humanities and the social sciences. The thesis emphasizes that bibliometric studies of research fields in the humanities need to incorporate non-English and non-journal publications in order to produce valid and fair results. Moreover, bibliometric methods must be modified in accordance with the organization of research in a particular field, and differences in referencing practices and citation patterns ought to be considered. Consequently, it is advised that bibliometric measures for evaluating research in these fields should, if used at all, be applied with great caution.

Bibliometric evaluation for research in the field of sciences can be a good way toassess the quality and factual basis of claims and can lead to more funding forauthors and for research work. However, due to the more diverse fields covered, thistype of evaluation is less effective in the world of humanities. Many professionalsand researchers in humanities fields believe that bibliometric evaluation is meantonly for STEM research and can’t properly assess any findings made in humanities.Four common claims made about bibliometrics in humanities are that bibliometricsdo not adequately cover the non-uniform nature of humanities; greater bibliometriccoverage will not solve all the research problems in humanities subjects; metrics usealready has an impact on humanities research practices and finally; other evaluationmethods, like altmetrics, are conventional.

Purpose – The aim of this article is to study a locally-oriented and book-based research field usingtwo Swedish language sources. Knowledge about citation patterns outside journal-based, Englishlanguage databases is scarce; thus a substantial part of research in the humanities and the socialsciences is neglected in bibliometric studies.Design/methodology/approach – Citation characteristics (publication type, language, gender andage) in the journal Tidskrift fo¨ r Litteraturvetenskap (2000-2009) and in grant applications (2006-2009)are studied. The datasets are analyzed further, adopting an author-co-citation approach for depictingand comparing the “intellectual base” of the field.Findings – It is shown that monographs and anthologies are the main publication channel inSwedish literary research. English, followed by Swedish, is the major language, and the gender ofauthors seems to influence citation practices. Furthermore, a common intellectual base of literarystudies that is independent of publication type and language could be identified.Practical implications – Bibliometric analysis of fields within the humanities needs to go beyondestablished databases and materials. The extensive use of recent English language monographs inSwedish literary studies informs the acquisition policy of university libraries serving literaturescholars.Originality/value – Citation analysis of non-English sources offers further knowledge aboutscholarly fields with a local and “rural” profile. The approach of using references in grant applicationsprovides a novel and promising venue for bibliometric research.

This article studies interdisciplinarity and the intellectual base of 34 literaturejournals using citation data from Web of Science. Data from two time periods, 1978–1987and 1998–2007 were compared to reveal changes in the interdisciplinary citing ofmonographs. The study extends the analysis to non-source publications; using the classificationof monographs to show changes in the intellectual base. There is support forincreased interdisciplinary citing of sources, especially to the social sciences, and changesin the intellectual base reflect this. The results are explained using theories on the intellectualand social organization of scientific fields and the use of bibliometric methods onthe humanities is discussed. The article demonstrates how citation analysis can provideinsights into the communication patterns and intellectual structure of scholarly fields in thearts and humanities.

Purpose: In this article, the ideas and methods behind the “patent-paper citation” are scrutinised by following the intellectual and technical development of approaches and ideas in early work on patentometrics. The aim is to study how references from patents to papers came to play a crucial role in establishing a link between science and technology. Design/methodology/approach: The study comprises a conceptual history of the “patent paper citation” and its emergence as an important indicator of science and technology interaction. By tracing key references in the field, it analyses the overarching frameworks and ideas, the conceptual “hinterland”, in which the approach of studying patent references emerged. Findings: The analysis explains how interest in patents – not only as legal and economic artefacts but also as scientific documents – became evident in the 1980s. The focus on patent citations was sparked by a need for relevant and objective indicators and by the greater availability of databases and methods. Yet, the development of patentometrics also relied on earlier research, and established theories, on the relation between science and technology. Originality/value: This is the first attempt at situating patentometrics in a larger societal and scientific context. The paper offers a reflexive and nuanced analysis of the “patent-paper citation” as a theoretical and historical construct, and it calls for a broader and contextualised understanding of patent references, including their social, legal and rhetorical function. © 2021, Björn Hammarfelt.

Purpose – The publication oeuvre of a researcher carries great value when academic careers are assessed, and being recognised as a successful candidate is usually equated with being a productive author. Yet, howpublications are valued in the context of evaluating careers is so far an understudied topic. The paper aims to discuss these issues.

Design/methodology/approach – Through a content analysis of assessment reports in three disciplines – biomedicine, economics and history – this paper analyses how externalities are used to evaluate publication oeuvres. Externalities are defined as features such as reviews and bibliometric indicators, which can be assessed without evaluating the epistemological claims made in the actual text.

Findings – All three fields emphasise similar aspects when assessing: authorship, publication prestige, temporality of research, reputation within the field and boundary keeping. Yet, how these facets of quality are evaluated, and the means through which they are assessed differs between disciplines. Moreover, research fields orient themselves according to different temporal horizons, i.e. history looks to the past and economics to the future when research is evaluated.

Research limitations/implications – The complexities involved in the process of evaluating candidates are also reflected in the findings, and while the comparative approach taken effectively highlights domain specific differences it may also hide counter-narratives, and subtle intradisciplinary discussion on quality.

Originality/value – This study offers a novel perspective on how publications are valued when assessing academic careers. Especially striking is how research across different fields is evaluated through different time horizons. This finding is significant in the debate on more overarching and formal systems of evaluation.

Purpose: The “Norwegian model” has become widely used for assessment and resource

allocation purposes. This paper investigates why this model has becomes so widespread and

influential.

Approach: A theoretical background is outlined in which the reduction of “uncertainty” is

highlighted as a key feature of performance measurement systems. These theories are then

drawn upon when revisiting previous studies of the Norwegian model, its use, and reactions

to it, in Sweden.

Findings: The empirical examples, which concern more formal use on the level of universities

as well as responses from individual researchers, shows how particular parts—especially the

“publication indicator”—are employed in Swedish academia. The discussion posits that the

attractiveness of the Norwegian model largely can be explained by its ability to reduce

complexity and uncertainty, even in fields where traditional bibliometric measurement is less

applicable.

Research limitations: The findings presented should be regarded as examples that can be

used for discussion, but one should be careful to interpret these as representative for broader

sentiments and trends.

Implications: The sheer popularity of the Norwegian model, leading to its application in

contexts for which it was not designed, can be seen as a major challenge for the future.

Originality: This paper offers a novel perspective on the Norwegian model by focusing on

its general “appeal”, rather than on its design, use or (mis)-use.

The prospects of altmetrics are especially encouraging for research fields in thehumanities that currently are difficult to study using established bibliometric methods. Yet,little is known about the altmetric impact of research fields in the humanities. Consequently,this paper analyses the altmetric coverage and impact of humanities-orientedarticles and books published by Swedish universities during 2012. Some of the mostcommon altmetric sources are examined using a sample of 310 journal articles and 54books. Mendeley has the highest coverage of journal articles (61 %) followed by Twitter(21 %) while very few of the publications are mentioned in blogs or on Facebook. Books,on the other hand, are quite often tweeted while both Mendeley’s and the novel data sourceLibrary Thing’s coverage is low. Many of the problems of applying bibliometrics to thehumanities are also relevant for altmetric approaches; the importance of non-journalpublications, the reliance on print as well the limited coverage of non-English languagepublications. However, the continuing development and diversification of methods suggeststhat altmetrics could evolve into a valuable tool for assessing research in thehumanities.

The prospects of altmetrics are especially encouraging for research fields in the humanities that currently are difficult to study using established bibliometric methods. Yet, little is known about the altmetric impact of research fields in the humanities. Consequently, this paper analyses the altmetric coverage and impact of humanities-oriented articles and books published by Swedish universities during 2012. Some of the most common altmetric sources are examined using a sample of 310 journal articles and 54 books. Mendeley has the highest coverage of journal articles (61 %) followed by Twitter (21 %) while very few of the publications are mentioned in blogs or on Facebook. Books, on the other hand, are quite often tweeted while both Mendeley’s and the novel data source Library Thing’s coverage is low. Many of the problems of applying bibliometrics to the humanities are also relevant for altmetric approaches; the importance of non-journal publications, the reliance on print as well the limited coverage of non-English language publications. However, the continuing development and diversification of methods suggests that altmetrics could evolve into a valuable tool for assessing research in the humanities.

Hanne Kjöller kastar sig in i den forskningspolitiska debatten med en genuin önskan om att förstå den bättre. Hon ställer sig tydligt på den skattebetalande allmänhetens och patienternas sida och för bitvis insiktsfulla resonemang. Men Kjöllers granskning av forskningens politisering haltar och ibland hamnar hon i motsägelsefulla positioner.

This paper highlights disadvantages of conceptual impreciseness, and advocates further attention to the labels and concepts used when classifying clusters or groups based on bibliographic data. The main focus of the analysis is on the concept of ‘discipline’ and how it is used in bibliometric research, but the implications concern a broader array of related terms. 

Given the increased role of bibliometric measures in research evaluation, it is striking that studiesof actual changes in research practice are rare. Most studies and comments on ‘a metric culture’in academia focus on the ideological and political level, and there is a clear shortage of empiricalstudies that analyze how researchers handle demands for accountability in context. In adopting amixed-methods approach involving both bibliometric data and answers form questionnaires, weprovide an in-depth study of how researchers at the faculty of Arts at Uppsala University (Sweden)respond to the implementation of performance-based research evaluation systems. Publicationpatterns from 2006 to 2013 show that journal publications, especially English-language ones, areincreasing, and the proportion of peer-reviewed publications has doubled. These changes are inline with the incentives of the evaluation systems under study. Answers to the survey confirm thatscholars are conscious about this development, and several respondents articulate a disagreementbetween disciplinary norms and external demands. However, disciplinary background aswell as career stage or academic age appears to have a significant influence on how individualresearchers react to the instigation of evaluation systems. Finally, responses to national andlocal evaluation regimes are complex, localized, and dependent on many factors. In-depthcontextualized studies of research practices are needed in order to understand how

Introduction. Our study critically engages with techniques of self-quantification in contemporary academia, by demonstrating how social networking services enact research and scholarly communication as a 'game'.

Method. The empirical part of the study involves an analysis of two leading platforms: Impactstory and ResearchGate. Observed qualities of these platforms will be analyzed in detail with concrete examples of gaming features in focus. Subsequently, we relate the development of these digital platforms to a broader 'quantified self movement'. Special attention will also be paid to how these platforms contribute to a general quantification of the academic (authorial) self. 

Theory. Theoretically we relate the 'gamification' of research to neoliberal ideas about markets and competition. Our analysis then extends to long-standing and fundamental ideas about self-betterment expressed in the philosophy of Peter Sloterdijk. 

Findings. Our study shows how social networking services, such as ResearchGate and Impactstory, enact researchers as 'entrepreneurs of themselves' in a marketplace of ideas, and the quantification of scholarly reputation to a single number plays an important role in this process. Moreover, the technologies that afford these types of quantifiable interactions affect the 'unfolding ontology' of algorithmic academic identities. 

Conclusions. The gamification of quantified academic selves intensifies the competitive nature of scholarship, it commodifies academic outputs and it might lead to goal displacement and cheating. However, self-quantification might also serve as a liberating and empowering activity for the individual researcher as alternative measures of impact and productivity are provided by these platforms.

Global university rankings have become increasingly important ‘calculative devices’ for assessing the ‘quality’ of higher education and research. Their ability to make characteristics of universities ‘calculable’ is here exemplified by the first proper university ranking ever, produced as early as 1910 by the American psychologist James McKeen Cattell. Our paper links the epistemological rationales behind the construction of this ranking to the sociopolitical context in which Cattell operated: an era in which psychology became institutionalized against the backdrop of the eugenics movement, and in which statistics of science became used to counter a perceived decline in ‘great men.’ Over time, however, the ‘eminent man,’ shaped foremost by heredity and upbringing, came to be replaced by the excellent university as the emblematic symbol of scientific and intellectual strength. We also show that Cattell’s ranking was generative of new forms of the social, traces of which can still be found today in the enactment of ‘excellence’ in global university rankings.

While bibliometric indicators, such as the journal impact factor, have long played an important role in many STEM disciplines it has been repeatedly shown that established bibliometric methods have limited use in the humanities. Using a questionnaire on metrics use and publication practices in Australia and Sweden, we tested the assumption that indicators play a minor role among humanities scholars. Our findings show that our respondents use indicators to a considerable degree, with a range of indicators and rankings being employed. The scholars use metrics as part of institutional policy, in CVs and applications, as well as for general promotion of their work. Notable in our results is that a much larger share of researchers (62%) in Australia used metrics compared to Sweden (14%). Scholar’s attitudes regarding bibliometrics are mixed; many are critical of these measures, while at the same time feeling pressured to use them. One main tension described by our respondents is between intradisciplinary criteria of quality and formalized indicators, and negotiating these “orders of worth” is a challenging balancing act, especially for younger researchers.

In this article, we problematize the notion that the continuously growing use of bibliometric evaluation can be effectively explained by ‘neoliberal’ ideology. A prerequisite for our analysis is an understanding of neoliberalism as both denoting a more limited set of concrete principles for the organization of society (the narrow interpretation) or as a hegemonic ideology (the broad interpretation). This conceptual framework, as well as brief history of evaluative bibliometrics, provides an analytical framing for our approach, in which four national research evaluation systems are compared: Norway, Russia, Sweden, and the United Kingdom. On basis of an analysis of the rationales for implementing these systems, as well as their specific design, we discuss the existence or non-existence of neoliberal motivations and rationales. Overall, we find that a relatively homogeneous academic landscape, with a high degree of centralization and government steering, appears to be a common feature for countries implementing national evaluation systems relying on bibliometrics. Such characteristics, we argue, may not be inductively understood as neoliberal but as indications of national states displaying strong political steering of its research system. Consequently, if used without further clarification, ‘neoliberalism’ is a concept too broad and diluted to be useful when analyzing the development of research evaluation and bibliometric measures in the past half a century.

Although established forms of peer review are often criticized for being slow, secretive, and even unfair, they are repeatedly mentioned by academics as the most important indicator of quality in scholarly publishing. In many countries, the peer review of books is a less codified practice than that of journal articles or conference papers, and the processes and actors involved are far from uniform. In Sweden, the review process of books has seldom been formalized. However, more formal peer review of books has been identified as a response to the increasing importance placed on streamlined peer-reviewed publishing of journal articles in English, which has been described as a direct challenge to more pluralistic publication patterns found particularly in the humanities. In this study, we focus on a novel approach to book review, Kriterium, where an independent portal maintained by academic institutions oversees the reviewing of academic books. The portal administers peer reviews, providing a mark of quality through a process which involves reviewers, an academic coordinator, and an editorial board. The paper studies how this process functions in practice by exploring materials concerning 24 scholarly books reviewed within Kriterium. Our analysis specifically targets tensions identified in the process of reviewing books with a focus on three main themes, namely the intended audience, the edited volume, and the novel role of the academic coordinator. Moreover, we find that the two main aims of the portal–quality enhancement (making research better) and certification (displaying that research is of high quality)–are recurrent in deliberations made in the peer review process. Consequently, we argue that reviewing procedures and criteria of quality are negotiated within a broader discussion where more traditional forms of publishing are challenged by new standards and evaluation practices.

We analyse how the Nordic contribution to health economics has evolved over the past three decades – in quantitative and qualitative terms. Using a dataset of publications from five prominent field journals for health economics, we combine different empirical methods to analyse the general trends in terms of number of distinct publications, topics covered, and co-authorship relationships between countries and individuals. We find that the Nordic countries are responsible for a stable share of international publications in health economics. The topics that Nordic health economists publish on are relatively similar to those most prevalent in the international community, even though health insurance is remarkably absent as a research topic in Nordic countries. In terms of links between countries and co-authors, we see that Nordic researchers are well embedded in the international community, and that the Nordic research community has moved toward less hierarchical relationships.

The use of bibliometric indicators on individual and national levels has gathered considerable interest in recent years, but the application of bibliometric models for allocating resources at the institutional level has so far gathered less attention. This article studies the implementation of bibliometric measures for allocating resources at Swedish universities. Several models and indicators based on publications, citations, and research grants are identified. The design of performance-based resource allocation across major universities is then analysed using a framework from the field of evaluation studies. The practical implementation, the incentives as well as the ‘ethics’ of models and indicators, are scrutinized in order to provide a theoretically informed assessment of evaluation systems. It is evident that the requirements, goals, possible consequences, and the costs of evaluation are scarcely discussed before these systems are implemented. We find that allocation models are implemented in response to a general trend of assessment across all types of activities and organizations, but the actual design of evaluation systems is dependent on size, orientation, and the overall organization of the institution in question.

The number of publications has been a fundamental merit in the competition for academic positions since the late 18th century. Today, the simple counting of publications has been supplemented with a whole range of bibliometric measures, which supposedly not only measure the volume of research but also its impact. In this study, we investigate how bibliometrics are used for evaluating the impact and quality of publications in two specific settings: biomedicine and economics. Our study exposes the extent and type of metrics used in external evaluations of candidates for academic positions at Swedish universities. Moreover, we show how different bibliometric indicators, both explicitly and implicitly, are employed to value and rank candidates. Our findings contribute to a further understanding of bibliometric indicators as “judgment devices” employed to evaluate individuals and their published works within specific fields. We also show how “expertise” in using bibliometrics for evaluative purposes is negotiated at the interface between domain knowledge and skills in using indicators. In fact, examiners in these documents emerge as experts in three roles: 1) as domain experts 2) experts on metrics and 3) experts on how metrics are used and valued within their field. In short expertise here means evaluating not only publications but also judgment devices. In line with these results we propose that the use of metrics in this context is best described as a form of “citizen biblometrics” – an underspecified term which we build upon in this paper.

A researcher’s number of publications has been a fundamental merit in the competition for academic positions since the late 18th century. Today, the simple counting of publications has been supplemented with a whole range of bibliometric indicators, which supposedly not only measures the volume of research but also its impact. In this study, we investigate how bibliometrics are used for evaluating the impact and quality of publications in two specific settings: biomedicine and economics. Our study exposes the various metrics used in external evaluations of candidates for academic positions at Swedish universities. Moreover, we show how different bibliometric indicators, both explicitly and implicitly, are employed to assess and rank candidates. Our findings contribute to a further understanding of bibliometric indicators as ‘judgment devices’ that are employed in evaluating individuals and their published works within specific fields. We also show how ‘expertise’ in using bibliometrics for evaluative purposes is negotiated at the interface between domain knowledge and skills in using indicators. In line with these results, we propose that the use of metrics we report is best described as a form of ‘citizen bibliometrics’—an underspecified term which we build upon in the article.

This paper builds on emerging concerns with how temporality and spatiality unfold in, and order, academic evaluation practices. We unpack how the notion of ‘trajectory’ – a simultaneously prospective and retrospective narrative device permeating contemporary academic evaluation discourses – is mobilized within a particular evaluation site. Materials for our study are drawn from reports commissioned by Swedish universities when hiring for new professors. These texts are authored by external referees who rank and compare candidates, in this case for associate and full professorship positions in biomedicine. By using the theoretical perspective of ‘narrative infrastructures’ we explore how the referee reports mobilize ‘trajectories’ to weave together disparate bits of evidence extracted from the bylines of biomedical researchers’ CVs: publication numbers, impact factors, authorship positions and ‘earning power’. Our analysis finds certain resemblances across reports of what constitutes an ideal candidate’s career trajectory, but none of these are completely identical. We consider how ‘the trajectory’ is evoked as a singularity within this genre of writing, thereby bestowing retrospectively a sense of coherence and purpose on the past performance and prospective development of careers. We discuss the implications of our findings in terms of how ‘trajectorism’ shapes evaluation in academic biomedicine and possibly beyond, and propose suggestions for how this dominant narrative might be challenged.