In this article, we problematize the notion that the continuously growing use of bibliometric evaluation can be effectively explained by ‘neoliberal’ ideology. A prerequisite for our analysis is an understanding of neoliberalism as both denoting a more limited set of concrete principles for the organization of society (the narrow interpretation) or as a hegemonic ideology (the broad interpretation). This conceptual framework, as well as brief history of evaluative bibliometrics, provides an analytical framing for our approach, in which four national research evaluation systems are compared: Norway, Russia, Sweden, and the United Kingdom. On basis of an analysis of the rationales for implementing these systems, as well as their specific design, we discuss the existence or non-existence of neoliberal motivations and rationales. Overall, we find that a relatively homogeneous academic landscape, with a high degree of centralization and government steering, appears to be a common feature for countries implementing national evaluation systems relying on bibliometrics. Such characteristics, we argue, may not be inductively understood as neoliberal but as indications of national states displaying strong political steering of its research system. Consequently, if used without further clarification, ‘neoliberalism’ is a concept too broad and diluted to be useful when analyzing the development of research evaluation and bibliometric measures in the past half a century.