Beyond Flawed and Skewed Bibliometrics: the percentile model for ranking of universities and researchers
The seminar will take as its starting point the well-known Norwegian bibliometrician Per O Seglen and his analysis of the Journal Impact Factor (JIF) presented in several publications during the 1990s. This analysis was also his point of departure for severe criticism towards the so called Norwegian Model and the weighting of publication channels.
Time: Wed 2015-02-04 16.00 - 17.00
Location: E1, Lindstedtsvägen 3
Participating: seminar in english by prof. Ulf Sandström
Ever since the 1990s most serious bibliometric work has departed from Seglen’s conclusions. Several organizations have pointed out the flaws of JIF and a (final) follow up was published as editorial in a recent issue of Science, concluding: “Using (journal impact) as a proxy for the importance of a paper is just plain wrong. As compared with a paper published in a higher-impact journal, there is no assurance that a paper published in a lower-impact journal is less important.” (McNutt Editor in Chief Science Journals) There are several serious alternatives to Journal Impact Factors (JIF). Over a period of at least ten years actual citations instead of expected citations have been standard for a large part of research assessment, also in Sweden. However, today field normalized citations is no longer considered as the “crown indicator” in discussions of publication performance. The MNCS (mean normalized citation score)-indicator is to a large extent flawed due to its dependence on field averages. In short, mean values are not the best way to handle skewed distributions. Much of bibliometric work has been based on the illusion of citations as a case of a normal distribution. Instead, citations are highly skewed. Just as Per O. Seglen showed long time ago it is not uncommon for almost 70 percent of the articles to have citations below average in areas where a small number of highly cited articles pull up the average.
Percentiles and percentile groups has been proposed as the alternative to the MNCS indicator. The percentile indicator shows the percentage of publications that belong to the n_th% most cited publications on the global scale (in Web of Science). The normalization is made by comparing the number of citations to each publication to the n_th percentile of the average citations to all publications from the same year, in the same sub-field area and of the same document type (articles, letters, proceeding papers and reviews).
The seminar will present a new model for ranking and funding decisions based on Percentiles and Field Adjusted Production (FAP). The model will be applied to Swedish data and the profile of KTH will be compared to other Swedish universities.