...
The data used for bibliometric analyses can generally be regarded as inherently incomplete and inaccurate in some way, except perhaps for very small analyses. It is therefore not at all unimportant how sensitive the indicators used are to errors in the source data. We can ask how big a change removing or adding one publication or citation makes to the outcome, and through this try to assess the reliability of the results. In general, different datasets contain partly different publications and the ‘off-the-shelf’ pieces of software used to calculate citation impact indicators use different algorithms, so the results also vary depending on the data and the software. The citation time window also affects how much relative change a single citation can produce. A short citation window is clearly more sensitive. The higher number of citations over a longer time window means that the relative importance of a single citation is often relatively small.
Top x% indicators are not sensitive to a single, highly cited publication in the target group being reviewed. For example, it is irrelevant to the Top 10% indicator whether the publication is in the top 0.1% or the top 5% most cited in its class. On the other hand, near the limit of the Top 10% class, a single citation is enough to substantially increase the value of the publication. In such cases, the data used can determine the impact value of a publication in one direction or another. The effect may not be relatively significant if the set of publications being analysed is large (country-level analyses), but for small sets of publications (a research group, an individual researcher) the difference can become significant. The same applies, of course, if a publication is missing from the target group in the first place. Since the Top x% indicators are built on the idea that only a small proportion of publications are considered to have any value at all in terms of impact, it is very important to identify all publications by the analysed target group from the data used.
...