This is a preview of subscription content, access via your institution
Buy this article
- Purchase on SpringerLink
- Instant access to the full article PDF.
Prices may be subject to local taxes which are calculated during checkout
The Web of Science and the PatentsView data for the study were retrieved from the public repository (https://doi.org/10.5281/zenodo.7258379 (ref. 10)) made publicly available by Park et al.1. The SciSciNet data source4 (https://doi.org/10.6084/m9.figshare.c.6076908 (ref. 11)) and the DBLP citation network v146 (https://www.aminer.cn/open/article?id=655db2202ab17a072284bc0c) are publicly available to download. The data for the reanalysis that we performed on these two datasets are publicly available at https://github.com/VincentHolst/reanalysis_declining_disruption (ref. 12). The data used to recreate Supplementary Fig. 17 were obtained either as source data from ref. 1 or by digitizing figures from refs. 13,14 using WebPlotDigitizer (https://automeris.io), and are publicly available alongside the other reanalysis data.
The reanalysis code for this publication is publicly available at https://github.com/VincentHolst/reanalysis_declining_disruption (ref. 12). It is based on the original analysis code (https://doi.org/10.5281/zenodo.7258379 (ref. 10)) made publicly available by Park et al.1 and uses the same software packages as Park et al.1: pandas v1.4.3, numpy v1.23.1, matplotlib v3.5.2 and seaborn v0.11.2. To replicate the regression table, we used StataMP v18.0 (reghdfe v6.12.3). To digitize figure data for Supplementary Fig. 17, we used WebPlotDigitizer v5.2 (https://automeris.io).
Park, M., Leahey, E. & Funk, R. J. Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023).
Funk, R. J. & Owen-Smith, J. A dynamic network measure of technological change. Manage. Sci. 63, 791–817 (2017).
Waskom, M. Treat binwidth as approximate to avoid dropping outermost datapoints. GitHub https://github.com/mwaskom/seaborn/pull/3489 (2023).
Lin, Z., Yin, Y., Liu, L. & Wang, D. SciSciNet: a large-scale open data lake for the science of science research. Sci. Data 10, 315 (2023).
Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013).
Tang, J. et al. Arnetminer: extraction and mining of academic social networks, KDD ’08, 990–998 (ACM, 2008).
Macher, J. T., Rutzer, C. & Weder, R. Is there a secular decline in disruptive patents? Correcting for measurement bias. Research Policy 53, 104992 (2024).
Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019).
Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023).
Funk, R. J., Park, M. & Leahey, E. Papers and patents are becoming less disruptive over time (1.0) [Data set]. Zenodo https://doi.org/10.5281/zenodo.7258379 (2022).
Yin, Y. SciSciNet: a large-scale open data lake for the science of science research. Figshare https://doi.org/10.6084/m9.figshare.c.6076908.v1 (2023).
Holst, V. & Algaba, A. Dataset artefacts are the hidden drivers of the declining disruptiveness in science. GitHub https://github.com/VincentHolst/reanalysis_declining_disruption (2026).
Funk, R. J. X https://x.com/russellfunk/status/1761950261671526618 (retrieved 30 August 2024).
Park, M., Leahey, E. & Funk, R. J. The decline of disruptive science and technology. Preprint at https://doi.org/10.48550/arXiv.2106.11184 (2021).
Hofmann, H., Wickham, H. & Kafadar, K. value plots: boxplots for large data. J. Comput. Graph. Stat. 26, 469–477 (2017).
Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technol. Forecast. Soc. Change 172, 121071 (2021).
We thank M. Park, E. Leahey and R. J. Funk for making their code and source material public, and for responding to a previous version of this Comment, highlighting the different robustness checks in their manuscript. We thank S. J. Klein for helpful advice and M. Waskom for maintaining the open source seaborn library, which enabled us to quickly identify the bug in the plotting of the histograms.
Work was supported by the Research Council (OZR) of the VUB.
The authors declare no competing interests.
This figure replicates the observation that papers with CD5 = 1 are driving the decline in disruptive science for the SciSciNet data source4 (with 39,888,199 papers between 1944 and 2011), which originated from the Microsoft Academic Graph. a, The distribution of the CD5 index for SciSciNet, created using the binwidth parameter in seaborn 0.11.2. Here again, the largest data points are hidden. b, The correct histogram of the underlying dataset. A peak at CD5 = 1 is revealed, corresponding to 8,861,343 additional papers. c, The time evolution of the average CD5 index. When dropping the outliers with CD5 = 1, the decline in disruptiveness reduces by 97%, from −0.31 with outliers (top curve, 2011 vs. 1944) to −0.01 without (bottom curve, 2011 vs. 1944). Excluding papers with zero references impacts the data similarly (Fig. 2 and Extended Data Fig. 2), reducing the decline by 87%, to −0.04 (dashed curve, 2011 vs. 1944). The shaded bands correspond to 95% confidence intervals. Moreover, the curve with papers with CD5 = 1 omitted is the curve corresponding to the histogram (a).
This figure displays that most papers in the SciSciNet data source4 (n = 39,888,199) and most patents in the PatentsView data source (n = 2,926,923) with CD5 = 1 have zero references. a, Our analysis shows that PatentsView contains 142,362 patents with CD5 = 1 between 1980 and 2010, of which 78% appear in the database with zero references. b, Within the category of patents with CD5 = 1, the relative frequency of patents with zero references is stable between 1980 and 2010. c, The relative frequency of patents with CD5 index exactly equal to one and zero references is decreasing over time. Therefore, a substantial part of the reported decline in the disruptiveness of technological knowledge over time can be attributed to a relatively increasing metadata quality over time. It is also intriguing to note how well the shape of this curve resembles the shape of the top curve shown in Fig. 1f. d, SciSciNet4 shows a similar behaviour with 8,861,343 papers having CD5 = 1 between 1944 and 2011, of which 97% appear in the database with zero references. e, Within the category of papers with CD5 = 1, the relative frequency of papers with zero references is stable between 1944 and 2011. f, The relative frequency of papers with CD5 index exactly equal to one and zero references is decreasing over time. Therefore, a substantial part of the observed decline in the disruptiveness of scientific knowledge over time can be attributed to a relatively increasing metadata quality over time. It is also intriguing to note how well the shape of this curve resembles the shape of the top curve shown in Extended Data Fig. 1c.
This figure displays the average CD5 index over time for six data sources and five different patent categories. The data sources are JSTOR (1,588,088 papers), the American Physical Society corpus (461,359 papers), Microsoft Academic Graph (random sample of 1,000,000 papers), and PubMed (1,563,211 papers). For reference, the Web of Science (22,479,429 papers) and PatentsView (2,926,923 patents) data sources are also included. The patent categories are Chemical (517,964 patents), Computers and communications (748,849 patents), Drugs and medical (321,449 patents), Electrical and electronic (734,769 patents), and Mechanical (603,892 patents). Shaded bands correspond to 95% confidence intervals. a, The temporal evolution of the average CD5 index for different data sources as presented in Park et al.1 (Extended Data Fig. 6 in1). b, When dropping the outliers with CD5 = 1, the decline in disruptiveness reduces by 100%, 107%, 100%, and 61% for JSTOR, the American Physical Society corpus, Microsoft Academic Graph, and PubMed, from −0.16, −0.27, −0.31, and −0.18 with outliers (panel a, 2006 vs. 1930 for JSTOR, 2010 vs. 1930 otherwise) to −0.00, +0.02, −0.00, and −0.07 without (panel b, 2006 vs. 1930 for JSTOR, 2010 vs. 1930 otherwise), respectively. c, The time evolution of the average CD5 index for different patent categories as presented in Park et al.1 (Fig. 2b in1). d, When dropping the outliers with CD5 = 1, the decline in disruptiveness reduces by 70%, 58%, 69%, 63 % and 71% for Chemical, Computers and communications, Drugs and medical, Electrical and electronic, and Mechanical patents, from −0.33, −0.24, −0.35, −0.32, and −0.38 with outliers (panel c, 2010 vs. 1980) to −0.10, −0.10, −0.11, −0.12, and −0.11 without (panel d, 2010 vs. 1980), respectively.
Supplementary Methods, including Supplementary Figs. 1–17, Supplementary Tables 1–6 and Supplementary References.





The author’s companion, Camas, in bear country.