With so much scientific and technological progress nowadays, it seems strange to consider whether innovation is slowing down in general. Yet, this question is indeed worth asking. For example, some believe that there have been no major breakthroughs in the foundations of physics for decades, unlike in the early 20th century.
A paper published in Nature, the academic journal, on Jan 4, 2023 takes another look at the issue . The title "Papers and patents are becoming less disruptive over time" sounds alarming and immediately attracts a lot of attention, far beyond the circle of nerdy scientists. People are pondering why science and technology, across diverse disciplines, become more incremental on average. Soon, a lot of blames are going around and criticisms from all angles arrive. Here, let us have a quick look at what the article actually says, against some common blames inspired by it.
What does it mean by disruptive?
Papers and patents all cite previous works. The definition of disruptiveness is based on how much the citation pattern changes after a specific study. For example, study A cites article 1,2,3. Some later studies (such as B, C, D) cite study A. Now the question is, do study B, C, D also cite article 1,2,3? If article A is disruptive, it pushes the scientific field towards a different direction, then study B, C, D are less likely to cite previous articles (1,2,3). Using the citation data, the researchers calculated a value to represent how disruptive a paper is. There are other citation-based and text-based measures used, such as whether the wording in papers suggests breakthroughs. Those measures lead to similar results.
What is the main finding?
Using data on 25 million papers (from 1945 to 2010) and 3.9 million patents (1976 to 2010), the research shows that the average disruptiveness declined cross all the scientific and technical fields. The results are replicated with other databases with 20 million research papers. The results are robust, after considering changing publication practices, such as the increasing tendency of papers to cite previous work.
Is it because of frauds in science?
Some people argue that innovation and scientific progress are damaged by unethical behavior, such as frauds. Some scientific studies did turn out to be fraudulent. But for this explanation to take the blame, there needs to be a connection between fraud and low disruptiveness, as well as an increasingly large number of fraudulent studies in all scientific domains. Fraud seems unlikely to be the major factor.
What about low research quality?
Some people suggest that scientific research, in general, is declining in quality. Perhaps too many low-quality papers got published. However, the decline in disruptiveness is also found in top scientific journals. Therefore, low quality is unlikely the main reason behind the decline in disruptiveness.
What about the depletion of ‘low-hanging fruits’?
Still, others argue that the easy-to-achieve breakthroughs have already been achieved, leading to the depletion of ‘low-hanging fruits’. However, according to the Nature article, there is a quite stable number of highly disruptive papers across years. In other words, there are still as many breakthroughs. Just the average disruptiveness of publications went down.
So what is it then?
The study found that, on average, recent publications cited less diverse work, older work, and had more self-citations than decades ago. The use of narrower knowledge probably reduced the disruptiveness.
So, what do we do?
Let’s be clear: non-disruptive innovations are useful and desirable. We still benefit from upgrading operating systems in our computers or getting moderately better phones. Yet, the new study suggests, to facilitate disruptive innovation, we are better off to 1) focus more on quality than quantity in creation, and 2) use cutting-edge knowledge in diverse fields. This is not exactly a surprising insight, but it is much harder to do than say. Innovators often focus on their specific domains and don't bother to read about new progress in various other fields. Some intentional effort is needed from innovators, as well as those who fund them, such as universities and governments.
Kai Wang is the founder of Idea Early (ideaearly.com), a website dedicated to introducing new possibilities inspired by innovative ideas in various business and technological fields.