Replace "Impact Factors"!Can it achieve "transcendence"?

Text ｜ "China Science News" reporter Han Yangmei Profile Reporter Yan Wenyi

A few days ago, the Chinese Academy of Sciences Literature and Intelligence Center (hereinafter referred to as the Literature Intelligence Center) issued a notice saying that in 2022, the partition indicators no longer adopt the "three -year average impact factor", but instead replaced it with the "journal transcendence index" (upgraded version).

In 2019, the Literature Intelligence Center proposed the "Journal Excellence Index" for the first time, including the basic version and upgrade version. The advantage of the upgraded version is to weaken the dependence on the prefabricated discipline system, the interdisciplinary interdisciplinary of the journal, and the bottleneck of the breakthrough mean index. After 3 years of trial, in the future, only the upgraded version of the "journal surpass index" will be adopted.

Yang Liying, director and researcher of the Measurement and Evaluation Department of the Literature and Intelligence Center, told the Journal of China to publish the "Journal Excellence Index". The core is to correct the limitations of the influence factors calculation of the past journal to better reflect the periodic influence. It is worth noting that it is not a universal indicator, especially for absolutely determining the quality and scientific research level of a scientific research personnel.

How to calculate the journal transcendence? What are the applicable scenarios? What are the help of the improvement of the scientific research evaluation system? Focusing on related issues, the "China Science News" interviewed Yang Liying, the deputy director and associate researcher of the measurement and evaluation department of the Literature and Intelligence Center, Shen Shisi, and researcher Li Menghui.

The limitations and misuse of the influence factor

"China Science News": Why no longer use the "three -year impact factor"?

Shen Zi Si:

From the perspective of measurement, we see the problem of influencing factor.

First of all, in terms of calculation method, the papers involved in the nominance of the molecular denominator of the influence factor calculation are different. The denominator only considers research and review papers, while the molecules are all the papers in this journal (including research papers, reviews and review articles, etc. ) The reference situation makes some journals may improve the impact factors through some "manipulation", such as sending more reviews and comments, because the references of the review will be more than research papers.

The influencing factor only cares about the dissertation of the dissertation two years after the publishing of the paper, which is not suitable for some disciplines that take 3 to 5 years or even longer to reflect value.

Secondly, it is the problem of reference distribution partial state. In fact, the number of references of different papers in a journal is distributed. In other words, whether it contains a high -quotation article, which has a great impact on the fluctuations of the impact factors of the journal.

For example, the impact factor in a period of 2020 reached 508, which may be the highest influence factor to date. The key is that one article is close to 15,000 references. After removing it The paper with a volume of 5600 dropped to 74 immediately.

This method will make some journals "speculative", and more high -high articles are quoted to quickly improve the influencing factors. In fact, the number of references of most papers is lower than the impact factor. This reference distribution is highly biased, and the references cannot reflect the overall influence of the journal.

"China Science News": Recently, Kori Wei'an released the latest list of influence factor. Some journal influence factors have grown rapidly, causing widespread doubts from domestic scholars. For a while, everyone talked about "suffering factor for a long time". What do you think of this phenomenon?

Shen Zi Si:

We have observed that the rapid growth of the influence factors of domestic journals this year is part of the reason for the huge citations brought by some new crowns related papers published.

In fact, the surge in the influencing factors of international medical magazines such as "Liu Ye Dao" and "New England Medical Magazine" is also caused by this. Once these papers are excluded, the impact factor actually returns to the original level.

Published hot papers and review papers, all of which are some practices of the rapid improvement of the journal. However, it is important to emphasize that the number of references can only explain that the influence of the paper is relatively high, and it cannot directly represent its value and level.

"China Science News": Will the above phenomena lead to more and more scientific researchers pursuing popular disciplines, but unpopular majors are not interested. Over time, the distribution of academic internal volumes and academic resources has been brought over time?

Shen Zi Si:

I think so. Therefore, the evaluation must be distinguished by disciplines. The influence factors in the popular discipline journals may also be higher than the highest impact factor in unpopular discipline journals.

For example, the influence factors of mathematics disciplines and materials disciplines are very different, but they cannot represent the level of mathematical journals.

Different disciplines (such as reference length and reference cycle) of different disciplines are different. In addition, there are differences within the field (internal discipline).

Journal beyond index correction

"China Science News": What is the advantage of the "journal beyond index" released this time? Can it surpass the limitations of the influencing factor?

Shen Zi Si:

The original intention of proposing the "journal transcendence index" is to improve the impact factor, and better evaluate the influence of the journal.

The computing mechanism of the journal beyond index is to choose a paper from the journal, which is greater than the probability of the number of the same themes and the same literary papers selected from other journals.

Just like the students in the two classes are taller than they are, the students who randomly pick out from Class A are higher than Class B randomly picked out. Then, Class A's height is higher than Class B.

This calculation method avoids the inconsistency of molecular denominator. For example, comparison between research papers and comparison between reviewing thesis, the two are not comparable, so that the journals that publish a large number of comprehensive papers are no longer directly dominant. This method also better solves the problem of partial state. The role of the high -citation papers and the general papers in comparison is the same, and the role of a high -cited paper is greatly weakened.

In addition, by extracting the WOS full -library thesis and its reference relationship, the classification system of the theme level of thesis themes has been established to divide the discipline more detailed. Each paper has the research discipline and theme of its belonging. Compare the papers of two different themes together.

"Journal of Science": How does the "Journal Excellence Index" more scientifically and reasonable to reflect the influence of the journal?

Shen Zi Si:

We say that a journal is more influential than another journal. The intuitive feeling is the influence of a casual article of a journal, which is higher than the influence of another journal article.

The journal surpasses the index, which specially requires a high reference rate for each paper. If there is only a single or several reference rate, the overall advantage will not be dominant.

In other words, the influence factor is to compare the average number of references of the two journals, and the journal transcendence index is to compare each other in each article of the journal.

But is it absolutely good to be a single paper, and you cannot use the journal to surpass indexes to evaluate. When using evaluation indicators, you should pay attention to this problem.

No "factor" can be used to measure the level of research on a single paper

"China Science News": In 2019, "Nature" proposed the "disruptive index" to evaluate the level of research on the research. In your opinion, whether it is influencing factor or journal beyond index, what is its core role?

Li Menghui:

These "factor" were originally designed to evaluate the journal, but they were slowly misused and the original purpose was solidified.

The impact factor is more about evaluation journals, so it is not suitable for evaluating the quality of a single papers. Evaluating the quality of a single papers still needs to be judged through peer reviews.

From the perspective of influencing factors and transcendence indexes, the evaluation is actually the influence and whether it has received more attention. The "subversive index" commented on a result. Compared with the old achievements, it has a novelty and pioneering degree in content.

"China Science News": How to use the journal beyond index? How to view its role in scientific research and evaluation?

Yang Liying:

Any scientific evaluation indicators will have problems with appropriate application scenarios. The scientific research community's question of influencing factors is divided into two levels: First, the disadvantages of influencing factor index algorithms, but more are the excessive use of factors.

Beyond the index solves the main disadvantages of the influencing factor index algorithm, but it is not a universal indicator. It has its applicable scenarios and needs to be used reasonably.

The application of the journal beyond index aims to evaluate the academic influence of the SCI journal, provide a reference for academic submission, and support the macro judgment of the scientific research management department.

"China Science News": What kind of help will this change bring to the domestic scientific research evaluation system?

Yang Liying:

One of the important tasks in our "Fourteenth Five -Year Plan" is to build a scientific evaluation system with Chinese characteristics.

The evaluation system with Chinese characteristics refers to the evaluation orientation of Chinese scientific research management, evaluation data, including Chinese data, and evaluation methods to reflect China's contribution.

In fact, during the "Thirteenth Five -Year Plan" period, we have carried out a series of work around this goal. Among them, the "International Journal Early Warning List" reflects the orientation of serving Chinese scientific research management, and the transcendence index reflects the contribution of the journal evaluation method.

In 2022, we launched a series of solutions to continue to strengthen the above -mentioned Chinese characteristics, especially in the evaluation system of Chinese data, such as publishing the "Chinese Journal Partition Form" to achieve the equivalent evaluation of Chinese and English journals.

"China Science Journal" (2022-08-03 The first edition of the original title "Review Journal is not equal to the paper")

Edit | Fangyuan

Capture | Guo Gang

= PNG "Data-NickName =" Science Network "Data-Alias =" ScienceNet-Cas "Data-Signature =" In-circle major events, industry insights, occasionally gossip ... produced by the world's largest Chinese science community, Chinese Science News Agency.We: [email protected]; submitting [email protected]. "Data-from =" 0 "data-is_biz_ban =" 0 " /> Cooperation matters: [email protected]

Submission: [email protected]

Like this article?Praise + watch support!

- END -