ABSTRACT
The use of lists of questionable journals as a means to ensure research quality and integrity is the subject of an ongoing debate due to their ambiguous criteria. To assess the reasonableness of these lists from a typological perspective, we examined how effectively they reflect differences in bibliometric attributes among distinct groups and whether these differences are consistent. Using the Early Warning Journal Lists from the National Science Library of the Chinese Academy of Sciences as a case study, we categorized listed journals by warning levels and publication years. Our findings indicate potential inconsistencies in the criteria used for assigning warning levels, as we observed varying degrees of differences (or their absence) among groups across different key academic indicators. Notably, when it comes to citation metrics like journal impact factor and journal citation indicator, it appears that these criteria may not have been considered for grouping, although this lack of clarity from the creators is apparent. This underscores the importance of conducting more scientific and thorough evaluations of lists of questionable journals, along with a greater emphasis on sharing precise standards and data. Our study also provides recommendations for future iterations of such lists by different institutions.
Acknowledgments
We would like to express our gratitude for the constructive suggestions provided by the associate editor and the anonymous reviewers. Their invaluable advice has greatly improved the quality of this study.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
The data that support the findings of this study are available at: https://doi.org/10.5281/zenodo.8245329.
Authors’ contributions
GYT developed the protocol for the study, conducted the search, extracted, updated, and analyzed the data, interpreted the results, and wrote, revised and edited the manuscript.
JYP conducted the search and extracted the data for some samples, and contributed to the revision of the manuscript.
Ethical considerations
This article does not contain any studies with human participants or animals performed by any of the authors.
Notes
1. The Beall’s List is a compilation of predatory journals. The original website of Beall’s list has been removed for certain reasons, and the list has been archived on the following website: https://beallslist.net/.
2. Cabell’s journal blacklist can be accessed through the following website, but it requires payment: https://noaa.cabells.com/about-predatory.
3. While some studies utilize the concept of “journal blacklists,” we have opted for the alternative concept of “the lists of questionable journals” due to its more inclusive nature.
4. NSLC is an affiliated institution of the Chinese Academy of Sciences, the highest-level academic institution for natural sciences in China (https://www.cas.cn/zz/yk/201410/t20141016_4225142.shtml). NSLC is recognized as one of the most prolific and cited research organizations in the field of library and information sciences in China, possessing a strong representation in this domain (http://english.las.cas.cn/About/about/).
5. Detailed information about the EWJL 2020 can be accessed through the following link: https://earlywarning.fenqubiao.com/#/en/early-warning-journal-list-2020.
6. Detailed information about the EWJL 2021 can be accessed through the following link: https://earlywarning.fenqubiao.com/#/en/early-warning-journal-list-2021.
7. MDPI is a scholarly publishing institution headquartered in Switzerland. This mega-publisher was initially incorporated on Beall’s list and was subsequently excluded on 28th October 2015 (https://www.mdpi.com/about/announcements/534).
8. Since EWJLs have not disclosed any details about the data, and considering its update frequency is once a year, we speculate that EWJLs focus on dynamic and real-time data. Therefore, we utilized data from the status of the journal in 2020 (before the flow) and in 2021(after the flow), without addressing its long-term data variations.
9. As Zhang et al. (Citation2022) pointed out, “The geographic diversity of authors may reflect the widespread recognition of the journal in the world.” Therefore, this indicator is considered an important academic metric in China.
