![]() ![]() 3 In academia, it is true that “we are drowning in information, but starving for knowledge” ( 4, p12).Įvidence syntheses-the collective term for robust summaries of evidence-aim to mitigate the issues of decoupled empirical evidence amidst ever‐growing research output. As a consequence, the relevance and impact of their research is reduced. Scientific discourse and cumulative knowledge are threatened if researchers fail to connect their empirical or theoretical analyses with past knowledge. As a result, research agendas neither build on nor advance previous findings but exist in isolation from the greater body of evidence. The growing volume of research makes it ever harder for practitioners and researchers to keep track of past and current findings in a specific discipline and across disciplines. Research output, as measured by the number of academic publications, continues to grow exponentially, 1, 2 placing scientists in danger of becoming decoupled from the discourse with which they are engaged. Our findings aim to support researchers in conducting better searches for better evidence synthesis. We call for database owners to recognize the requirements of evidence synthesis and for academic journals to reassess quality requirements for systematic reviews. Particularly, our findings demonstrate why Google Scholar is inappropriate as principal search system. Indeed, only half of the search systems analyzed and only a few Open Access databases can be recommended for evidence syntheses without adding substantial caveats. We found substantial differences in the performance of search systems, meaning that their usability in systematic searches varies. The study is the first to show the extent to which search systems can effectively and efficiently perform (Boolean) searches with regards to precision, recall, and reproducibility. A novel, query‐based method tests how well users are able to interact and retrieve records with each system. This study investigates and compares the systematic search qualities of 28 widely used academic search systems, including Google Scholar, PubMed, and Web of Science. Advice on which search engines and bibliographic databases to choose for systematic searches is limited and lacking systematic, empirical performance assessments. To date, it remains unclear which search systems are most appropriate for evidence synthesis and why. Yet, the search systems allowing access to this evidence provide varying levels of precision, recall, and reproducibility and also demand different levels of effort. Rigorous evidence identification is essential for systematic reviews and meta‐analyses (evidence syntheses) because the sample selection of relevant studies determines a review's outcome, validity, and explanatory power. ![]()
0 Comments
Leave a Reply. |