Optimizing evidence synthesis with AI: an example of systematic review on post-harvest losses and gender

Advances in technology create efficiency for evidence synthesis

The process of evidence synthesis begins with identifying all available information related to a topic. This is followed by thoroughly screening relevant studies using specific inclusion and exclusion criteria. The selected documents are reviewed in their entirety to extract key findings. Finally, the findings are synthesized to highlight key conclusions and identify gaps in knowledge.

Due to the rapid growth in publications and the emergence of diverse, fragmented research streams, the manual searches traditionally associated with evidence synthesis have become time-consuming and costly. For instance, a single search on a broad topic may yield over 30,000 results, rendering manual review unfeasible and creating a risk of bias due to omitted information.

Technological advances have enabled more efficient systematic reviews. Scientific literature databases store published information and offer bibliometric tools. These tools generate data on authors, years of publication, document types, languages, abstracts, and keywords. This facilitates searching and filtering based on selection criteria.

Recently, artificial intelligence (AI) has emerged as a way to further streamline the literature screening process, as well as the management, collection and analysis of data. Leveraging AI in systematic reviews allows researchers to overcome the limitations of traditional methods and achieve more accurate and reliable results. AI improves the efficiency and effectiveness of the review process by allowing researchers to focus on higher-level tasks, such as developing new research questions, or developing complex conceptual frameworks, which leads to higher-quality outputs.

However, despite the significant advancements in AI tools, challenges remain. Recent studies highlight transformative impact of AI, as well as the ongoing hurdles for systematic reviews, including data quality issues and the need for manual adjustments. For example, although AI algorithms have demonstrated remarkable proficiency in automating repetitive tasks, reducing bias and identifying patterns across vast datasets, they still require careful calibration and oversight.

Using AI in a systematic literature: a case study

AI played an important role in simplifying the review process for our recent review on gender dimensions of post-harvest losses. We used Bibliometrix and topicmodels, two free R libraries, to conduct this research. The systematic literature review began with the clear definition of the research question: How post-harvest loss research literature has integrated gender dimensions? This question informed the development of inclusion and exclusion criteria. These criteria guided the collection of relevant documents. We then reviewed titles and abstracts to ensure they aligned with our research topic, filtering out any that did not meet the inclusion criteria. Finally, we thoroughly reviewed the remaining documents to identify key characteristics and analysis themes, laying the groundwork for a comprehensive understanding of the topic. 

Using the Bibliometrix and topicmodels libraries allowed us to process large volumes of information, detect key concepts in the literature and visualize emerging trends. These tools are part of “scientometrics”, a field that quantitatively studies the development of science. In our case, we applied topic modeling through latent dirichlet allocation (LDA), which analyzes texts and generates topics based on the representativeness of the words used in each article. These topics supported our analysis by helping us to refine the inclusion criteria by identifying differences in concepts that we had previously dismissed as unimportant. For instance, although we initially believed that the terms “losses” and “wastage” were closely related to the post-harvest stage. However, when we used AI to identify relationships between titles, abstracts and keywords, we discovered that, in fact, “losses” are associated with reductions in the quantity and quality of agricultural products due to damage, decay and poor storage during the post-harvest stage. However, “waste” mainly focuses on the household stage of food consumption and how habits and relationships with food lead to food waste in households.

After refining our selection criteria, we searched for articles and found more than 7500. Using AI for topic analysis allowed us to significantly narrow down this pool of information by identifying the relevance of articles based on word count, author analysis and topic.

Topic analysis is a text-mining technique used to identify and categorize the main themes within a large collection of documents. After carefully checking of 75 articles, we arrived at a final selection of 48 documents. The final selection considered the criteria of representativeness and reliability, as well as the inclusion of gender as a central theme rather than a disaggregation of percentages of men and women. 

Other examples of how we used AI and its benefits to our study are outlined in Table 1.

Table 1. How AI was used as part of a systematic literature review and the benefits of using AI for the review

Although the use of AI offers advantages for data analysis, the researcher’s judgment remains important for interpreting the results, since the topics captured by the tool must be interpreted by the researcher, and sometimes limiting the number of topics or the repetition of words between topics is a challenge.

Our experience in using AI

Although we had previously conducted literature reviews, this was our first experience using AI in a review. Thanks to it, we were able to develop a first research document for review in four months, representing a reduction in the time spent. In our favor, we had a team with expertise in qualitative and quantitative methods, which allowed us to obtain the topics using computer tools and then analyze them.

In this process, we believe that AI helped us gain insight into the development of the literature and that by reviewing the documents, we learned more about the relationship between post-harvest losses and gender, since, as the study indicates, this is a growing field that requires further observation.

In summary, integrating AI into systematic literature reviews has transformed the process, significantly improving efficiency and accuracy. AI streamlines the review process, reducing time and costs while refining selection criteria by identifying nuanced differences between search terms, thereby improving the quality and relevance of research findings. Despite these advancements, challenges such as data quality and the need for manual oversight remain. In our case, the literature on post-harvest losses and gender is growing and much of the literature has not been reviewed by academic peers or the information comes from secondary sources that are not taken into account in a documentary corpus. Our experience demonstrated that, although AI greatly facilitated our review process, manual validation was crucial for ensuring a comprehensive analysis. Addressing these challenges and further developing AI capabilities will be essential for unlocking its full potential in research.