Revolutionizing Evidence-Based Research: A Pioneering Project

In an era marked by exponential growth in clinical studies and the continuous evolution of patient populations, the conventional methods of evidence-based medicine face a formidable challenge. The need for innovative approaches in evidence synthesis becomes apparent as the research community and industry drive advancements in clinical research methodologies.

Evidence-based research stands as the cornerstone of both scientific and clinical practice, guiding informed decision-making by systematically assessing the strength of interventions in terms of risks and benefits. It is a systematic approach to problem solving that allows the integration of the best available research evidence with the subject’s expertise and values.

In response to this evolving landscape, Amaris Consulting embarked on an applied industrial research project, aiming to revolutionize evidence-based research through the integration of cutting-edge natural language processing (NLP) and machine learning (ML) techniques. This initiative seeks to address the limitations of the traditional approach and enhance the efficiency and rigor of systematic review processes.

A promising avenue

The general process of evidence-based research involves a series of meticulous phases—formulating research questions, searching literature, evaluating evidence, integrating clinical experience and patient values, and disseminating results. This traditional approach, particularly through systematic reviews of the literature, has long been considered the gold standard, ensuring methodological soundness and unbiased evidence selection.

However, the landscape of clinical research is undergoing significant transformations. New effects evaluation methods, alternative study designs, and the continuous evolution of patient populations contribute to the increasing complexity of studies. The surge in both the volume and intricacy of biomedical research poses formidable challenges to the conventional paradigms of evidence synthesis.

The systematic review process, structured with specific steps to guarantee coherence and reproducibility, involves the creation of protocols, electronic search terms, manual literature selection through titles, abstracts, and full text, based on predefined criteria, and subsequent data extraction and synthesis of findings. As research volumes escalate, research groups find themselves under mounting pressure to deliver comprehensive and conclusive results while maintaining workflow efficiency.

Recognizing this growing challenge, there is a compelling need to enhance the systematic review process by introducing automation. Advances in computing, particularly the application of large-scale natural language processing, offer a promising avenue for practical research use cases. In the context of systematic reviews, the aspiration is that these technologies could usher in a higher level of automation, efficiency, and rigor in identifying evidence.

Structuring knowledge

Phase 1: Automatic selection of scientific studies

The initial phase of the project addresses the labor-intensive screening process by implementing multiple machine learning strategies. This not only accelerates the early identification of relevant studies but also significantly reduces the manual workload.

In the figure below, the horizontal axis represents the number of screened publications or studies, while the vertical axis illustrates the number of relevant studies found. Optimal screening occurs when all relevant studies are identified at the outset, reflected by the curve between the number of relevant studies found and those screened.

For reference, we computed the area under the curve (AUC) of optimal screening, depicted by the 100% perfect yellow line. This serves as a benchmark for comparison. The red line on the graph represents the performance of screening using machine learning (ML), while the green line represents manual screening without any form of support.

Our findings reveal a compelling story— the incorporation of ML significantly accelerates the early identification of relevant studies during the screening process, achieving an impressive 92% optimal performance.

Phase 2: Structuring of the selected studies

Building upon the success of Phase 1, this stage focuses on structuring the information extracted from selected studies. It introduces innovative features such as:

• Automatic Extraction of PICO Terms: The system automatically extracts PICO (Patient, Intervention, Comparison, Outcome) terms to aid in decision-making.
• Full Text Extraction from Selected Publications: The system assists in downloading PDF files of publications that have been selected as relevant.
• Extraction of Tables from Full Text PDFs: A new feature enables the organized transfer of tables from PDF articles to Excel files, facilitating data manipulation.
• Forms for Data Extraction from Full Texts: Specialized forms have been integrated into the tool for meticulous annotation of evidence from final publications. This enhances data collection efficiency and consistency.

Phase 3:  Visualization and interpretation of the data

The final phase revolves around the development of visual representations for a detailed characterization of selected studies. These visualizations offer a graphical perspective, aiding in the identification of patterns and trends. Additionally, an application is created, seamlessly integrating functionalities from all three phases, providing a centralized resource for analysis, interpretation, and evidence-based decision-making.

Connecting the dots

Amaris Consulting’s Health Economics & Market Access Center of Excellence is strategically implementing a measured results dissemination plan tailored to the unique needs of diverse potential clients and users. Given the versatile applicability of the research results across various sectors connected to the company, it is imperative to execute the dissemination plan with a meticulous approach, ensuring that innovative project outcomes are safeguarded and competitively advantageous information remains confidential.

The main stakeholders of the dissemination plan will be:

• Administration sector, decision-making and funding agencies / actors:

Key stakeholders in this sector include reimbursement and regulatory authorities, such as Regional Health Services, Departments of Health, and other bodies specializing in Research, Technology, and Innovation. Additionally, organizations and institutions responsible for the evaluation and financing of research, along with insurance companies and public and private foundations. This initiative aims to set a new standard for the quality and level of systematic reviews, enhancing transparency and minimizing the risk of bias.

• Health sector:

The health sector stakeholders comprise doctors and hospitals. The project positions itself as a new gold standard in evidence-based medicine, contributing to enhanced clinical decision-making in daily practice. It assists hospital therapeutic committees in comprehending complex biomedical research comparatively, thus elevating the quality of clinical guidelines.

• Private sector:

Noteworthy actors in the private sector include pharmaceutical companies and medical device companies. The project serves as a tool for analyzing and visualizing substantial volumes of biomedical research, enabling the industry to make informed strategic decisions swiftly.

Pioneering a new era

By engaging with these diverse stakeholders, we aim to establish our Center of Excellence as a pivotal contributor to advancements in evidence-based research, with tailored benefits for each target sector. This strategic dissemination plan underscores the commitment to delivering transformative solutions while safeguarding the competitive edge of the project’s innovative outcomes.

The successful completion of this research project is poised to revolutionize evidence-based research practices. The envisioned impact extends beyond traditional systematic reviews, reaching into medical decision-making, comparative research, clinical decision support systems, and health technology assessment. The ambition is to establish a new standard in the field, making advanced technologies commonplace and facilitating more informed and efficient practices across diverse sectors.

At Amaris Consulting, we stand at the forefront of innovation, facilitating the responsible integration of groundbreaking technologies to redefine the possibilities of healthcare and life sciences. Visit our Health Economics & Market Access Center of Excellence and discover how we can help you bring your technology to life by guiding you through your strategic evidence planning options and ensuring high-quality and robust project execution.

Project details

El Proyecto “Investigación de la aplicación de procesamiento de lenguaje natural y extracción de texto para la automatización de revisiones sistemáticas de evidencia biomédica.”  ha sido cofinanciado por el programa TORRES QUEVEDO, convocatoria 2019, del Ministerio de Ciencia e Innovación (PTQ2019-010389 / AEI / 10.13039/501100011033).