AI for the Evidence Synthesis
ijorrina
Created on April 10, 2024
The guide to AI tools that aid systematic reviews and other types of evidence synthesis
More creations to inspire you
Transcript
The guide and the map of the AI tools.
Start
AI to Support Evidence Synthesis
by Dr. olga Koz, alumni of ESI, 2021
This guide focuses on AI applications in evidence synthesis (meta-studies). It follows the systematic review protocol to map recently developed AI tools. Due to the rapid development of AI, this is a living guide that is constantly updated. I hope it helps to answer the following questions: - What is our (evidence synthesis librarians) future in the era of AI? - Should we be prudent and wait for the AI hype to be over? - How and where should we integrate AI tools?
Introduction
This guide maps AI tools to the evidence synthesis or a systematic review process.
Index
Proposal
AUTOMATION OF SR
EVIDENCE RETRIEVAL
COMPARE AI TOOLS
SR SOFTWARE
MY RESPONSE
Source
'Use this space to put a great sentence'
'Your content is liked, but it hooks even more if it is interactive'
Survey of Health Professionals about AI use in SR
Future
Needs
Replace
Minimal
Screening
Extraction
CredibilityReliability (replicability)ConfirmabilityTransferabilityTransparencyThe source of the evidenceEvidence Retrieval
Concerns
Meta-analysis vs Meta-Synthesis Evaluation CriteriaShaheen, N., Shaheen, A., Ramadan, A., Hefnawy, M. T., Ramadan, A., Ibrahim, I. A., … & Flouty, O. (2023). Appraising systematic reviews: a comprehensive guide to ensuring validity and reliability. Frontiers in Research Metrics and Analytics, 8.
AI and regulatory criteria for research
AMSTARCASPRoB-2
Criteria
Concern - Response
AConcern -Response
Concern - Response
Dependability, Consistancy
Confirmability/Neutrality
confidence in the 'truth' of the findings
showing that the findings are consistent and could be repeated
The extent of a researcher bias,
The authority of the source. LLM treated as a source of evidence
Report all LLMs
Data Bias
LLM or GPT is not a search engine
Modifying questions for LLMReport prompts
Is AI an author?
Modifying LLM anwers (RAG, Fine-tuning). Report the datasets and prompts.
+info
What does it mean?
Credibility
Measuring RoB with AI
Should we trust the ES "findings" based on AI?
Learn your LLMs
ArtificialIntelligence
Learn about AI apps: sources for data, LLM, and AI techniques.
Learn about AI techniques (RAG, Fine-Tuning, and so on)
Learn your LLMs
Synthesis Writing
5
Analysis
4
Evidence preparation
3
Evidence Collection
2
Design
1
PROTOCOL
Field Search, Thesaurus Browsing, Syntax and Operators, Filters, Pre-indexed Databases
Natural Language Understanding (NLU). Query meaning, context, and content.
LLM + Neural?
LLM empowers the search & generates the response and a summary with sources. RAG, SLM, Fine-Tuning, plug-ins
...
Semantic search
Lexical Search
Evidence Retrieval
1
2
Open Alex
Elicit
Consensus
3
Scite.ai
Datasets
AI Tools 4 LR
Semantic Scholar
Keenious
Scite.ai
LitMap
by Dr. Olga Koz
hhttps://bit.ly/LR-AI
A bar graph showing the concerns of AI in literature review software AI Viewed as Inevitable, Despite Concerns
Compare
Automation of SR
Source: Jimenez et al., 2022
AI to assist with the systematic review
Source: Jimenez et al, 2022
SR Software with AI
Next
AI
AI for Citation Analysis and literature mapping
https://libguides.kennesaw.edu/litMap
Scoping Review with AI
Next
Guide: AI for discovery & Literature review
Embedded AI
Link
Course: LR for EdD students
CREATING AI RESEARCH ASSISTANT
Next
The article
Mozgai, S., Kaurloto, C., Winn, J., Leeds, A., Heylen, D., Hartholt, A., & Scherer, S. (2023). Machine learning for semi-automated scoping reviews. Intelligent Systems With Applications, 19, 200249. https://doi.org/10.1016/j.iswa.2023.200249
Cierco Jimenez, Ramon & Lee, Teresa & Rosillo, Nicolás & Cordova, Reynalda & Cree, Ian & González Wong, Angel & Indave, Blanca. (2022). Machine learning computational tools to assist the performance of systematic reviews: A mapping review. BMC Medical Research Methodology. 22. 10.1186/s12874-022-01805-4.
Mozgai, S., Kaurloto, C., Winn, J., Leeds, A., Heylen, D., Hartholt, A., & Scherer, S. (2023). Machine learning for semi-automated scoping reviews. Intelligent Systems With Applications, 19, 200249. https://doi.org/10.1016/j.iswa.2023.200249
Reference
Spillias, S., Tuohy, P., Andreotta, M., Annand-Jones, R., Boschetti, F., Cvitanovic, C., … & Trebilco, R. (2023). Human-ai collaboration to identify literature for evidence synthesis.. https://doi.org/10.21203/rs.3.rs-3099291/v1
Jardim, P.S.J., Rose, C.J., Ames, H.M. et al. Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system. BMC Med Res Methodol 22, 167 (2022). https://doi.org/10.1186/s12874-022-01649-y
Post - presentation survey. I appreciate your answer to 3 questions
Please follow up!
okoz@kennesaw.eduhttps://libguides.kennesaw.edu/AI4LR
5
Your content is likable, but it only engages if it's interactive. Capture your audience's attention with an interactive photo or illustration.
6
Your content is liked, but it only engages when it's interactive. Capture your audience's attention with an interactive photograph or illustration.
With this function...You can add additional content to excite the brains of your students: videos, images, links, interactivity...Whatever you want!
1
Your content is appreciated, but it only engages if it's interactive. Capture your audience's attention with an interactive photo or illustration.
Did you know...In Genially you will find more than 1,000 templates ready to showcase your content and 100% customizable, which will help you in your classes?
Did you know that... We retain 42% more information when the content is dynamic? It is perhaps the most effective resource to capture the attention of your students.
Tip:Interactivity is the key to capturing the interest and attention of your students. A genially is interactive because your group explores and interacts with it.
Did you know...In Genially you will find over 1,000 templates ready to input your content and 100% customizable, which will help you in your classes?
3
Your content is liked, but only engages if it is interactive. Capture the attention of your audience with an interactive photograph or illustration.
Did you know...The window allows for adding more extensive content? You can enrich your genially by incorporating PDFs, videos, text... The content of the window will appear when clicking on the interactive element.
Bring it to life with an interactive window. Create a new layer with all the Genially features; include text, images, videos, tables, PDFs... Even interactive questions!
Got an idea?
¡QUIZ!
- Generate experiences with your content.
- It’s got the Wow effect. Very Wow.
- Make sure your audience remembers the message.
- Activate and surprise your audience.
Create a new layer with all the Genially features.
Bring it to life with an interactive window
Got an idea?
With this function...You can add additional content that excites the brains of your students: videos, images, links, interactivity... Whatever you want!