3. Search for evidence: AI or not AI

Searching for evidence: AI or not AI

Presented by Dr. Koz

AI for Discovery & Literature Review https://libguides.kennesaw.edu/AI4LR/about

Literature or Evidence Review

Objectives

Develop a search strategy

What

A thoroughly considered and systematic discovery protocol or a plan

Be able to clarify if you are searching for research reports, data, information or literature.

+ info

+ info

How

Where

Selecting appropriate search techniques and tools, including AI-driven ones

Indicate the sources for literature or evidence refer to repositories, collections, databases or datasets (corpus).

+ info

+ info

Section 1

Objective 1 Develop a search strategy

Phases of Searching for Evidence

Search Strategies

Phases

Concept Paper; Research Planning, "preliminary review"

•Develop a topic or research question, •Get familiar with topical concepts and sources

Preliminary Search

"Systematized traditional review" Systematic Review

•Plan and structure the process of searching •Maintain consistency in searching •Document the search process and results

Systematic Search

•Set up Articles Alerts *Topical journals and conferences

Monitoring

https://libguides.kennesaw.edu/lit_search

Search Strategy: What? How? Where?)

Indexes, Databases, Repositories, Datasets

Where

How

Search techniques & engines

Concepts, evidence, types of publications, level of evidence, and scope limits

What

Search Strategy: an illustration

Interdisciplinary indexes > Subject databases > Semantic and AI integrated discovery tools

Where

Thesaurus browsing (APA PsycArticles, ERIC , IEEE lexical search > citation chaining and mapping, pearl growing (Scopus, Google Scholar, Research Rabbit)> ML/NLP/Neural networks (SciSpace)

How

Concepts, Level of evidence,Type of Publications, limits by year, population, setting

What

Concepts

Context, purpose, review type: A psychology graduate student reviews studies to propose a new research study. Subjects: Education, Game Design, Psychology, Artificial Intelligence.

Section 2

Where

Objective 2 Indicate the sources for literature or evidence refer to repositories, collections, databases or datasets (corpus).

Where? Data sources

Thesaurus browsing /Subject search

Curated data - indexes, Databases

Lexical search

Knowledge Graphs, Vector Embeddings

Semantic search

Not structured data

Hybrid, Neural, Agentic

LLM training data

Where: Academic Databases

Check your library Databases!

Databases A to Z list

Select a database Tool

Corpus or Datasets

Ai2 PaperFinder

Comparison of AI-based academic search tools

Allen Institute

Section 2

Where

Objective 2 Be able to clarify what you are searching for? Concepts, research reports, data, information or literature.

What: Research Evidence Retrieval

Research Evidence ≠ Information

"Prompt engineering" can improve the probability of generation of relevant information, not evidence retrieval.

Raw LLMs are not intended to retrieve "evidence," they generate text.

What: Quality and accuracy of evidence retrieval

Recall – a measure of the quantity of relevant results a search returns.Precision – a measure of the quality of relevant results

Level of evidence - the relevant value of the evidence.

Quality

The evidence = dataset quality

What? Research Evidence level by the research design

Systematic Review/ Meta analysis

Meta-Analysis

Experiment, Control intervention

Experimental Studies

Descriptive, causal, comparative, correaltional

Quantitative Studies

Meta-synthesis

Case study, ethnography, phenomenology

Qualitative Studies

What: Inclusion/Exclusion Criteria

Some hybrid and semantic search engines allow basic filters. Check Advanced search. Prompts should include filters as well

What: Filters by type of Research

Section 3

How

Objective 3: Selecting appropriate search techniques and tools, including AI-driven ones

How: Types of Search

Info

Best Match/ ML

Network or Citation analysis

Lexical Search

Pearl growing

Citation Chaining

How? Algorithms &Search Techniques

Citation Chaining

Semantic/ NLP

Probabilistic

Hybrid

Thesaurus browsing

Language models

Exact match

Boolean

Vector search

https://libguides.kennesaw.edu/search_tech

Lexical Search

Boolean/Keywords

Thesaurus terms

Syntax & Filters

Indexes based on the listing of terms/concepts relationships, variants and synonyms,

Most of databases and traditional search engines (keywords from a title, abstract)

To construct search query and limit search results

Lexical Search: ThesaurusBrowsing

https://libguides.kennesaw.edu/search_tech/concept

List of databases with a thesaurus and tutorials how to do this search by a database

AI empowered search

Semantic

Conversational

Hybrid

Natural Language Processing (NLP) understand the meaning of a query

Asking questions, communicating (prompts) with LLM or Research Assistant

Search supported by LLM, RAG, knowledge graphs, embeddings, AI agents

AI-based Search Engines

Semantic Scholar, Dimensions, Google Scholar

Semantic

General Search engines + LLM

Microsoft Copilot, Google Gemini, Perplexity

Academic Search + ML/DL

Scite.ai, Consensus, Ellicit, Ai2 PaperFinder

Hybrid Search (Lexical + AI based)

Summary

Filters

Scite.ai

Related Theories & Methods

References & Citations

Pearl- Growing

Journals

Concepts & Subjects

Source or Database

Search for similar documents based on the ”pearl” metadata. From the initial pearl or a seed (article), grow (collect) other pearls.

Pearl (An article)

Pearl (A relevant article)

Research Rabbit - pearl growing /citation chaining

Citation Chaining

Indexes such as Web of Science, Scopus, or a search engine like Google Scholar allow you to see the number of citations and find who cited the publication to find seminal works and the newest research.

Click in Cited by to see all works that cited the original publication

Learn more about citation chaining https://libguides.kennesaw.edu/search_tech/chain

Citation Chaining using AI tools

When should I stop searching?

When to Stop to Search?

Protocol is a systematic search with inclusion and exclusion criteria

Systematic Search - follow the protocol

Include filters, indicate limits to the search

Best match search algorithm," the most prominent and widely used algorithm is Okapi BM25, or simply BM25. It is a ranking function used by search engines to estimate the relevance of documents to a given search query. Modern, sophisticated systems increasingly use machine learning and deep learning to augment or replace traditional ranking. These models can learn more nuanced and context-aware patterns by analyzing user behavior, like which results get clicked.

When we're told a story, it touches us. It can even move us, making us remember the stories up to 20 times more than any other content we can consume.

Start date:

Write an awesome headline

00 / 00 / 20XX

End date:

00 / 00 / 20XX

Motivation

Actions

Date

Multimedia content is essential in a presentation, to leave everyone amazed. Furthermore, it will help you summarize the content and entertain the entire class.

What haveI learned?

1. Improve communication on any topic

00 / 00 / 20XX

00 / 00 / 20XX

2. Make a 'match' with your audience

00 / 00 / 20XX

3. And it makes them part of the message

4. It has a suitable color for its theme

00 / 00 / 20XX

00 / 00 / 20XX

5. Represent data with graphics

00 / 00 / 20XX

6. Use timelines to tell stories

When referring to a "best match search algorithm," the most prominent and widely used algorithm is Okapi BM25, or simply BM25. It is a ranking function used by search engines to estimate the relevance of documents to a search query. Machine learning and deep learning support semantic search, contextualizing, augmenting or replace traditional ranking with vector search/embeddings and hybrid search

When we're told a story, it touches us. It can even move us, making us remember the stories up to 20 times more than any other content we can consume.

Start date:

Here you can put a highlighted title

00 / 00 / 20XX

End date:

00 / 00 / 20XX

Motivation

Actions

Date

Multimedia content is essential in a presentation, to leave everyone amazed. Furthermore, it will help you summarize the content and entertain the entire class.

What haveI learned?

1. Improve communication on any topic

00 / 00 / 20XX

00 / 00 / 20XX

2. Make a 'match' with your audience

00 / 00 / 20XX

3. And it makes them part of the message

4. It has a suitable color for its theme

00 / 00 / 20XX

00 / 00 / 20XX

5. Represent data with graphics

00 / 00 / 20XX

6. Use timelines to tell stories

When we're told a story, it touches us. It can even move us, making us remember the stories up to 20 times more than any other content we can consume.

Start date:

A great title

00 / 00 / 20XX

End date:

00 / 00 / 20XX

Motivation

Actions

Date

Multimedia content is essential in a presentation, to leave everyone amazed. Furthermore, it will help you summarize the content and entertain the entire class.

What haveI learned?

1. Improve communication on any topic

00 / 00 / 20XX

00 / 00 / 20XX

2. Make a 'match' with your audience

00 / 00 / 20XX

3. And it makes them part of the message

4. It has a suitable color for its theme

00 / 00 / 20XX

00 / 00 / 20XX

5. Represent data with graphics

00 / 00 / 20XX

6. Use timelines to tell stories

When we're told a story, it touches us. It can even move us, making us remember the stories up to 20 times more than any other content we can consume.

Put here an awsome title, something that catches the class's attention

Start date:

00 / 00 / 20XX

End date:

00 / 00 / 20XX

Motivation

Actions

Date

Multimedia content is essential in a presentation, to leave everyone amazed. Furthermore, it will help you summarize the content and entertain the entire class.

What haveI learned?

1. Improve communication on any topic

00 / 00 / 20XX

00 / 00 / 20XX

2. Make a 'match' with your audience

00 / 00 / 20XX

3. And it makes them part of the message

4. It has a suitable color for its theme

00 / 00 / 20XX

00 / 00 / 20XX

5. Represent data with graphics

00 / 00 / 20XX

6. Use timelines to tell stories