Explaining AI Presentation

Explaining AI

Thanks

AI Conversation ToolBox

Cheating

Why Not Detectors?

Introductions

index

Session Intentions

• Explain how AI detectors operate and provide a robust WHY they don't work for academic integrity.

• Share relevant research and a teacher toolbox for academic integrity in the age of AI.

Allison Keefe

Instructional CoachEnglish, Music, Art

Emily Sattler

Instructional Coach Social Studies, World Lang

Your Presenters!

Share your thoughts.

How do you think AI Detectors Work?

Finish the phrase...

Burstiness

PErplexity

What Detectors Measure

High

Medium

Low

One way that AI detectors operate is by analyzing the uniqueness of each sentence compared to what they expect to find based on their training—what data scientists call perplexity—meaning that the more probable any word is to come after the word before it, the less novel the sentence is, and the more probable it is to be AI-generated, as LLMs work via probability (Edwards 2023).

PerplexIty

Center for Integrated Professional Development at Illinois State University

A problem with using perplexity as a measure of whether something was written by AI or not, though, is that we use common sentences all the time—that is what makes them common. This is also very true of academic writing, which uses formulae and common phrases consistently both out of necessity and for the sake of transparent communication. This applies to both novice and expert writers, but writers who are less confident or new to a particular genre may lean more into the language their teachers use or language from examples they have found, which makes them more likely to be flagged.

Burstiness is a measure of the consistency in sentence structure and length in a text. Students, specially EL’s and those writing below grade level, are much more likely to have low burstiness and low perplexity. As a result, it is our most vulnerable students who risk being targeted by AI Detectors. This becomes even more true when we consider how widely available tools like grammarly and microsoft "suggestions" are. Students that may be using tools like this support are also more likely to be flagged.

Burstiness

Student fights AI cheating allegations for using Grammarly

Parents sue school in Massachusetts after son punished for using AI on paper

+info

False Positives

A study of submissions to a peer-reviewed medical journal by Cooperman and Brandão showed commercially available AI detectors correctly identifying AI-generated content approximately 63% of the time with false positive rates of 24.5% to 25%.

Cheating isn't New

According to new research from Stanford University, about 60 to 70 percent of high school students surveyed in the fall of 2023 have engaged in cheating behavior — the same number prior to the debut of ChatGPT in the fall of 2022.

Test questions

Test prompts with AI

Real Life Writing

Is it Plagiarism?

AI cheating?

Build critical thinking

Reframe test anxiety

Social Contract

Analyze

Reflection

Share what's working on this Padlet!

Toolbox

Allison Keefe

Index

Click here for bookings!

Emily Sattler

Click here for bookings!

Survey

Please fill out this short survey!

Frontline

Click here to get your Act 48 hours!

Thanks!

Build in a reflection element to writing pieces. This can be written or a video reflection, right in edio, or using a tool like Padlet. Reflections aid personal growth and encourage critical thinking. This is hard for learners to use AI on since it's very personal to what they learned while writing.

AI detection software doesn’t work. Do this instead.

Personal Reflection

Edit your test questions to make them better!

Create better questions

• Use hotspots
• Create questions that are relevant
• Make them more personal
• Focus on process over product

Instead of having learners write their own essays, generate your own essay using CoPilot using your prompt. Then, have your learners analyze it and make it better.

Why Make Students Write in the Age of AI?

Analyze CoPilot Essays

• Are all the facts correct?
• Did it cite correct information?
• Does it answer the question?