Explaining AI Presentation

Explaining AI

index

Introductions

Why Not Detectors?

Cheating

AI Conversation ToolBox

Thanks

Session Intentions

• Explain how AI detectors operate and provide a robust WHY they don't work for academic integrity.

• Share relevant research and a teacher toolbox for academic integrity in the age of AI.

Your Presenters!

Emily Sattler

Allison Keefe

Instructional Coach Social Studies, World Lang

Instructional CoachEnglish, Music, Art

How do you think AI Detectors Work?

Share your thoughts.

Finish the phrase...

What Detectors Measure

PErplexity

Burstiness

PerplexIty

Low

Medium

High

One way that AI detectors operate is by analyzing the uniqueness of each sentence compared to what they expect to find based on their training—what data scientists call perplexity—meaning that the more probable any word is to come after the word before it, the less novel the sentence is, and the more probable it is to be AI-generated, as LLMs work via probability (Edwards 2023).

A problem with using perplexity as a measure of whether something was written by AI or not, though, is that we use common sentences all the time—that is what makes them common. This is also very true of academic writing, which uses formulae and common phrases consistently both out of necessity and for the sake of transparent communication. This applies to both novice and expert writers, but writers who are less confident or new to a particular genre may lean more into the language their teachers use or language from examples they have found, which makes them more likely to be flagged.

Center for Integrated Professional Development at Illinois State University

Burstiness

Burstiness is a measure of the consistency in sentence structure and length in a text. Students, specially EL’s and those writing below grade level, are much more likely to have low burstiness and low perplexity. As a result, it is our most vulnerable students who risk being targeted by AI Detectors. This becomes even more true when we consider how widely available tools like grammarly and microsoft "suggestions" are. Students that may be using tools like this support are also more likely to be flagged.

False Positives

A study of submissions to a peer-reviewed medical journal by Cooperman and Brandão showed commercially available AI detectors correctly identifying AI-generated content approximately 63% of the time with false positive rates of 24.5% to 25%.

Parents sue school in Massachusetts after son punished for using AI on paper

Student fights AI cheating allegations for using Grammarly

+info

Cheating isn't New

According to new research from Stanford University, about 60 to 70 percent of high school students surveyed in the fall of 2023 have engaged in cheating behavior — the same number prior to the debut of ChatGPT in the fall of 2022.

Toolbox

Reflection

AI cheating?

Analyze

Is it Plagiarism?

Social Contract

Real Life Writing

Reframe test anxiety

Test prompts with AI

Build critical thinking

Test questions

Share what's working on this Padlet!

Thanks!

Frontline

Survey

Emily Sattler

Allison Keefe

Click here to get your Act 48 hours!

Please fill out this short survey!

Click here for bookings!

Click here for bookings!

Index

Personal Reflection

Build in a reflection element to writing pieces. This can be written or a video reflection, right in edio, or using a tool like Padlet. Reflections aid personal growth and encourage critical thinking. This is hard for learners to use AI on since it's very personal to what they learned while writing.

AI detection software doesn’t work. Do this instead.

Create better questions

Edit your test questions to make them better!

• Use hotspots
• Create questions that are relevant
• Make them more personal
• Focus on process over product

Analyze CoPilot Essays

Instead of having learners write their own essays, generate your own essay using CoPilot using your prompt. Then, have your learners analyze it and make it better.

• Are all the facts correct?
• Did it cite correct information?
• Does it answer the question?

Why Make Students Write in the Age of AI?