USMAPS GenAI Workshop

USMAPS

Generative AI Workshop

Jenn ChessCurriculum and Instructional Design LibrarianCompany Librarian - A1, B1, C1jennifer.chess@westpoint.edu Lori Mullooly Curriculum and Instructional Design Librarian Company Librarian - G3, H3, I3 lori.mullooly@westpoint.edu Erin Tolman Instructional Design LibrarianCompany Librarian - D4, E4, F4erin.tolman@westpoint.edu

Generative AI Essentials: Responsible Use at West Point

• Poll
• GenAI Basics: Overview
• Key capabilities and limitations
• Common GenAI tools
• Ethical considerations

• Hallucinations
• Getting the most from your GenAI Tool: Prompt Engineering
• Hands-on Activity
• Q&A

Agenda

Scan QR code or go to mentimeter.com and use this code: 9794 2441

Mentimeter Poll

BLUF: ALL output is based on probability patterns it has learned.

• Type of AI that generates new content rather than just analyzing existing content

• It learns patterns from training data and makes predictions based on probability

• Does not have semantic knowledge on what it is generating

Generative AI

Probabilistic vs. Deterministic

BLUF: Tools are NOT designed to be factual, they are designed to be generative.

This probabilistic nature is: ✅ Beneficial for creative tasks and natural conversation ✅ Good for generating diverse suggestions ✅ Great for: Content generation, summarization, paraphrasing, recognizing themes, brainstorming, customizing content, contextual conversations, providing insights, explaining concepts, providing examples, showing alternatives ❌ Less ideal for tasks requiring exact consistency ❌ May need control through specific instructions ❌ Not great for: Exact calculations, factual consistency, historical dates, legal citations, real time data, current events, ethical decisions, medical diagnosis, emergency response, scientific breakthroughs

This means they are great at certain things, but not everything...

Big 3 Frontier Models: they are from different companies, using different different neural networks and with different personalities and abilities. The paid versions are often substantially better and smarter. Anthropic's Claude 3.5: Known for detailed analysis and reasoning Strong at handling long documents. Has real-time web access. Open AI's Chat GPT-4o: Most widely used consumer AI Good for writing, analysis, coding. Has real-time web access. Google's Gemini: Good at both text and image analysis Strong mathematical and coding capabilities. Gemini 2.0 has real-time web access.

Common GenAI Tools

• model based on GPT-4 architecture
• has real-time web access
• https://copilot.microsoft.com/
• Enterprise-level security through our institution's license, with enterprise data protection (Your data isn’t used to train foundation models)

• Microsoft 365 Integration

CoPilot

Ethical Considerations

Intellectual Property Violation

During training phaseDuring use with various tools

Bias

Flawed datasetsLimitations of GenAI models

Privacy

No way to know if the model is following the rules surrounding collection and use of information

Lack of Transparency

In training data setsIn how the models work

Why hallucinations?

Nonsensical

Visual Distortions

Logical

Fabricated Citations

Factual

Generative AI Hallucinations

Specific Features or Details Instructions References

Target Audience Assignment Intent DetailsAssignment Goals

Example

Example

StyleToneVoice

Example

Example

Point of ViewExpertisePersonal Experience

Prompt Engineering for Success

DETAILS Develop assignment instructions & a grading rubric

Determine during which lesson the assignment should be assigned. JUSTIFY!

Generate a list of assignment ideas that use GenAI

Step 3

Step 2

Step 1

ACTIVITY

GARBAGE IN = GARBAGE OUT

• Prompt Engineering is Iterative. Tell it what you liked and what you didn’t. Be very clear and explicit.
• GenAI has a tough time with multifaceted prompts. Break it up into steps.
• Don’t assume it knows anything.
• To be a responsible GenAI user, you must be a CRITICAL GenAI user.

Key Takeaways

Questions?

Specific Features or Details | Instructions | References

Please generate a list of 5 ideas for an assignment that involves using GenerativeAI. This assignment should take 30-60 minutes to complete. It should be interesting and offer an engaging pedagogical approach. You can refer to the syllabus and the textbook https://open.umn.edu/opentextbooks/textbooks/588. They must also hand in a copy of their GenAI transcripts.

Generative AI can provide very authoritative sounding outputs that contain logical errors such as:

• Contradictory statements - Making claims that directly contradict each other within the same response
• Circular reasoning - Using the conclusion as part of the premise in an argument
• False causality - Incorrectly attributing cause-and-effect relationships between unrelated events
• Category errors - Applying properties to things in inappropriate categories (e.g., discussing the "color" of a mathematical concept)

Style | Tone | Voice

I am an expert in my field. The tone should be formal and academic. However, the language should be appropriate so that a B average high school graduate can understand.

Point of View | Expertise | Personal Experience

I am an English instructor who teaches a foundational composition writing English course to undergraduate freshmen. I am an expert in my field and have published on Faulkner’s narrative techniques and stream of consciousness. I pride myself on developing creative assignments for my students that engage them in ways that lead to further curiosity, critical thinking, and new ideas.

Target Audience | Assignment Intent Details | Assignment Goals

The target audience for this assignment is an undergraduate freshmen who has only basic writing, reading, and comprehension skills. In order to get a 100% on this assignment, the student must exhibit strong grammar skills and must write a reflection on their experience in a way that shows growth and self-awareness. The intent of the assignment should be to engage in using GenerativeAI in a way that strengthens a student's critiquing skills.

STEP 1 Prompt

I am an English instructor who teaches a foundational composition writing English course to undergraduate freshmen. I am an expert in my field and have published on Faulkner’s narrative techniques and stream of consciousness. I am an expert in my field. The tone should be formal and academic. However, the language should be appropriate so that a high school graduate can understand. The target audience for this assignment is an undergraduate freshmen who has only basic writing, reading, and comprehension skills. In order to get a 100% on this assignment, the student must exhibit strong grammar skills and must write a reflection on their experience in a way that shows growth and self-awareness. The intent of the assignment should be to engage in using GenerativeAI in a way that strengthens a student's critiquing skills.Please generate a list of 5 ideas for an assignment that involves using GenerativeAI. This assignment should take 30-60 minutes to complete. It should be interesting and offer an engaging pedagogical approach. You can refer to the syllabus and the textbook https://open.umn.edu/opentextbooks/textbooks/588. They must also hand in a copy of their GenAI transcripts.

GenAI systems can hallucinate because they're trained to predict statistically likely text patterns rather than retrieve verified facts from a knowledge base. These models lack true understanding of the world and instead generate content based on patterns they've observed during training, without an ability to verify information against reality. When faced with uncertainty, rather than admitting knowledge gaps, AI systems often produce plausible-sounding but potentially incorrect information to maintain conversational flow. Additionally, the training data itself may contain inaccuracies, biases, or outdated information that the model reproduces without the ability to critically evaluate its validity.

STEP 3 Prompt

Please write detailed instructions for the assignment. The instructions should be clear and concise and use a basic 8th grade reading level. The instructions should be bulleted, with important details bolded. The instructions should start with the most important details such as when the assignment is due and which reading should be used. Make sure to clearly state that Generative AI should be used, but only in the first part of the assignment and not for the reflection component. Please create a grading rubric for this assignment. It should have 5 criteria, with each criterion being worth 20 points. One of the criteria should be grammar. Please present it as a chart, with the criteria as a column and points across the top in a row with labels Excellent (16-20 pts), Proficient (11-15 pts), Developing (6-10 pts) and Needs Improvement (0-5 pts). [You could ask it if there are recommendations for additional components.]

Image generators can produce visual distortions in its output. Some common distortions are:

• Anatomical distortions - Human figures with too many fingers, misshapen limbs, or facial features in unnatural positions
• Text rendering issues - Garbled, nonsensical text or letters that appear to be writing but are actually unintelligible
• Spatial inconsistencies - Objects that defy physics by floating, merging with other objects, or existing in impossible spatial relationships
• Perspective errors - Buildings, rooms, or objects with physically impossible angles or perspective distortions

Generative AI will provide citations for its anwers...but while they sound legitimate, some of them may be:

• Completely fabricated sources - Inventing non-existent books, articles, or papers with plausible-sounding titles and authors
• Author hallucinations - Attributing real works to incorrect authors or creating fictional authors for real publications
• Date/year distortions - Citing real works but with incorrect publication dates or years
• Journal/publisher fabrications - Referencing non-existent journals or publishing houses that sound legitimate

Generative AI outputs can also be complete nonsense.

• Word salad - Grammatically correct sentences that contain no actual meaning when examined closely
• Semantic drift - Responses that start coherently but gradually lose meaning as they progress

• Pseudo-profound nonsense - Generating deep-sounding but ultimately meaningless philosophical statements
• Technical gibberish - Creating technical-sounding explanations using real terminology in ways that make no sense

Generative AI outputs can simply be incorrect. Some examples of factual hallucinations are:

• Stating the wrong date of a historical event
• Providing an incorrect answer to a math problem
• Creating fictional people and presenting them as real
• Claiming a non-existant geographic landmark is real

STEP 2 Prompt

I really like idea #4. Please review the course schedule [USE WHATEVER TERMINOLOGY YOU USE IN YOUR SYLLABUS] and suggest 5 lessons that this assignment best aligns with. Please justify each suggestion and rank the suggestions from strongest justification to least.