Teaching with AI Conference Poster

2024 Teaching and Learning with AI Conference Poster

Project Overview

94%

of instructors are interested in integrating AI [slightly to very]

Teaching with AI

Despite its potential benefits, faculty often face significant barriers in understanding and effectively utilizing AI technologies in their teaching (Holmes & Porayska-Pomsta, 2023). We aim to address the gap between the potential of AI and educators’ readiness through the design, development, and evaluation of an Open Educational Resource (OER) on teaching with AI

More info

A Cross-Institutional Research Study for an Open Educational Resource (OER)

More info

Key Content Needs

+ Author List

PRESENTER: FANG YI, JOSH THORUD

Research Methods and Design

Research Questions

Spring 25

Spring 24

Summer 24

Fall 24

Ongoing

What are the key content areas and features that instructors prioritize in an OER resource regarding teaching with AI?

Content Review and User Feedback

Needs Assessment Survey

Content Development

Documenting challenges

Content Revision

More info

Desired OER Format

Needs Assessment Survey Result

Key Takeaways

• Limited pedagogical understanding, technical skills, and lack of training and resources pose obstacles to integrating AI into the classroom.

• Interactivity, adaptability, and community crowdsourcing are important, making OER an ideal format for developing AI resources.
• AI literacy, AI pedagogies, and AI ethics are the most important topics to include in the AI resources.

AI Integration Goals

AI Integration Challenges and Concerns

Utilization of AI Resources

Learn More! Get Involved!

Desired OER Format

• Strongest Preference:
• Interactive online modules
• Adaptability (openly-licensed, modifiable, reusable)
• Community: discussion threads and building a learning community for sharing experiences and resources
• Moderate Preference:
• Videos
• Structured courses (online or in-person)
• Text
• Low Preference:
• Podcasts
• Other

Research Design

Phase 5 – Document Challenges

The research team will capture the evolving challenges, benefits, decisions, and changes made during the creation process with AI integration. We will analyze the documentation at the end for emerging themes and insights.

Research Design

Phase 3 – Content Review and User Feedback

We plan to recruit 30-200 instructors with limited knowledge of AI and from diverse disciplines to review the initial version of the OER resource developed by the researchers. After instructors have reviewed the OER resource, we will conduct follow-up surveys (30 min per survey) to collect quantitative data on their experiences, challenges, perceived benefits, and suggestions for improvement. The analysis of these survey results will be utilized to enhance the initial version of the OER resource. This analysis will also help identify specific individuals to participate in interviews and determine additional data that should be gathered during these interviews. The interview questions will be formulated based on the survey analysis and the data gaps we have identified

N = 113

Q:Have you used any resources on teaching with AI developed by your own or other institutions?

Our results showed that 78% instructors have not used any institutional AI resources. Therefore, it's important to provide them with useful materials to help them develop AI literacy and pedagogies, enabling them to integrate AI into their teaching.

Key Takeaway:

Instructors Are Interested!

The results reveal substantial enthusiasm among respondents, with only 6% indicating no interest, 16% displaying slight interest, 24% expressing moderate interest, 21% showing great interest, and 33% demonstrating high interest in leveraging AI for educational enhancement.

Author List

Presenters (present at conference):

• Fang Yi, Assistant Director, Learning Design and Technology, UVA , fy5g@virginia.edu

• Josh Thorud, Multimedia Teaching and Learning Librarian, UVA, jdt8zh@virginia.edu

Collaborators (not present):

• Bethany Mickel, Instructional Design and OER Librarian, UVA.

• Sevinj Iskandarova, Assistant Professor of Business Administration, Bridgewater College

• Katya Koubek, Professor in the Educational Foundations and Exceptionalities Department, JMU

• Bisi Velayudhan, Associate Professor of Biology, JMU

• Jessica Taggart, Assistant Director, Center for Teaching Excellence , UVA

• Tim Ball, Professor in the School of Communication Studies, JMU

• Jaira Ferreira de Vasconcellos, Assistant Professor of Biology,

• Jess Marquardt, Lecturer, Biology, JMU

• Breana Bayraktar, Blended Learning/Hybrid Pedagogy Specialist, Stearns Center for Teaching and Learning , GMU
• Dayna Henry, Interim Assistant Director, Center for Faculty Innovation, JMU

Research Design

Phase I - Needs Assessment Survey

Before developing the OER, needs assessment surveys (15 - 30 min per survey) was administered to instructors to identify their current understanding of AI, their perceived needs, and what they would find most beneficial in an OER resource. This will ensure the content is relevant and tailored to their needs.

Research Design

This multi-phase mixed method study aims to design, develop, and evaluate an adaptable OER resource, using AI as a development tool, for instructors to aid in understanding the principles of AI, methodologies for incorporating AI into teaching, ethical considerations surrounding AI in education, examples of AI-driven assignments, recommended AI class policies, and an overview of current AI tools available for teaching. This study has been approved by George Mason’s Institutional Review Board (IRB #2035233-1). Click on the info button below to learn more about each phase of study.

Research Design

Phase 4 – OER Content Revision

Based on feedback from our survey and interview analyses, we will make revisions and develop new content to address the identified gaps.

Q: Please indicate the extent to which each topic is important to you

Key Content Needs:

• Highest interest
• AI Literacy
• AI Pedagogy/Assignments
• AI Ethics

• Moderate interest
• AI Tools (see Tools)

• AI Policy and Best Practices
• Low interest
• AI Fundamental Concepts

Q: What types of AI-driven assignments or activities you would be interested in learning about

Detailed Breakdown

Needs Assessment Methods

145 instructors of record who taught within the past two years in Virginia institutions of higher education completed our survey. Although participants were allowed to skip questions, which resulted in varying sample sizes for different questions. Researchers followed a rigorous process of examining the responses to uncover common themes, patterns, and meanings that emerged from the participants' narratives. This process involved coding and categorizing the qualitative data to discern recurring themes and insights.

Desired OER Format

• Strongest Preference:
• Interactive online modules
• Adaptability (openly-licensed, modifiable, reusable)
• Community: discussion threads and building a learning community for sharing experiences and resources
• Moderate Preference:
• Videos
• Structured courses (online or in-person)
• Text
• Low Preference:
• Podcasts
• Other

Project Overview

This resource, tailored for instructors from all academic fields, covers AI principles, methodologies for teaching with AI, ethical considerations, AI-driven assignments, class policies, and current AI tools. The study explores instructors' priorities in OER content, examines the impact of AI in resource development, and assesses the effectiveness of the adaptable OER in boosting instructors' confidence and competence in AI integration. This is a cross-institutional and cross-disciplinary Scholarship of Teaching and Learning project funded by the State Council of Higher Education for Virginia and University of Virginia Center for Teaching Excellence. The team includes faculty, staff, and students from the University of Virginia, James Madison University, Bridgewater College, and George Mason University.

27%

Course Design and Lesson Plan

Q: What do you hope to achieve by integrating AI into your teaching?

17%

Enhancing Teaching Efficiency

Assessment and Evaluation

9%

Professional Development and Staying Current

Goals of Integrating AI into Teaching

The needs assessment survey results categorized educators' qualitative responses into two main groups:

• Enhancing Teaching Efficiency
• Enhancing Learning Efficiency

13%

Career Preparation

12%

Ethical and Responsbile AI Usage

Enhacing Learning Efficiency

11%

Student Engagement

10%

Critical Thinking and Creativity

Enhancing Learning Efficiency:

• Career Readiness (13%): Educators seek to prepare students for future professional settings by teaching critical and appropriate AI usage.
• AI Ethical Considerations and Responsible Utilization (12%): Emphasis is on teaching ethical AI use, fostering discussions on moral responsibilities, and guiding students to develop critical thinking skills.
• Student Engagement (11%): AI is seen as a tool to enhance learning outcomes and engagement by improving assignment solutions and digital literacy.
• Enhancing Creativity and Critical Thinking (10%): Educators aim to use AI to foster critical thinking and creativity, and enhance learning by providing tools to develop skills in various areas of interest.

Enhancing Teaching Efficiency:

• Course Design and Lesson Plans (27%): Educators aim to improve instruction effectiveness and efficiency, creating diverse prompts, tasks, and quizzes to enrich teaching materials, improving accessibility of teaching materials.
• Assessment and Evaluation (17%): Educators hope to reduce grading workload, assist in evaluation, generate practice problems, and provide targeted feedback for students' assignments.
• Professional Development and Staying Current (9%): Educators emphasize the need to stay updated with AI technology to engage tech-savvy students and guide them to use AI ethically and responsibily.

Q: What challenges or concerns do you foresee, or have you encountered, in incorporating AI into your teaching?

19%

Lack of AI Knowledge and Skills

Lack of Knowledge, Resources and Support

Lack of Pedagogical Support

10%

Lack of Support

8%

Challenges and Concerns

The needs assessment result identified two main challenges educators face in integrating AI into teaching

• Lack of Academic Integrity
• Lack of Knowledge, Resources and Support

23%

Ethical Concerns

17%

Lack of Critical Thinking

Lack of Academic Integrity

14%

IInaccurate Information and Data

9%

Efficiency and Effectiveness

Lack of Academic Integrity:

• Ethical Concerns (23%): Issues with plagiarism, academic dishonesty, and unauthorized use of AI.
• Lack of Critical Thinking (17%): Students may rely on AI, leading to reduced critical thinking and originality.
• Information Inaccuracy (14%): AI can provide incorrect or biased information, complicating grading and learning.
• Efficiency and Effectiveness (9%): Students might use AI as a shortcut, compromising the learning process.

Lack of Knowledge, Resources and Support:

• Lack of AI Knowlege and Skills (19%): Both students and faculty may lack the necessary knowledge and skills to use AI effectively.
• Lack of Pedagogical Support (10%): Lack of pedagogical support to understand the benefits of AI integration and effective approaches.
• Lack of Support (8%): Insufficient institutional support, training, and resources for integrating AI.

Learn More! Get Involved!

Share your contact information for more information, to stay updated, or to contribute to the development of our OER!

QR Code Above: Share your contact information (Google Form)QR Code Below: Link to this poster at.virginia.edu/Poster_TeachAI2024

Research Design

Phase II – Content Development

We are in the process of designing and developing the OER content based on the results from our needs assessment survey. We also reviewed exsiting OER content and will focus on adapting relevant content to avoid reinventing the wheel.

Research Questions:

The study seeks to answer the research questions below. However, this poster only focuses on presenting our results regarding the first question.

• What are the key content areas and features that instructors prioritize in an OER resource regarding teaching with AI?

• How does the integration of AI in the development process affect the creation and adaptability of the OER resource for teaching with AI?

• How effective is the adaptable OER resource in enhancing instructors' confidence and competence in integrating AI into their teaching?