Evaluation Multimedia Presentation

Evaluation Model

Terra D. McCrae-GreeneTECH 8533, Dr. CorneliousMississippi State University

START

Index

Exploring several different ISD and evaluation models with reflections.

Kirkactrick's 4 Levels of Training Eval

ADDIE Model

Dick & Carey's Systematic Approach

Kemp's ISD

Gange's Nine Events of Instruction

Merrill's First Principles of Instruction

Bloom's Taxonomy

Revised Bloom's Taxonomy

Bloom's Taxonomy was originally coined in 1956 as a framework for categorizing educational goals: originally they were: Knowledge, Comprehension, Application, Analysis, Synthesis, & Evaluation. In 2001, it was revised using action words (gerunds) to describe the "cognitive processes by which thinkers encounter and work with knowledge, (Armstrong, 2010).

Etienne du Plessis (2021) used the above image to show how this framework could be used by young professionals in any industry. Du Plessis (2021) suggests that skills-based occupations could benefit from the framework's steps as a troubleshooting guide to identify where improvements are needed. The framework also lends itself to a more thorough evaluation while using a higher level of thinking.

+ info

+ info

+ info

+ info

Kirkpatrick's 4 Levels of Training Evaluation

Level 1

Level 2

Reaction

Learning

Behavior

Level 3

Results

Level 4

The degree to which participants find the training enjoyable, engaging, and relevant to their jobs (Kirkpatrick Partners, 2024).

At the learning level, participants acquire the intended knowledge, skills, and commitment based on the training, (Kirkpatrick Partners, 2024).

This level is based on the degree to which participants apply the new skills and information learned during training when they are back on the job, (Kirkpatrick Partners, 2024).

This level shows the degree to which targeted outcomes are achieved post-training, with support, accountability, and sustainability, (Kirkpatrick Partners, 2024).

According to MindTools (2024), Kirkpatrick's model, used for more than 60 years, has drawbacks in Levels 3 and 4, "which arguably yield the most useful information, are time-consuming, resource-intensive, and expensive to implement," (MTCT, 2024).

ADDIE Model

Analysis, Design, Development, Implementation, & Evaluation

ADDIE represents five phases of instructional design and has been used in developing training and instruction since the 1970s. Designed for the U.S. Army and then adapted by the U.S. Armed Forces, the original ADDIE Model was considered linear, starting with analysis and ending with an evaluation, (Bouchrika, 2024).The image selected demonstrates the five phases, but also identifies corresponding action words with a Bloom's Taxonomy feel. What I like about the model is that it serves as the basis for many other design model variations, (Culatta, 2024). What I do not like is that it seems to exclude space for creativity.

+ info

Kemp's ISD Model

Nine elements of the Kemp Model

1. Identify instructional problems and specify goals.2. Examine learner characteristics.3. Analyze task components for the stated goals and purposes.4. Specify the instructional objectives.5. Sequence content for logical learning.6. Design instructional strategies for mastery.7. Plan the instructional message and develop the instructions8. Develop evaluation instruments to assess objectives9. Select resources to support instruction and learning activities.

The image selected is an integration of Kemp's model with ADDIE. that was discovered in my search for images. What I like about this model is that it is learner-centered and provides flexibility to be used on an ongoing basis to provide formative and summative results, (Escobedo, et al., 2014). There is nothing about this model that I dislike. Student-centered or participant-centered training drives relevant instruction.

A closer look at this model demonstrates how Kemp aligns with the ADDIE Model, and how the components mesh and/or overlap, (Kosina, 2014).

+ info

Dick & Carey Model

The Ten Components

Based on ADDIE principles

• Assess needs to identify goals
• Conduct instructional analysis
• Analyze the learners and contexts
• Write performance objectives
• Develop assessment instruments
• Develop instructional strategy
• Develop and select instructional materials
• Design & conduct the formative evaluation of instruction
• Revise instruction
• Conduct summative evaluation,

The Dick & Carey (DC) model appears to leave no room for deviations. According to Lozada Rivera (2022), it is rectilinear, suggesting that it must be followed in a sequential order. Purporting that the model is best for novice instructional designers with its rigidity and step-by-step order, Lozada Rivera (2022) also suggests that following the DC model closely restricts any creativity and adaptability for collaborators when designing as a team.

I like this model, and if I could modify it, I would make it more flexible, possibly in a helix design, allowing each component to connect throughout, because all of them may not be necessary at times, depending on the context of instruction.

Gagne's Nine Events of Instruction

An Adaptive Model

• Gain learner's attention
• Inform learners of the objective
• Stimulate recall of prior learning
• Present the [new] content
• Provide learning guidance
• Elicit performance/practice
• Provide feedback
• Assess performance/practice
• Enhance retention and transfer

Dr. Kurt (2021) suggests the following to get students' attention:

• Ask an unexpected question
• Present an interesting point of trivia or a problem
• Use a loud & unexpected tone, other audio or eye-catching stimuli
• Establish a peer-to-peer exercise/activity

For retention & transfer:

• Find ways to link concepts together versus isolating them;
• Give assignments requiring students to think about lessons in dynamic ways;
• Be transparent about learning outcomes and revisit them, (Kurt, 2021).

A few ways to elicit performance are to provide multiple exposures to the information in various ways and provide ways to demonstrate learning using "tests, quizzes, presentations, essays, group projects, & [task-based] lab exercises," (Kurt, 2021).

Dr. Kurt (2021) suggests the following for accessing prior learning:

• Quickly review past lessons;
• Ask students to explain what they recall;
• Use engaging audiovisual presentations of material;
• Incorporate elements of prior learning into the new lessons.

Merrill's Principles of Instruction

Merrill's Principles of Instruction features five principles of instruction. 1. Beginning with a relevant real-world task or problem-centered learning; 2. Activation: involves capturing the learner’s attention, establishing relevance, and building their confidence. 3. Demonstration: involves providing a preview of the skills or knowledge learners will be experiencing.4. Application: involves providing learners with opportunities to apply their new knowledge and skills to real-world problems. 5. Integration: involves helping learners connect the new knowledge and skills to their existing knowledge base, (Saha, 2024),

According to Merrill, 'learning is most effective when learners are actively engaged in the learning process, and when the learning experience is designed to facilitate their understanding and retention of new information'," (Saha, 2024).

What I like most about Merrill's Principles of Instruction is the task-based and problem-based nature. What I greatly dislike is the potential for learners who may need more guidance or other accommodations to not be able to benefit from this method.

This interactive presentation was designed using Genial.ly

I hope that you consider one of these evaluation models in your teaching practice. The overall goal is for students to have the opportunity to put 21st Century Skills to work through their learning processes and to retain and be able to transfer their learning to real-world problems and find real-world solutions.

Thank you for Visiting

Please continue to explore the resources.

Instructional Systems and Workforce DevelopmentMississippi State University

Terra D. McCrae-Greene

REFERENCES

du Plessis, E. (2021). Bloom's taxonomy - 6 levels of effective thinking. [Blog]. Spencer Tom. Retrieved from: https://www.spencertom.com/2021/02/07/blooms-taxonomy-6-levels-of-effective-thinking/.

Armstrong, P. (2010). Bloom’s Taxonomy. Vanderbilt University Center for Teaching. Retrieved from https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/.

Culatta, R. (2024). ADDIE Model. Instructional Design. InstructionalDesign.org. Retrieved from: https://www.instructionaldesign.org/models/addie/

Kirkpatrick, J. & Kayser Kirkpatrick, W. (2024). The Kirkpatrick Model. [Website]. Kirkpatrick Partners, LLC. Retrieved from: https://www.kirkpatrickpartners.com/the-kirkpatrick-model/

Kosina, K. (2014). The Morrison, Ross, and Kemp Model. [Image]. PBWorks. Retrieved from: http://edtc6321group3fall14.pbworks.com/ w/page/85214398/Kemp. (2024).

Mind Tools Content Team. (2024). Kirkpatrick's Model: Four Levels of Training Evaluation. [Article]. Mind Tools. Retrieved from: https://www.mindtools.com/ak1yhhs/kirkpatricks-four-level-training-evaluation-model.

Legault, N. (2011). Infographic: Kirkpatrick's 4 Levels of Evaluation. [Infographic]. Retrieved from: https://nlegault.ca/2011/09/01/ infographic-understanding-kirkpatricks-4-levels-of-evaluation/

Escobedo, J., Herrera, R., Ing, M., Kosina, K.,San Miguel, J., & Tucker, M. (2014). Instructional Design Models: Comparing ADDIE, Kemp, and IPISD. [Workspace]. PBWorks. Retrieved from: http://edtc6321group3fall14.pbworks.com/w/page/85214398/Kemp.

Kurt, S. (2021). Teaching & Learning: Gagne's Nine Events of Instruction. [Article] Educational Technology. Retrieved from: https://educationaltechnology.net/gagnes-nine-events-of-instruction/

Northern Illinois University Center for Innovative Teaching and Learning. (2020). Gagné’s nine events of instruction. In Instructional guide for university faculty and teaching assistants. Retrieved from https://www.niu.edu/citl/resources/guides/instructional-guide

Saha, S. (2024). Merrill's Principles of Instruction: A Guide to Effective Teaching. [Blog]. MBA Simplified. Simpli MBA. Retrieved from: https://www.simplimba.com/merrills-principles-of-instruction/.

Lozada Rivera, V. (2022). Morrison-Ross-Kemp vs. Dick & Carey Model. Writings [Blog]. Retrieved from: https://victorlozadarivera.com/morrison-ross-kemp-vs-dick-and-carey-model/.

The framework is versatile and can guide the evaluation of any program. I would not revise it. What I like about the model is its usefulness in creating goals. Teachers in my building have used this framework to write lesson objectives and to clarify the goals of units of learning along with their pacing guides. EX: SWBAT select (understanding) evidence from the text to support (evaluating) their reason for agreeing with or disagreeing with the author's perspective.

When to use Bloom's

Although designed for educational purposes, this framework can be used in any industry and is especially beneficial for program evaluation.

The above image was selected, as it shows the interconnectivity of the 9 events and although placed in numerical order, are all important components of this instructional model, which can also be found using the Educational Technology link above.

What I love about this model is its student-centered approach to teaching and learning. I would not modify it, but can see how beneficial it is to learning and retention of that learning through guidance, assessment, and feedback. Akin to the Universal Design for Learning (UDL), Gagne suggests that nine events of instruction lead to learning for all students.

A student-centered approach:

Gagne's 9 Events of Instruction

Center for Innovative Teaching and Learning

• May be too rigid for learners needing more flexibility
• Learners with special needs may not be reached
• May not be suitable for all contexts and with all learners

Advantages and Disadvantages

Merrill's 5 Principles

Merrill's 5 Principles - SimpliMBA

The most obvious advantage is the student- and task-centered basis of this model. It is flexible enough to facilitate in-person and virtual teaching and learning situations. It also considers learners individually and in a group setting.

From what I have read, this model, although only containing four levels of evaluation, not only has the potential to be time-consuming but is an expensive route to go. I have read where newer trainers take the model in reverse, beginning with the results level (MTCT, 2024).

Kirkpatrick's 4 Levels of Evaluation

Mind Tools - Kirkpatrick's Model

The infographic was designed by Nicole Legault (2011) and appeared to provide a visual representation of this model. I neither like, nor dislike Kirkpatrick's model, but believe I understand why it may have been designed at the time. Several revisions have already been made to this model. you can find out more at the link below, (MTCT, 2024).