Stakeholder Feedback JAN CNP Summit

January 2023

Stakeholder Feedback:

CNP Summit Session

This session will provide an overview of CNP's menu survey initiative, the data curation process, and district-wide data points from the 21-22 and the 22-23 menu surveys.

start

Introduction Stakeholder Feedback

Brief intro to our stakeholder feedback goals, vision, and growth.

Goals & Vision

Purpose:

Vision

To incorporate stakeholder feedback mechanisms in our menu development process & deliver meal services that not only reaffirm our commitment to our students' health & wellness, but that are also reflective of our stakeholders' voices.

Short Term Goals

Long Term Goals

🔘 Lay groundwork to strengthen stakeholder feedback mechanisms 🔘 Embed survey data into menu development process 🔘 Establish an open communication line with students, families & staff.

🔘 Improve our meal service🔘Assess performance of current menu items (district-wide)

Stakeholder Feedback Goal

01

02

03

19-20 SY

20-21 SY

Collect stakeholder feedback from 75% of IDEA Campuses

21-22

Collect stakeholder feedback from 100% of campuses at least 2 times by LDOS.

04

22-23

Collect stakeholder feedback from 100% of TX campuses at least 3 times by LDOS.

N/A

Goal progression over time

Goal has evolved over the course of the last 3 years. Every year we strive for our goal to be intentional and compatible with latest campus norms and health protocols.

Met & Exceeded

Met

On track

3 Year Growth

The Spectucular Now!

An overview of survey work trajectory at IDEA CNP

22-23 SY

19-20 SY

20-21 SY

21-22 SY

Strong advocacy to maintain or increase rigor

Minimal advocacy & guidance

COVID Year: Stronger advocacy but with limitations

Increased advocacy with less limitations

Highlights 1. Paper (with limitations) & Online 2. Increased rigor on HKH Checklist 3. Comprehensive Resources still available 4. Communication on survey work & progress in various platforms remains a best practice 5. Encouraged 5x per year (to avoid fatigue) with sampling events. 6. Widespread sharing of surveys remains a best practice

Highlights: 1. Paper 2. Optional 3. Minimal Communication 4. Minimal guidance was available. 5. Encouraged 2x per year (NSLW & NSBW)

Highlights: 1. Online 2. Still Optional 3. Accompanied with more guidance and resources 4. Encouraged 6x per year.

Highlights 1. Paper & Online 2. Part of the HKH Checklist 3. More comprehensive resources made avaialble. 4. More communication about survey work/progress in various platforms (Showbacks, CNP National Call) 5. Encouraged 6x per year and with sampling events. 6. Widespread sharing of surveys (via email with campus staff & on social media)

Types of Surveys

21-22 & 22-23 SYs

Student Survey

General Survey

NSBW & NSLW

Voting Survey

3x a year

3x a year

1x a year

1x a year

Measured HOW MUCH a students liked a featured menu item by asking 1-5 star ratings.STUDENTS ONLY

Measured HOW MUCH students like a menu item by asking for 1-5 star ratings. Included Open-Ended Question.STUDENTS ONLY

Measured IF menu items were liked by students & perceptions of taste, healthfulness & likeliness to recommend meals. Included Open-Ended Question.OPEN TO PUBLIC

Measured student interest in possible new menu item options as well as perceptions of taste & healthfulness. Included Open-Ended Question.STUDENTS ONLY

+ Info

+ Info

+ Info

+ Info

Overview 21-22 SY Data at a Glance

Summary of district-wide data for: 1) Campus Participation; 2) Regional Representation; 3) Grade Level Representation; 4) Measures of Campus Performance; 5) Satisfaction levels with meal service; 6) NSLW & NSBW data; 8) Theme Analysis

Volume Matters

Importance of representative sample sizes

What is a sample size: the number of subjects included in surveys, studies, and/or experiments. It can be calculated using an online sample size calculator that takes into account the following: Ideal sample size for IDEA district-wide is anywhere between 2.23% - 3% of student population.

• Students participating in meal program (60K) = 1,787 responses (2.23%)
• Entire student population (80K students) = 1,801 responses (2.25%)

Margin of Error

Population size

Confidence Level

Survey Events during 21-22 SY

Brief summary of last year's survey event timeline

E5

E6

E1

E2

E3

E4

NSBW 2022

May 22

Sep - Oct 21

NSLW - Oct 21

Dec 21

Feb 22

5th survey event: FOCUS: Featured Breakfast Entrees

6th survey event: FOCUS: Spring Menu

3rd survey event: FOCUS: Fall Winter Menu

1st survey event: FOCUS: Fall Winter Menu

2nd survey event: FOCUS: Featured Regional Lunch Entrees

4th survey event: FOCUS: Possible new menu items for the 22-23 SY

3k

12.6k

4.5k

*28k

1.3k

12.9k

8%

5%

18%

2%

19%

6%

Over 63K responses all year!

= % of student population

*About 5.7K responses per day

Campus Participation by Survey Event

District-Wide Data

AVG Campus Participation for 21-22 SY

September-October 21

February 22

NSLW 21

NSBW 2022

97 %

December 21

May 22

Measuring Campus Performance

21-22 SY

Response Rate (RR)

sURVEY cOUNT

February 22 - May 22

September 21 - December 21

1. Peformance of schools determeind using total # of survey responses collected & avg meals served

RR = (total # of surveys / avg meals served) / 100

&

1. Performance of schools determined by the total # of surveys collected.

Region Participation

Average Regional Representation through the Year

1%

1%

4%

3%

3%

2%

18%

12%

1%

45%

26%

79%

7%

Representation by Menu Line

Representation by Region

Breakdown by Grade Level

District-Wide Year Long AVGs

2%

4%

5%

6%

6%

7%

7%

7%

8%

43%

57%

8%

8%

9%

12%

13%

Overall perceptions of meal service

Tracked throughout the year

may

sep

dec

feb

Breakfast Taste

How many stars (1-5) for breakfast meals taste?

3.6

3.6

3.7

3.9

Lunch Taste

How many stars (1-5) for lunch meals taste?

3.6

3.5

3.4

3.9

How healthy is Breakfast

% of "somewhat healthy" & "very healthy"

82%

82%

87%

88%

How healthy is Lunch

% of "somewhat healthy" & "very healthy"

81%

78%

86%

85%

Likely to recomnend

% of "somewhat likely & "very likely"

n/a

63%

79%

81%

February Survey

Student votes for possible new menu options

Breakfast

Total Votes from all Regions

Students displayed a preference to hot/warm breakfast options. Most votes came from older and established regions in TX (RGV, SA, ATX). ✔= Menued for the Spring 2023 Menu

February Survey

Student votes for possible new menu options

Lunch

Total Votes from all Regions

4⭐

Of the 7 options, 71% (5 options) have been either incorporated into our menu or were featured during NSLW 2022. Like with Breakfast options, most votes came from older and established regions in TX (RGV, SA, ATX).

4 ⭐

Rating unavailable

*3⭐

✅ = Currently on one of our menu lines 🌟 = Featured during NSLW. ✔ = Menued for the Spring 2023 Menu

🌟

🌟

🌟

* = non representative sample was collected for this item's NSLW survey

February Survey

Student votes for possible new menu options

Veggie Sides

Total Votes from all Regions

Students displayed a preference to hot/warm veggie options. Most votes came from older and established regions in TX (RGV, SA, ATX).

✅ = The item or a variation of the item is currently on all or one of our menu lines

NSLW 2021

ENTREE RATINGS

Entree Ratings

Total Surveys Responses

1-5 ⭐ Scale

28,369

100%

Campus Participation

Brown: TX & LA Blue: TX Yellow: FL Orange: LA Green: TX & FL Pink: LA & FL Purple: All

NSLW 2021

Highlights

Survey event with highest participation at IDEAAVG Responses / Day = 5.7K, or 8% First time featuring regional dishes in different states.

NSBW 2022

ENTREE RATINGS

Entree Ratings

Total Surveys Responses

1-5 ⭐ Scale

12,565

99%

Campus Participation

NSBW 2022

Highlights

AVG Responses / Day = 2.5K, or 3.5% of student population.Highest NSBW survey participation at IDEA

✅ = Currently on all or one of our menu lines

In-depth Theme Analysis Case Study of ATX

An introduction to Data Curation & Theme Analysis with a concentration in the ATX region.

Data Curation & Coding

How do we analyze and organize our data?

Data Curation: the process of creating, organizing & maintaining data sets so they can be accessed & used by people looking for information in an organization.Coding: The process of labeling & organizing qualitative data to identify different themes & the relationships between them.

I. Extrapolate

II. Code

III. Summarize

From Microsoft Forms

For Open Ended Questions

Breakdown

• Data sets need to be extracted from MicrosoftFORMS platform in EXCEL format .
• This has to be done for each active survey administered.

• EACH open ended question is reviewed and assigned "codes" or themes. Codes are normally predetermined, but door is open for new codes.
• Recurrence for each "code/theme" is calculated.
• Responses coded as "Addition" go through additional analysis to identify menu requests.

• Data is summarized over time and is available on our Survey Reporting Tool
• Further analysis by region and campus can be feasible with enough response volume.

Prevalent Themes - District Wide

Themes that appeared on all or almost all administration events

Other

Dissatisfaction

Addition Requests

Praise

Variety

Modify Menu

Satisfaction

Health Perceptions

Quality Issues

Special Diets

N = 4.6K

**BOLD = present in every survey administration with option for open-ended response. **Responses that fit criteria for more than one theme were coded for multiple themes

Theme Analysis ATX

May 2022

Dissatisfaction

Addition

9.4% of all Open-Ended Responses (1,270) in May General Survey

9.4%

Variety

Satisfaction

Quality Issues

n=120

Modify Menu

6 out 7 schools in data set. Parmer Park was absent.

**Responses that fit criteria for more than one theme were coded for multiple themes

Theme Analysis ATX

May 2022

Theme Analysis ATX May

Montopolis & Kyle

n=33

n=13

Theme Analysis ATX Cont

Bluff Springs & Pflugerville

n=28

n= 25

Theme Analysis ATX Cont

Round Rock Tech & Health Professions

n=9

n=10

**Note:

• There was 2 responses from Rundberg that were coded as "Dissatisfaction", "Variety" & "Other" (commentary on waste).
• There were no responses from Parmer Park.

Activity

Coding in Kahoot

We will be engaging in a simplified version of coding. Pre-work (2 min):

1. Review the definitions of recurring themes/codes found on pages 7-8 of your SummitOCT22_DataReportSY21-22. Add a thumbs up in the chat once you are ready.

To log in (1-2 min):

1. Utilizing your phone, ipad, or computer, type in "kahoot.it", OR click on the link dropped in the teams chat.
2. Once there, type in the Game Pin you are seeing on the screen.
3. Add in your preferred "Nickname" and select "Ok, go!"

The game (10-15 min)

1. You will read the "quote" on the screen and select the most appropriate "code/theme". Reference your definitions guide for theme clarifications.

Current Data for 22-23 SY

Survey Events during 22-23 SY

Brief summary of this year's survey event timeline

E1

E2

E5

E6

E4

Oct 3- Oct 21

NSLW: Oct 24-25

NSBW 2023

May 2023

Feb 23

1st survey event: FOCUS: Fall Winter Menu

2nd survey event: FOCUS: NSLW Featured Entrees

4th survey event: FOCUS: NSBW Featured Entrees

5th survey event: FOCUS: Spring Menu

3rd survey event: FOCUS: New items to consider

4.8K

11.3k

tbd

tbd

tbd

5.9%

2.24%

Over 17k responses so far!

*About 1.8K responses per day

Campus Participation by Survey Event

District-Wide Data

Participation by Survey Event

AVG Campus Participation for 22-23 SY

98%

1st event (Oct 1-21)

2nd event (NSLW Oct 24-28)

Region Participation

Average Regional Representation YTD

0.1%

2%

1%

6%

6%

3%

4%

13%

8%

3%

44%

20%

79%

8%

Representation by Menu Line

Representation by Region

Breakdown by Grade Level

District-Wide YTD

2%

2.4%

3%

4.7%

5.1%

5.4%

7%

8%

9%

10%

13%

14%

17%

0.3%

34% n=5.5K

66% n=10.7K

All student survey responses in data sets =16,168

Overall perceptions of meal service

LA

FL

TX

OH

Breakfast Taste

n=86

n=0

n=119

n=4k

How many stars (1-5) for breakfast meals taste?

4.4

na

3.3

4.2

Lunch Taste

How many stars (1-5) for lunch meals taste?

4.4

na

3.3

3.7

Supper Taste

How many stars (1-5) for Suppermeals taste?

4.3

na

3.2

3.9

How healthy is Breakfast?

% of "somewhat healthy" & "very healthy" in %

91

na

81

85

How healthy is Lunch?

% of "somewhat healthy" & "very healthy" in %

93

na

79

80

How healthy is Supper?

% of "somewhat healthy" & "very healthy" in %

94

na

75

77

NSLW 2022

Entree Ratings

Total Surveys Responses

1-5 ⭐ Scale

11,305

99%

Campus Participation

NSLW 2022

Highlights

Collected representative samples on entrees served in regions with most students. All items with reprentative samples receivd at least a 3⭐ Second time featuring regional dishes in different states.

Shades of blue: Adequate sample size Shades of yellow: Over 150 responses Shades of brown/orange: FL&OH entrees Shades of gray: OH only

Prevalent Themes

Themes that appeared on all or almost all administration events

5,839

Total Survey Responses

1,852

Total Open-Ended Responses

32%

of Total Survey Responses

**Responses that fit criteria for more than one theme were coded for multiple themes

Final Thoughts & Next Steps

Limitations

Lessons Learned so far.

Trend Data vs. Campus Data

Manpower

Platform

Team of 1-2 people

Volume of data matters

Seeking to replace MicrosoftForms

Surveys help identify general trends for menu preferences & levels of satisfaction in our meal service because of the volume of responses we receive as a district. Campus-specific data does not always match this same level of volume, making campus-specific "findings" difficult to discern from data sets.

1 person is extracting, managing, curating & analyzing data sets of at least 1K responses.

Format capabilities have remained challenging in our efforts to create a user-friendly survey for our students.

Looking Forward

Upcoming projects and efforts

Increase recurrence of representative samples

Emphasis on Sampling Events

Pilot(s) with NC

Already happening

Nutrition Committee members have agreed to help us pilot a survey initiative where we will track student responses during the school year.

Already happening

Already happening

Expanded our sampling list to get more targeted feedback on the menu items that interest us the most.

3% of our student population is a representative sample size district-wide, a % we have proven to be able to meet already.

Any questions?

Thanks!