Strengthening Holistic Learning Outcome Measurement Systems
Click to explore!
Why focus on education assessment, data, and evidence systems? Watch to learn more!
Project Overview
The HOLAS Framework
The Pilot Results
With the support of Education Cannot Wait, NYU Global TIES for Children and the Universidad de los Andes have collaborated to develop and pilot a suite of resources that supports stakeholders' ability to understand, map, and assess holistic learning measurement systems with a focus on equity, inclusion, and well-being. Click to your right to explore our project overview, the Holistic Learning Assessment Systems (HOLAS) framework, and our results and recommendations piloted in Colombia and Peru!
hat is the purpose of the HOLAS framework?
hy develop the HOLAS framework?
ow did you develop the HOLAS framework?
HOME
nteract with the framework!
PREVIOUS PAGE
FRAME WORK
HOME
hat is the purpose of the HOLAS framework?
In order to strengthen coherence and equity in the Colombian and Peruvian holistic learning outcome measurement systems, we needed a framework that would allow us to bound, map, and communicate about such systems. The HOLAS framework was developed for three purposes:
FRAMEWORK FAQS
To facilitate clear and inclusive communication about the elements of holistic learning M&E systems.
We sought to clearly identify and describe elements of holistic learning monitoring and evaluation (M&E) systems using user-friendly language and incorporating multiple stakeholder voices.
PREVIOUS PAGE
To build consensus on areas of (mis)alignment within holistic learning M&E systems.
We piloted a methodology for education systems analysis which facilitates replication and transparency through the use of open-source quantitative and qualitative tools.
To prioritize strategies for strengthening alignment within systems to support equitable holistic learning.
We designed the framework to provide detailed information that enables the development and prioritization of concrete strategies for strengthening alignment in holistic M&E systems.
NEXT PAGE
Click each number to read more!
Three main elements - information, goals, and support - define holistic learning outcome measurement systems.
HOME
Diversify, understand, and align the purposes of assessment
Use information for supporting equitable holistic learning
Ensure quality of information
Enable access to information
INFORMATION
Each systemic element consists of four dimensions.
Click to learn more about the elements and dimensions of the framework!
Promote agency among frontline providers and authoriities in monitoring and evaluation
Establish clear goals for holistic learning
Align assessments with key elements of education systems
Take evidence-based decisions
GOALS
Establish strong organizational structures for effective monitoring and evaluation
Provide high-quality assessment guidance materials
Provide high- quality holistic learning opportunities to support equity and well-being among authorities and front-line providers
Ensure access to high-quality professional development opportunities around data, evidence, and measurement
SUPPORT
PREVIOUS PAGE
HOME
Diversify, understand, and align the purposes of assessment
Use information for supporting equitable holistic learning
Ensure quality of information
Enable access to information
INFORMATION
Promote agency among frontline providers and authoriities in monitoring and evaluation
Establish clear goals for holistic learning
Align assessments with key elements of education systems
Take evidence-based decisions
GOALS
Establish strong organizational structures for effective monitoring and evaluation
Provide high-quality assessment guidance materials
Provide high- quality holistic learning opportunities to support equity and well-being among authorities and front-line providers
Ensure access to high-quality professional development opportunities around data, evidence, and measurement
SUPPORT
PREVIOUS PAGE
HOME
Diversify, understand, and align the purposes of assessments
INFORMATION
This dimension relates to the variety of and alignment between assessments in the educational system and the extent to which stakeholders understand their "fit for purpose." We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Stakeholders' understanding of assessment purposes
The extent to which different stakeholders can clearly and accurately identify the purposes for which educational assessments were originally designed.
BACK TO FRAME WORK
Variety of assessment purposes and types
Alignment between assessments for different purposes
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which a variety of types of assessments are available to assess different holistic outcomes and processes.
The extent to which assessments of similar skills or outcomes that were designed for different purposes provide information that is aligned or coherent
NEXT SUB- ELEMENT
Colombia
Peru
HOME
Use information for supporting equitable holistic learning
INFORMATION
This dimension relates to the extent to which information from a variety of types of assessments - as well as from monitoring and evaluation systems - is used responsibly by stakeholders to make holistic learning-oriented decisions. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Fair use
Type of decisions
The extent to which information is used in a way that is fair and equitable. This includes the extent to which information does not explicitly or implicitly sitgmatize childhoods in contexts of marginalization, and actively supports equity in the allocation of resources and opportunities.
The extent to which information is used (or not) for decision making that support holistic learning outcomes.
BACK TO FRAME WORK
Eco-system information flows for decision-making
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which information informs decision-making by authorities, and is also shared back with and used by schools, teachers, or community stakeholders.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
Ensure quality of information
INFORMATION
This dimension relates to the quality of education measures, assessments, evaluation tools, monitoring and evaluation systems and/or the data obtained from them. We specifically consider four criteria within this dimension.
NAVIGATE THIS PAGE
Availability of evidence of information quality
Sufficiency of evidence of information quality
The extent to which reports of the psychometric properties/quality of assessments are publicly available.
The extent to which there is sufficient evidence that information provided by the assessments or tools is valid, reliable, and/or fair. Sufficiency is determined based on the purpose of the assessment.
BACK TO FRAME WORK
Mechanisms and practices to ethnically ensure information quality
Fairness
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which there are mechanisms and practices in place to ensure inclusion and meaningful representation of childhoods in contexts of marginalization at all stages of the assessment process, from the design to data collection to analysis and dissemination.
The extent to which practices and mechanisms are in place and utilized to ensure data from assessments or tools meets quality standards commensurate with the intended use of the information.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
Enable access to information based on open-science principles
INFORMATION
This dimension relates to the degree (or not) of physical or digital access to various objects used in or resulting from the monitoring and evaluation process, including results, data, measures, materials, and analytic code (hereafter, monitoring and evaluation or M&E objects). It also relates to stakeholders' practices of sharing such M&E objects. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Sharing of evaluations, data, results and materials
The extent to which there are mechanisms, processes, and expectations in place for stakeholders to share and disseminate M&E objects to diverse stakeholders.
BACK TO FRAME WORK
Principled availability of/access to guidance documents, data, or results
The extent to which M&E objects from various stages of the monitoring and evaluation process can be obtained. This includes whether there are clear pathways for accessing or retrieving objects and whether objects are readily provided upon request.
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Establish clear goals for holistic learning
INFORMATION
This dimension relates to whether clear learning objectives for both academic and social and emotional learning are outlined in official documents for all children within the education system. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Clarity of objectives to support holistic learning for refugees and groups in contexts of marginalization
The extent to which there are official documents that outline clear and explicit objectives to strengthen, promote, and enhance the holistic learning outcomes of groups in contexts of marginalization.
Clarity on the objectives of holistic learning
BACK TO FRAME WORK
The extent to which academic and social and emotional learning objectives are outlined in official documents in a coherent, consistent and explicit manner.
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Align assessments with key elements of education systems
INFORMATION
This dimension relates to the extent to which education measures, assessments, evaluation tools, and/or monitoring and evaluation systems are aligned with curricula, standards, and professional development trainings within the education system. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Alignment of asessments with other elements of the system
Alignment of educational assessments with corresponding standards or regulations
The extent to which the design and results of educational assessments and tools are aligned with other elements of the education system, such as non-formal education.
The extent to which there are clear and specified linkages between an educational assessment or tool and corresponding national or sub-national standards or regulations.
BACK TO FRAME WORK
Alignment of educational assessments with frontline provider training opportunities
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which the design and results of educational assessments and tools are aligned with professional development opportunities for frontline service providers.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Take evidence-based decisions
INFORMATION
This dimension relates to the extent to which decisions are made based on a cumulative body of knowledge about the drivers of holistic learning outcomes. We specifically consider one criteria within this dimension.
NAVIGATE THIS PAGE
BACK TO FRAME WORK
Evidence of factors that enhance student holistic learning
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which educational authorities make decisions based on evidence of drivers of holistic learning outcomes, including access, quality, and continuity of education.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Promote agency among front-line providers and authorities in monitoring and evaluation
INFORMATION
This dimension refers to the extent to which frontline providers such as teachers and principals and sub-national educational authorities are perceived as capable of undertaking monitoring and evaluation activities. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Stakeholder support for school staff design and use of assessments
The extent to which diverse stakeholders perceive that frontline providers are capable of designing and implementing their own assessments or tools, as well as using the resulting data.
BACK TO FRAME WORK
Stakeholder support for sub-national staff design and use of assessments
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which diverse stakeholders perceive that sub-national educational authorities are capable of designing and implementing their own educational assessments or tools, as well as using the resulting data
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Provide high-quality assessment guidance materials
INFORMATION
This dimension relates to the existence and quality of guidance materials that allow sub-national authorities and front-line service providers to design, administer, and effectively use assessments. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Materials for monitoring and evaluation assessment design, understanding, and data utilization
The quality of guiding materials that foster knowledge and skills related to the design, comprehension, and the effective utilization of education monitoring and evaluation data. Such resources may include manuals, frameworks, item banks, infographcs, and more.
BACK TO FRAME WORK
Materials for classroom assessments design and use
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which materials to support classroom assessment design and use - such as dashboards, rubrics, item banks, and workbooks with assessment examples - are perceived as practical and high quality.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Establish strong organizational structures for effective monitoring and evaluation
INFORMATION
This dimension relates to the extent to which there are strong national and sub-national organizational structures to support monitoring and evaluation (M&E) efforts. We specifically consider five criteria within this dimension.
Research, practice, policy, and partnerships
The extent to which national and sub-national M&E offices are capable of establishing external partnerships that facilitate and enhance M&E processes at all stages.
NAVIGATE THIS PAGE
Funding
Existence and leadership
The extent to which adequate funding is secured on an annual basis for national and sub-national monitoring and evaluation offices.
The extent to which national or sub-national M&E offices exist and have clear and established mandates to lead monitoring and evaluation processes; and the degree to which these offices coordinate with each other when required.
BACK TO FRAME WORK
Continuity of M&E efforts in challenging political contexts
Dedicated and trained staff
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which M&E organizational structures maintain (or not) their long-term vision and operational capacity despite the changes that occur within educational systems, particularly during times of political and economic crisis.
The extent to which there are qualified, permanent staff who are part of the offices leading the monitoring and evaluation processes.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Provide high-quality holistic learning opportunities to support equity and well-being among educational authorities and frontline providers
INFORMATION
This dimension relates to the extent to which there are system-level mechanisms in place to strengthen educational authorities and front-line service providers' capacities for equitable and inclusive practices. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Availability of peer support for front-line providers to improve equity and inclusion
Opportunities to foster the social and emotional skills of frontline service providers
The extent to which front-line providers have access to regular peer and professional supports to implement specific strategies to promote holistic well-being, equity, and inclusion.
The extent to which there are evidence-informed pre-service and in-service professional development opportunities that acknowledge and support the well-being and social and emotional skills of front-line service providers.
BACK TO FRAME WORK
Training opportunities for education authorities and frontline providers to improve equity and inclusion
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which educational authorities and front-line service providers have access to pre-service or in-service initiatives to strengthen capacities for addressing the needs and ensuring equitable and inclusive participation of childhoods in contexts of marginalization in curriculum and assessment.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Ensure access to high-quality professional development opportunities around data, evidence, and measurement
INFORMATION
This dimension relates to the extent to which there are system-level mechanisms in place to strengthen educational authorities' and front-line service providers' capacities for generating and using assessments, data and evidence. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Availability of peer support and supervision in data, testing and evaluation for front-line providers
Content of professional development opportunities around assessments, data, and evidence
The extent to which a strong peer network is available at the school or program level to provide regular feedback and improve front-line providers' use of assessments, data, and evidence.
The extent to which professional development opportunities for educational authorities and front-line service providers includes content related to assessment, data, and evidence.
BACK TO FRAME WORK
Quality of professional development opportunities around data, evidence and assesments
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which professional development opportunities for educational authorities and front-line service providers around assessment, data, and evidence allow for hands-on experience, knowledge application, and certification as opposed to "chalk and talk" methods.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
he Pilot Results
We piloted the HOLAS Framework in the current initiative in Colombia and Peru. In doing so, we aimed to provide a proof of concept of the utility of such an approach for strengthening holistic monitoring and evaluation (M&E) systems in contexts of emergency and protracted crisis, beginning in two countries with strong governmental educational M&E systems. You can review our diagnostic reports, which summarize the methodology, the results, and the recommendations of the systems analysis below.
PREVIOUS PAGE
HOME
hy develop the HOLAS framework?
As a starting point, we identified two existing approaches to education systems analysis: The RISE diagnostic framework and the SABER tool. However, we recognized that each framework had complementary strengths and shortcomings for our purposes, leading to our decision to adapt, integrate, and extend them to create HOLAS. Specifically, we noted three, interrelated limitations that required the extension of the RISE and SABER frameworks:
FRAMEWORK FAQS
The need for a systemic focus on the development–humanitarian nexus.
Neither RISE nor SABER specifically focus on education systems within contexts of emergency and protracted crisis. Yet, as climate change, conflict, and human rights violations have displaced over 100 million people there is a need for greater coordination and collaboration between actors working towards national and sub-national education development and actors supporting education as part of humanitarian response to crisis.
PREVIOUS PAGE
The need for a systemic focus on holistic learning outcomes.
Both RISE and SABER focus only on the assessment of academic learning outcomes. However, education systems globally are being called upon to foster holistic learning - both academic knowledge and social and emotional skills and well-being - that can support children to navigate uncertain futures.
The need for a systemic focus on equity.
Neither RISE nor SABER explicitly attend to power dynamics and equity among childhoods in contexts of marginalization. Systems frameworks can provide a powerful tool for interrogating power and relational dynamics.
NEXT PAGE
HOME
ow did you develop the HOLAS framework?
First, we reviewed the RISE and SABER frameworks and recognized that the resources and timeline available for this project did not permit us to fully adapt and extend the frameworks.
FRAMEWORK FAQS
Given our team’s expertise, assets, and relationships, we made the decision to center on the relationship between educational authorities and organizations and frontline service providers, and on the information, support, and goals elements that define interactions within this relationship.
We mapped indicators from the SABER Student Assessment national large-scale and classroom assessment instruments to the different elements and sub-elements within RISE.
In doing so, we added new sub-elements or dimensions to the RISE framework to highlight key M&E processes. We also revised the RISE elements to more systematically consider the role of different types of assessments. This resulted in version 1.0 of the HOLAS framework. We then revised version 1.0 of the HOLAS framework with an eye towards three criteria: extension, usability, and replicability.
PREVIOUS PAGE
We conducted a mixed-methods study with a wide range of stakeholders in Colombia and Peru to triangulate information about the elements and sub-elements within the HOLAS framework.
After collecting a first tranche of data, we held a workshop with our Steering Committees to review preliminary results and provide evidence on the validity of the HOLAS framework 2.0. Based on the results of these validation activities, we revised the HOLAS framework with a focus on (1) Improving the naming conventions of the sub-elements within the HOLAS framework; (2) Clearly identifying the actors and levels of the education system analyzed within each sub-element; and (3) Strengthening the focus within sub-elements on inclusion.
NEXT PAGE
GOALS
Goals
The goals element focuses on the definition and clarity of holistic learning objectives within the system, the alignment of information from education monitoring and evaluation systems with these objectives and other crucial education system components, and the established norms governing the use of this information in decision-making, including the level of autonomy stakeholders have in doing so. We specifically consider four dimensions within this element.
Information
The information element focuses on how and with what quality the information produced by education monitoring and evaluation systems is generated, accessed, used, and shared by education authorities, organizations, frontline providers, and researchers for a variety of purposes. We specifically consider four dimensions within this element.
The Project
The HOLAS framework was developed by NYU Global TIES for Children (NYU-TIES) and Universidad de los Andes (Uniandes) researchers in the scope of the "Strengthening Holistic Learning Measurement Systems in Education in Emergencies and Protracted Crises in Colombia and Peru" project, sponsored by Education Cannot Wait (ECW). Visit our website to learn more.
Support
SUPPORT
The support element includes the mechanisms that are in place to ensure comprehensive, evidence-informed holistic learning at different levels of the education system, including the availability and quality of resources, professional development opportunities, and organizational structures. We specifically consider four dimensions within this element.
ECW framework (ENG)
Carly Dolan
Created on June 12, 2023
Start designing with a free template
Discover more than 1500 professional designs like these:
Explore all templates
Transcript
Strengthening Holistic Learning Outcome Measurement Systems
Click to explore!
Why focus on education assessment, data, and evidence systems? Watch to learn more!
Project Overview
The HOLAS Framework
The Pilot Results
With the support of Education Cannot Wait, NYU Global TIES for Children and the Universidad de los Andes have collaborated to develop and pilot a suite of resources that supports stakeholders' ability to understand, map, and assess holistic learning measurement systems with a focus on equity, inclusion, and well-being. Click to your right to explore our project overview, the Holistic Learning Assessment Systems (HOLAS) framework, and our results and recommendations piloted in Colombia and Peru!
hat is the purpose of the HOLAS framework?
hy develop the HOLAS framework?
ow did you develop the HOLAS framework?
HOME
nteract with the framework!
PREVIOUS PAGE
FRAME WORK
HOME
hat is the purpose of the HOLAS framework?
In order to strengthen coherence and equity in the Colombian and Peruvian holistic learning outcome measurement systems, we needed a framework that would allow us to bound, map, and communicate about such systems. The HOLAS framework was developed for three purposes:
FRAMEWORK FAQS
To facilitate clear and inclusive communication about the elements of holistic learning M&E systems.
We sought to clearly identify and describe elements of holistic learning monitoring and evaluation (M&E) systems using user-friendly language and incorporating multiple stakeholder voices.
PREVIOUS PAGE
To build consensus on areas of (mis)alignment within holistic learning M&E systems.
We piloted a methodology for education systems analysis which facilitates replication and transparency through the use of open-source quantitative and qualitative tools.
To prioritize strategies for strengthening alignment within systems to support equitable holistic learning.
We designed the framework to provide detailed information that enables the development and prioritization of concrete strategies for strengthening alignment in holistic M&E systems.
NEXT PAGE
Click each number to read more!
Three main elements - information, goals, and support - define holistic learning outcome measurement systems.
HOME
Diversify, understand, and align the purposes of assessment
Use information for supporting equitable holistic learning
Ensure quality of information
Enable access to information
INFORMATION
Each systemic element consists of four dimensions.
Click to learn more about the elements and dimensions of the framework!
Promote agency among frontline providers and authoriities in monitoring and evaluation
Establish clear goals for holistic learning
Align assessments with key elements of education systems
Take evidence-based decisions
GOALS
Establish strong organizational structures for effective monitoring and evaluation
Provide high-quality assessment guidance materials
Provide high- quality holistic learning opportunities to support equity and well-being among authorities and front-line providers
Ensure access to high-quality professional development opportunities around data, evidence, and measurement
SUPPORT
PREVIOUS PAGE
HOME
Diversify, understand, and align the purposes of assessment
Use information for supporting equitable holistic learning
Ensure quality of information
Enable access to information
INFORMATION
Promote agency among frontline providers and authoriities in monitoring and evaluation
Establish clear goals for holistic learning
Align assessments with key elements of education systems
Take evidence-based decisions
GOALS
Establish strong organizational structures for effective monitoring and evaluation
Provide high-quality assessment guidance materials
Provide high- quality holistic learning opportunities to support equity and well-being among authorities and front-line providers
Ensure access to high-quality professional development opportunities around data, evidence, and measurement
SUPPORT
PREVIOUS PAGE
HOME
Diversify, understand, and align the purposes of assessments
INFORMATION
This dimension relates to the variety of and alignment between assessments in the educational system and the extent to which stakeholders understand their "fit for purpose." We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Stakeholders' understanding of assessment purposes
The extent to which different stakeholders can clearly and accurately identify the purposes for which educational assessments were originally designed.
BACK TO FRAME WORK
Variety of assessment purposes and types
Alignment between assessments for different purposes
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which a variety of types of assessments are available to assess different holistic outcomes and processes.
The extent to which assessments of similar skills or outcomes that were designed for different purposes provide information that is aligned or coherent
NEXT SUB- ELEMENT
Colombia
Peru
HOME
Use information for supporting equitable holistic learning
INFORMATION
This dimension relates to the extent to which information from a variety of types of assessments - as well as from monitoring and evaluation systems - is used responsibly by stakeholders to make holistic learning-oriented decisions. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Fair use
Type of decisions
The extent to which information is used in a way that is fair and equitable. This includes the extent to which information does not explicitly or implicitly sitgmatize childhoods in contexts of marginalization, and actively supports equity in the allocation of resources and opportunities.
The extent to which information is used (or not) for decision making that support holistic learning outcomes.
BACK TO FRAME WORK
Eco-system information flows for decision-making
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which information informs decision-making by authorities, and is also shared back with and used by schools, teachers, or community stakeholders.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
Ensure quality of information
INFORMATION
This dimension relates to the quality of education measures, assessments, evaluation tools, monitoring and evaluation systems and/or the data obtained from them. We specifically consider four criteria within this dimension.
NAVIGATE THIS PAGE
Availability of evidence of information quality
Sufficiency of evidence of information quality
The extent to which reports of the psychometric properties/quality of assessments are publicly available.
The extent to which there is sufficient evidence that information provided by the assessments or tools is valid, reliable, and/or fair. Sufficiency is determined based on the purpose of the assessment.
BACK TO FRAME WORK
Mechanisms and practices to ethnically ensure information quality
Fairness
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which there are mechanisms and practices in place to ensure inclusion and meaningful representation of childhoods in contexts of marginalization at all stages of the assessment process, from the design to data collection to analysis and dissemination.
The extent to which practices and mechanisms are in place and utilized to ensure data from assessments or tools meets quality standards commensurate with the intended use of the information.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
Enable access to information based on open-science principles
INFORMATION
This dimension relates to the degree (or not) of physical or digital access to various objects used in or resulting from the monitoring and evaluation process, including results, data, measures, materials, and analytic code (hereafter, monitoring and evaluation or M&E objects). It also relates to stakeholders' practices of sharing such M&E objects. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Sharing of evaluations, data, results and materials
The extent to which there are mechanisms, processes, and expectations in place for stakeholders to share and disseminate M&E objects to diverse stakeholders.
BACK TO FRAME WORK
Principled availability of/access to guidance documents, data, or results
The extent to which M&E objects from various stages of the monitoring and evaluation process can be obtained. This includes whether there are clear pathways for accessing or retrieving objects and whether objects are readily provided upon request.
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Establish clear goals for holistic learning
INFORMATION
This dimension relates to whether clear learning objectives for both academic and social and emotional learning are outlined in official documents for all children within the education system. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Clarity of objectives to support holistic learning for refugees and groups in contexts of marginalization
The extent to which there are official documents that outline clear and explicit objectives to strengthen, promote, and enhance the holistic learning outcomes of groups in contexts of marginalization.
Clarity on the objectives of holistic learning
BACK TO FRAME WORK
The extent to which academic and social and emotional learning objectives are outlined in official documents in a coherent, consistent and explicit manner.
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Align assessments with key elements of education systems
INFORMATION
This dimension relates to the extent to which education measures, assessments, evaluation tools, and/or monitoring and evaluation systems are aligned with curricula, standards, and professional development trainings within the education system. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Alignment of asessments with other elements of the system
Alignment of educational assessments with corresponding standards or regulations
The extent to which the design and results of educational assessments and tools are aligned with other elements of the education system, such as non-formal education.
The extent to which there are clear and specified linkages between an educational assessment or tool and corresponding national or sub-national standards or regulations.
BACK TO FRAME WORK
Alignment of educational assessments with frontline provider training opportunities
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which the design and results of educational assessments and tools are aligned with professional development opportunities for frontline service providers.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Take evidence-based decisions
INFORMATION
This dimension relates to the extent to which decisions are made based on a cumulative body of knowledge about the drivers of holistic learning outcomes. We specifically consider one criteria within this dimension.
NAVIGATE THIS PAGE
BACK TO FRAME WORK
Evidence of factors that enhance student holistic learning
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which educational authorities make decisions based on evidence of drivers of holistic learning outcomes, including access, quality, and continuity of education.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
GOALS
Promote agency among front-line providers and authorities in monitoring and evaluation
INFORMATION
This dimension refers to the extent to which frontline providers such as teachers and principals and sub-national educational authorities are perceived as capable of undertaking monitoring and evaluation activities. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Stakeholder support for school staff design and use of assessments
The extent to which diverse stakeholders perceive that frontline providers are capable of designing and implementing their own assessments or tools, as well as using the resulting data.
BACK TO FRAME WORK
Stakeholder support for sub-national staff design and use of assessments
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which diverse stakeholders perceive that sub-national educational authorities are capable of designing and implementing their own educational assessments or tools, as well as using the resulting data
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Provide high-quality assessment guidance materials
INFORMATION
This dimension relates to the existence and quality of guidance materials that allow sub-national authorities and front-line service providers to design, administer, and effectively use assessments. We specifically consider two criteria within this dimension.
NAVIGATE THIS PAGE
Materials for monitoring and evaluation assessment design, understanding, and data utilization
The quality of guiding materials that foster knowledge and skills related to the design, comprehension, and the effective utilization of education monitoring and evaluation data. Such resources may include manuals, frameworks, item banks, infographcs, and more.
BACK TO FRAME WORK
Materials for classroom assessments design and use
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which materials to support classroom assessment design and use - such as dashboards, rubrics, item banks, and workbooks with assessment examples - are perceived as practical and high quality.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Establish strong organizational structures for effective monitoring and evaluation
INFORMATION
This dimension relates to the extent to which there are strong national and sub-national organizational structures to support monitoring and evaluation (M&E) efforts. We specifically consider five criteria within this dimension.
Research, practice, policy, and partnerships
The extent to which national and sub-national M&E offices are capable of establishing external partnerships that facilitate and enhance M&E processes at all stages.
NAVIGATE THIS PAGE
Funding
Existence and leadership
The extent to which adequate funding is secured on an annual basis for national and sub-national monitoring and evaluation offices.
The extent to which national or sub-national M&E offices exist and have clear and established mandates to lead monitoring and evaluation processes; and the degree to which these offices coordinate with each other when required.
BACK TO FRAME WORK
Continuity of M&E efforts in challenging political contexts
Dedicated and trained staff
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which M&E organizational structures maintain (or not) their long-term vision and operational capacity despite the changes that occur within educational systems, particularly during times of political and economic crisis.
The extent to which there are qualified, permanent staff who are part of the offices leading the monitoring and evaluation processes.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Provide high-quality holistic learning opportunities to support equity and well-being among educational authorities and frontline providers
INFORMATION
This dimension relates to the extent to which there are system-level mechanisms in place to strengthen educational authorities and front-line service providers' capacities for equitable and inclusive practices. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Availability of peer support for front-line providers to improve equity and inclusion
Opportunities to foster the social and emotional skills of frontline service providers
The extent to which front-line providers have access to regular peer and professional supports to implement specific strategies to promote holistic well-being, equity, and inclusion.
The extent to which there are evidence-informed pre-service and in-service professional development opportunities that acknowledge and support the well-being and social and emotional skills of front-line service providers.
BACK TO FRAME WORK
Training opportunities for education authorities and frontline providers to improve equity and inclusion
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which educational authorities and front-line service providers have access to pre-service or in-service initiatives to strengthen capacities for addressing the needs and ensuring equitable and inclusive participation of childhoods in contexts of marginalization in curriculum and assessment.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
SUPPORT
Ensure access to high-quality professional development opportunities around data, evidence, and measurement
INFORMATION
This dimension relates to the extent to which there are system-level mechanisms in place to strengthen educational authorities' and front-line service providers' capacities for generating and using assessments, data and evidence. We specifically consider three criteria within this dimension.
NAVIGATE THIS PAGE
Availability of peer support and supervision in data, testing and evaluation for front-line providers
Content of professional development opportunities around assessments, data, and evidence
The extent to which a strong peer network is available at the school or program level to provide regular feedback and improve front-line providers' use of assessments, data, and evidence.
The extent to which professional development opportunities for educational authorities and front-line service providers includes content related to assessment, data, and evidence.
BACK TO FRAME WORK
Quality of professional development opportunities around data, evidence and assesments
Click on underlined text to see definitions of terms.
Click each flag to read pilot findings from each country on this dimension.
The extent to which professional development opportunities for educational authorities and front-line service providers around assessment, data, and evidence allow for hands-on experience, knowledge application, and certification as opposed to "chalk and talk" methods.
NEXT SUB- ELEMENT
Colombia
Peru
HOME
he Pilot Results
We piloted the HOLAS Framework in the current initiative in Colombia and Peru. In doing so, we aimed to provide a proof of concept of the utility of such an approach for strengthening holistic monitoring and evaluation (M&E) systems in contexts of emergency and protracted crisis, beginning in two countries with strong governmental educational M&E systems. You can review our diagnostic reports, which summarize the methodology, the results, and the recommendations of the systems analysis below.
PREVIOUS PAGE
HOME
hy develop the HOLAS framework?
As a starting point, we identified two existing approaches to education systems analysis: The RISE diagnostic framework and the SABER tool. However, we recognized that each framework had complementary strengths and shortcomings for our purposes, leading to our decision to adapt, integrate, and extend them to create HOLAS. Specifically, we noted three, interrelated limitations that required the extension of the RISE and SABER frameworks:
FRAMEWORK FAQS
The need for a systemic focus on the development–humanitarian nexus.
Neither RISE nor SABER specifically focus on education systems within contexts of emergency and protracted crisis. Yet, as climate change, conflict, and human rights violations have displaced over 100 million people there is a need for greater coordination and collaboration between actors working towards national and sub-national education development and actors supporting education as part of humanitarian response to crisis.
PREVIOUS PAGE
The need for a systemic focus on holistic learning outcomes.
Both RISE and SABER focus only on the assessment of academic learning outcomes. However, education systems globally are being called upon to foster holistic learning - both academic knowledge and social and emotional skills and well-being - that can support children to navigate uncertain futures.
The need for a systemic focus on equity.
Neither RISE nor SABER explicitly attend to power dynamics and equity among childhoods in contexts of marginalization. Systems frameworks can provide a powerful tool for interrogating power and relational dynamics.
NEXT PAGE
HOME
ow did you develop the HOLAS framework?
First, we reviewed the RISE and SABER frameworks and recognized that the resources and timeline available for this project did not permit us to fully adapt and extend the frameworks.
FRAMEWORK FAQS
Given our team’s expertise, assets, and relationships, we made the decision to center on the relationship between educational authorities and organizations and frontline service providers, and on the information, support, and goals elements that define interactions within this relationship.
We mapped indicators from the SABER Student Assessment national large-scale and classroom assessment instruments to the different elements and sub-elements within RISE.
In doing so, we added new sub-elements or dimensions to the RISE framework to highlight key M&E processes. We also revised the RISE elements to more systematically consider the role of different types of assessments. This resulted in version 1.0 of the HOLAS framework. We then revised version 1.0 of the HOLAS framework with an eye towards three criteria: extension, usability, and replicability.
PREVIOUS PAGE
We conducted a mixed-methods study with a wide range of stakeholders in Colombia and Peru to triangulate information about the elements and sub-elements within the HOLAS framework.
After collecting a first tranche of data, we held a workshop with our Steering Committees to review preliminary results and provide evidence on the validity of the HOLAS framework 2.0. Based on the results of these validation activities, we revised the HOLAS framework with a focus on (1) Improving the naming conventions of the sub-elements within the HOLAS framework; (2) Clearly identifying the actors and levels of the education system analyzed within each sub-element; and (3) Strengthening the focus within sub-elements on inclusion.
NEXT PAGE
GOALS
Goals
The goals element focuses on the definition and clarity of holistic learning objectives within the system, the alignment of information from education monitoring and evaluation systems with these objectives and other crucial education system components, and the established norms governing the use of this information in decision-making, including the level of autonomy stakeholders have in doing so. We specifically consider four dimensions within this element.
Information
The information element focuses on how and with what quality the information produced by education monitoring and evaluation systems is generated, accessed, used, and shared by education authorities, organizations, frontline providers, and researchers for a variety of purposes. We specifically consider four dimensions within this element.
The Project
The HOLAS framework was developed by NYU Global TIES for Children (NYU-TIES) and Universidad de los Andes (Uniandes) researchers in the scope of the "Strengthening Holistic Learning Measurement Systems in Education in Emergencies and Protracted Crises in Colombia and Peru" project, sponsored by Education Cannot Wait (ECW). Visit our website to learn more.
Support
SUPPORT
The support element includes the mechanisms that are in place to ensure comprehensive, evidence-informed holistic learning at different levels of the education system, including the availability and quality of resources, professional development opportunities, and organizational structures. We specifically consider four dimensions within this element.