NYU TIES - White paper

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

Supporting Children’s Holistic Development in Contexts of Conflict and Crisis

Lessons from a decade of striving towards research that is effective, equitable and sustainable

Before you dive in, check out our Reader's Guide!

Begin reading!

To cite this resource: Dolan, C.T., Shankar, A., Caires, R., Aber, L.T., Yoshikawa, H., Kim, H.Y., Rafla, J., Woulfin, D., Seminario, E., Anker, P., Boulares, D. (2024). “Supporting Children’s Holistic Development in Contexts of Conflict and Crisis: Lessons from a decade of striving towards research that is effective, equitable and sustainable". [White paper]. NYU Global TIES for Children, New York.

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION

REFLECTIONS

REFERENCES

ABOUT US

HOME

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION: Understanding the Motivation and Contributions

REFLECTIONS

THE "WHAT": Supporting Robust and Inclusive Developmental Science

THE "HOW": Transforming Science into Impact

“We need a buffet of options.” - Expanding Research Methods

“It’s hard to focus, because I might be asked to leave the country." - Investing in Staff

“Measurement for what?” - Ensuring Fit-for-Purpose Methods

“We care about you first and foremost” - Building Trust-Based Partnerships

The Need for Systematic Reviews and Field Mapping

“Measurement is an intervention in and of itself” - Prioritizing Process and Product

“Making the invisible visible.” - Integrating Tacit Knowledge

“There cannot be a single point of failure” - Integrating Robust Data Practices

"Whose Tools? Whose House?" - Reflections on Power and Equity

“Who is your audience?” - Prioritizing Strategic Communications

CONCLUSION

ABOUT US

SUGGESTED CITATION

REFERENCES

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION

REFLECTIONS

REFERENCES

ABOUT US

HOME

EXECUTIVE SUMMARY

In reflecting upon a decade of research at NYU Global TIES aimed at supporting children's holistic development within contexts of conflict and crisis, this white paper encapsulates our collective learnings from working with a network of actors in low- and middle-income and conflict affected contexts who aim to understand what works, how, and for whom. The goal of this paper is to distill insights from the diverse perspectives and experiences of dedicated researchers, inspire action, and provide recommendations for a future distinguished by inclusive and impactful outcomes.

THE "WHY"

THE "HOW"

THE "WHAT"

For a summary of the “why”, “how”, and “what” click into each of the patches! To read the full report, continue along by clicking on the arrow on the bottom right of this page.

Introduction:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

INTRODUCTION

Understanding the Motivation and Contributions What motivated us to do this work? Who is “us”? And how have we contributed?

To achieve this mission, we have aimed to strengthen our individual, organizational, and partnership capacities for learning explicitly through measurement, data, and multiple forms of evidence, and tacitly through stories, histories, and a culture of care. Through data and evidence, we aim to illuminate the educational landscape, challenge prevailing worldviews, and advocate for systemic changes that promote equitable learning experiences. In the face of armed conflicts, the COVID-19 pandemic, and climate change, education systems must be resilient, addressing the cascading effects that disproportionately impact marginalized children. In 10 years, we have experienced successes and failures in generating, communicating, and using evidence to inform program, policy, and systems change. The examples in this report are not exhaustive of Global TIES work (they primarily concentrate on work in primary education, for example, not early childhood development) , but represent the range of lessons learned during this decade.

NYU Global TIES –an international research center housed at NYU and comprised of a diverse staff of multi-disciplinary researchers and administrators- has undertaken this work to ensure that programs and policies in low- and middle-income and crisis settings more effectively support all children, their caregivers, teachers, communities, and systems to thrive. By collaborating with a complex ecosystem of actors—national and sub-national policymakers, NGOs, researchers, donors, teachers, and caregivers—we have sought to understand what interventions work, how they work, and for whom. This paper synthesizes insights from a broad range of perspectives, intending to inspire action and provide actionable recommendations for creating inclusive and impactful outcomes. Over the last decade, TIES has evolved to contribute to “the why” through a robust, culturally and contextually grounded, and inclusive science developed in partnerships within this ecosystem of actors.

Reflections:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

Each piece of this patchwork blanket has been crafted by NYU-TIES staff. Some are co-written, others individual reflections; but the hope in stitching it together was to build on the rich, targeted expertise of each team mate and paint a fuller picture of our experiences in and what it takes to conduct holistic child developmental research in low- and middle-income contexts. Hover over each of the patches to view their titles and click on it be directed to the reflection! Throughout this document, you can also view citations by hovering your mouse over superscripted numbers.

Dive into the reflections!

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: In the EQUAL network project, TIES as a global North-based institution applied principles of global South-based leadership and meaningful engagement to a seed grant initiative to advance research on SDG 4 (across early childhood to secondary education). The network fielded 100 applications from early- to mid-career scholars (within 10 years of their doctorate) to ultimately fund 27 seed grants to 16 scholars from sub-Saharan Africa (SSA) and 11 scholars from the Middle East / North Africa (MENA). The initiative was structured as 2 separate networks, one for SSA and one for MENA. What Worked: Multiple grantees have gone on to become rising stars in the field of education research. Two grantees (Pamela Wadende (a professor at Kisii University, Kenya and Missaye Mengstie, a professer at the University of Gondar, Ethiopia) received a Templeton Foundation full research grant and have become regional research leaders in East Africa. Christopher Kwaah (professor at the University of Cape Coast in Ghana) has gone on to lead a country-wide consortium on scholar-ministry partnerships in Ghana (funded by the Jacobs Foundation). Reflecting on what facilitated such professional development, the following principles were utilized to ensure that this initiative was equitable and sustainable:

• An intergenerational, within-region approach for mentorship: Each grantee was required to propose a senior mentor from their country (usually from their institution). In addition, each grantee had to propose a more junior member of their research team (a maters-level scholar), as a mentee. In this way, 3 generations of scholars were linked in mentorship relationships. Monthly on-line meetings served as opportunities for feedback, inter-generational mentorship and discussion of focused topics, such as culturally grounded measurement or dissemination and communication of results.

• Research-practice partnerships (RPPs). Each grant represented a RPP and each applicant was required to identify an NGO or government partner. These partners were invited to the online as well as in-person convenings along with the grantees (one in-person convening occurred in Tunis; the other in Nairobi). The partners gave some of the most insightful and engaged commentary during the in-person meetings.

• Senior advisory boards: for each region, advisory boards were created and composed of senior scholars in education research from a range of the target countries. Thus, in addition to their proposed mentors, scholars had ongoing access to a wider range of senior scholar mentors from their regions. These senior scholars worked on every phase of the project, from review of letter of intent to review of full applications to supporting the cohort of grantees as they met in monthly zoom meetings and in in-person convenings.

• Mentorship applied to early phases of the process, not just to the awardees’ experiences. The letter of intent reviews were shared with grantees, and questions were answered to facilitate their writing of as strong a full grant proposal as possible. Feedback on full grant proposals was also shared. In this way, mentorship in grant writing was received by all 100 applicants, not just the finalists and not just the ultimate grantees.

"We need a buffet of options."

Expand and integrate the repertoire of methods to support culturally and contextually grounded research

Hirokazu Yoshikawa, Co-Director

This is page 1/2 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

• The leadership within Global TIES for each network was drawn from staff members from each of the regions in question. The facilitators for the SSA network were Prof. Paul Oburu (from Maseno University, Kenya), and Jess Castelyn (from South Africa). The facilitator for the MENA network was Joyce Rafla (from Egypt). This served to bridge the divide between a global North institution (and a PI from neither of those regions, Yoshikawa) with the scholars from 14 SSA and MENA countries.

• Multiple skills were supported in the initiative: These were determined in large part by the phase and nature of the research that the grantees were engaged in, but also by some additional topics added by TIES staff. For example, grantees worked on measurement, psychometrics, quantitative analysis, qualitative and quantitative data collection, and qualitative data analysis. Beyond this, in the final phase of the project funds were provided to support formatting of briefs for practitioners and policy makers. Feedback was given by the grantees and the larger team on the briefs, and presentations were given on their dissemination to local ministry and NGO partners.

Room for improvement: several factors were barriers to effectiveness and sustainability:

• Contracting processes and COVID-driven delays: One way this project was not sustainable was the extent of work it took on the part of both NYU and the grantees’ universities to complete the contracting process. This delayed the projects in some cases by a full year. In addition, COVID delayed much of the grantees’ data collection and their project periods were essentially extended from 18 to 30 months.

• Engaging the higher administration of each university: Grantees’ work occurred within the context of a variety of university administration contexts that enabled or constrained their work in different ways. However, EQUAL did not explicitly engage the higher administration of each university. We had received feedback from a former vice chancellor of the University of Zambia to engage deans and higher-level research administration at each university to ensure that each grantee worked under conditions that would enable maximum involvement in their proposed research. However, at TIES we did not have the capacity to include a higher education institutional support component to the EQUAL initiative.

• Cost allocations: The amounts of the grants went further in one region (SSA) than the other (MENA) due to cost of research and exchange rate differences. We could have accounted for this by making the research grants tailored to country-specific costs of research and exchange rates.

The above challenges, in addition to others, meant that we did not implement an initial idea which was to support “hub” universities within each region, supporting each to become leaders in their region to lead networks of scholars from multiple countries working in education and human development research. This approach is being implemented in a new network grant – LEARN, funded by the Jacobs Foundation and focused on scholar-ministry partnerships in Colombia and Ghana – with the aim of further decentering Global TIES as a global North institution supporting global South institutions’ agency and leadership within their countries, regions, and globally.

"We need a buffet of options."

Expand and integrate the repertoire of methods to support culturally and contextually grounded research

Hirokazu Yoshikawa, Co-Director

This is page 2/2 of this section. Click the arrow below to go back to the previous page:

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: We wouldn’t use a kitchen scale to weigh an elephant. The scale doesn’t fit the purpose. Similarly, measures of children’s holistic learning and development – and of the home, school, and community factors that shape them - need to be selected and used for the purpose for which they were designed. Measures are designed to provide users with information or data. But how you intend to interpret and use that information and data - or the purpose of your assessment – can vary greatly. While it is apparent that an elephant would never fit on a kitchen scale, it is not always so easy to determine if educational measures are appropriate for the purpose for which you hope to use them. As researchers, we advocate for increased evidence-based decision-making, recognizing however, that data alone does not constitute evidence because it lacks the context and interpretation necessary to inform decision-making processes effectively. Evidence, on the other hand, provides the contextualized information and insights needed to verify hypotheses, draw conclusions, and guide actions .The Problem: The ability of the Education in Emergencies (EiE) sector to develop high-quality data collection tools that provide meaningful evidence that can be used to inform policy, practice, and decision-making is hindered by three concurrently pressing challenges:

• Historically, there has been a lack of access to tools that have been developed and/or tested in EiE contexts. The majority of tools that exist are those that have been developed for Global North contexts, but questions of whether or not these tools work in different contexts loom large. Particularly when it comes to the assessment of social and emotional learning (SEL) skills and behavior, wherein the knowledge about what it is and how it manifests is largely based on and generated by populations in the Global North, the application of such tools in non-Western contexts can impose a way of being through language and assessment. In our efforts to adapt SEL measures for Arabic-speaking regions, for instance, we encountered challenges such as the absence of direct equivalents for certain concepts that are common in Western contexts. For instance, the concept of "frustration" as understood in the West did not resonate in the Lebanese context we were studying. This experience underscores the importance of carefully aligning behavioral and social-emotional assessments with community linguistic and cultural perspectives.
• The EiE field is increasingly interested in evidence-based approaches, with many program implementers feeling the need to respond to donor requests for evidence and developing tools to collect data. Data is therefore collected, but insufficient support (including opportunities to develop the skills needed to design fit-for-purpose tools, conduct analyses, and interpret what the data are telling them) is provided to help contextualize the data, and then transform it into usable evidence through testing.
• Assessment tools often are and should be held to high standards – indeed, setting up systems and processes to evaluate the psychometric quality of assessment tools tested in EiE contexts has been a large part of our work in partnership with the .

"Measurement for what?”

10

9,

Ensure methods are fit for purpose, cost, and timelines

Roxane Caires, Project Managing DirectorCaroline Tubbs, Deputy Director Abiraahmi Shankar, Research Associate

This is page 1/2 of this section. Click the arrow below to go to the next page:

INEE Measurement Library

But in contexts where time and resource constraints may mean that tools cannot be tested with big enough samples, and the data from it cannot be used for complex enough analyses, it is important to right-size research designs with the contexts and available resources. Further, it is equally important to invest in the dissemination of such tools and sharing the evidence available on it. For the vast majority of assessment designers, the burden of preparation of assessment information for evaluation by a body such as the INEE Measurement Library, is typically greatly underfunded, if it is funded at all.

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

INEE Measurement Library

• We found that developing and sharing measurement resources (via collaborative efforts such as the

) works. Through these efforts, we systematically collected,

11

12

, and

MENAT Inventory

NYU-TIES and Uniandes’ HOLAS Assessment Bank

organized, and published measurement tools, alongside guidance on how to make informed decisions about how to use them. While many researchers and program implementers develop assessment tools for low- and middle-income contexts, finding information about their accessibility and psychometric quality is often challenging. Academic institutions and global networks like INEE are enhancing their support for the organized collection of such data, thanks to increased donor investments in global public goods. Access to a network of researchers, practitioners, and policymakers is crucial in facilitating this endeavor.

"Measurement for what?”

• Our successful partnerships, including those with governmental, non-governmental, and program implementing organizations, have been rooted in mutual capacity-building with clearly defined purposes. Key achievements, such as developing a framework for SEL for the Lebanese Ministry of Education , creating national monitoring and evaluation tools for the Ministry of Education in Peru , and conducting capacity-building training across various contexts during the UNHCR’s Humanitarian Education Accelerator , were all underpinned by clearly defining the assessment's purpose. We mapped the team’s theories of change to understand how inputs and activities lead to desired outcomes and impacts. By allowing partners to lead decision-making in the measure development process and maintaining open, transparent communications, we built trust and effectively identified needs. This approach ensured that capacity-building efforts facilitated mutual learning and that quality was sustained by leveraging existing capacities and resources. Our own capacity to engage in culturally grounded measurement was strengthened in the mutual capacity-building process.

13

14

15

16

Ensure methods are fit for purpose, cost, and timelines

• We also found that cultivating strong donor relationships and educating donors on the importance and value of supporting high-quality data collection and evidence generation is crucial for successful program implementation. Our projects have demonstrated that when donors understand the intricacies of how research can inform practice and policy, they are more likely to support comprehensive data initiatives. Over the past decade, we have observed a significant shift in donor advocacy towards greater research investment and the promotion of global public goods. Specifically, successful donor relationships are characterized by:

1. Mutual Flexibility: Conducting research in highly uncertain humanitarian contexts requires adaptability. Donors who provide flexible funding and allow for adjustments based on real-time challenges enable more effective and responsive research practices.
2. Close, Frequent Communication: Maintaining regular, transparent communication with donors about progress, barriers, and solutions fosters a collaborative environment. This open dialogue helps align expectations, build trust, and ensures that the research remains relevant and impactful.

By prioritizing these principles, we can enhance our ability to collect and use high-quality data, ultimately leading to better-informed policies and practices that support children's holistic development in crisis contexts.

Roxane Caires, Project Managing DirectorCaroline Tubbs, Deputy Director Abiraahmi Shankar, Research Associate

This is page 2/2 of this section. Click the arrow below to go back to the previous page:

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: Investment, programming and research on the education, development and well-being of refugee children and other children in conflict- and crisis-affected areas in the majority world has rapidly increased since the launch of UN Sustainable Development Goal 4(SDGs) in 2015, which called for ensuring inclusive and equitable quality education and promoting lifelong learning opportunities for all. This increase in investments was largely supported by funders, governments, and international NGOs (INGOs) working in the majority world. However, this rapid growth in programming and research efforts with diverse policy, programming, and research approaches came with increasing fragmentation of the research into different silos supported by different UN agencies and other multilaterals, iNGOs, governments, funders, academic disciplines, and other stakeholders.The Problem: an incoherent evidence base posing challenges in generating evidence-based programming and policy decisions to promote holistic learning in conflict- and crisis-affected contexts, resulting in inefficient and uncoordinated programming approaches and unnecessary duplication of research efforts due to the of a lack of inter-sector coordination and publicly accessible data across research projects. For example, the two largest topics/subdisciplines within this fragmented landscape, “Education in Emergencies” (EiE) and “Refugee Education” have little overlap in publication, reference, and authorship despite their similarity in research topics and target populations. Bibliographic data collected from

The Need for Systematic Reviews and Field Mapping

on October 20, 2023 showed that there was a 3.24% overlap in bibliographic records between the two searches

Web of Science

The fragmentation of research on how to better educate children in crisis

of “education in emergencies” and “refugee education”.

Figure 2. Overlap of references/citations for education in emergencies and refugee education publications in figure 1.

Figure 1. Overlap of education in emergencies and refugee education publications

Out of 309 publications, there were only 10 (or 3.24%) overlapping records. Going further, we can also examine the publications along with their references (figure 2).

Of the 12,181 articles and references cited in these 309 publications, only 736 are in both searches. This is 6.04%, highlighting how the topics are clearly siloed with little intersection.

Ha Yeon Kim, Senior Research Scientist Daniel Woulfin, Senior Data Scientist

This is page 1/3 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

This lack of intersection could be caused by a number of factors including differences in academic disciplines, funders, research/practice partnerships, and history. Research with a framing of “Education in Emergencies” tends to be advocated for, funded, and supported by UNESCO and UNICEF and INGOs working with these UN agencies organized around the Inter-agency Network for Education in Emergencies (INEE). In contrast, research framed as “Refugee Education” is mainly advocated for, funded, and supported by UNHCR and other organizations supporting refugee populations. The split between these two subdisciplines is also apparent, with different groups of researchers focusing on “Education in Emergencies” research while others publish research with a framing of “refugee education” the “refugee education” literature having a longer history and three times as much peer-reviewed journal publications (as of October 2023, n=241) compared to “education in emergencies” research (n=83), which instead has a large number of gray literature: aka non-indexed white papers, policy briefs, and other reports. This fragmentation has led studies, research and efforts being siloed. While these parallel universes of evidence are known and acknowledged by all parties, coordination, collaboration, and integration efforts to improve the education of children in EiE contexts have been hamstrung by different organizational and funding priorities, political divisions, and competition across scholars, funders, and policymakers. This has led to a segmentation of effort that is more likely to result in redundant studies, create contradictory program and policy advice, and lead to less effective interventions and outcomes.Given the status quo of political, academic, and funding division, achieving a cohesive, systematic, and efficient evidence base and research practice to support education in conflict and crisis seems like a daunting task. However, we see there are two solutions that can consolidate and integrate the disparate and fragmented evidence base. What Might Work: one approach to counteract forces creating this fragmentation lies in synthesizing studies and data together to find best practices to systematically organize the extant evidence across sectorial and disciplinary silos. We offer two solutions to address the challenges above:

1. Systematic review and meta-analysis based on a transdisciplinary framework

Systematic review and meta-analysis are scientific, evidence-based approaches to synthesize evidence on research questions from different studies that are studying a similar topic. They consist of a scientific process that is replicable and reproducible. Systematic review includes data collection of secondary literature from bibliographic indexes and gray literature using boolean search techniques and citation mining to create a corpus of related works. Researchers then can analyze the corpus to identify answers to specific research questions and synthesize findings across the studies. Systematic search and reviews also allow for meta analyses, which systematically assess a specific question through obtaining quantitative summary estimates of an effect from multiple studies, examine heterogeneity of the findings, and assess generalizability of the findings from a group of studies to derive conclusions about that body of research.Such approaches are necessary to synthesize large amounts of academic studies to find best practices and common conclusions that can be implemented in the field whether that’s policy, medicine, computer programming, education, etc. As many have noted, policy actions should not generally be based on single studies or small numbers of selected studies . Performing systematic reviews and meta analyses on questions that transcend fragmented topics would help synthesize disparate studies into evidence based guidance for researchers and practitioners to put in practice in the field.

The Need for Systematic Reviews and Field Mapping

The fragmentation of research on how to better educate children in crisis

Ha Yeon Kim, Senior Research Scientist Daniel Woulfin, Senior Data Scientist

This is page 2/3 of this section. Click the arrow(s) below to go to the next page or back to the previous page:

17

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

To conduct such synthesis across studies in fragmented fields, we need a conceptual framework that provides a heuristic for organizing research by main drivers of learning that can make meaningful changes for children’s learning and wellbeing. Recently, the TIES team developed a conceptual framework for Education Research in Conflict and Crisis (ERICC) that provided such a unifying heuristic for the field. By focusing on these key drivers, we can identify and assess the strength of the evidence and effectiveness of the full range of existing strategies, practices, and policies that drive learning and wellbeing. In addition, the ERICC framework provides a structure to organize evidence on pre-existing conditions for success, potential barriers and challenges, and mechanisms on how it may work. This comprehensive framework, combined with systematic literature search and analysis, provides a more comprehensive, coherent and actionable evidence base that focuses on improving children’s learning and wellbeing, transcending disciplinary silos and funding trends. Leveraging this framework, data and evidence can be systematically organized into a systematic and usable knowledge base that provides a comprehensive and holistic approach to build up the emerging field of education research and conflict and crisis.2. Better data sharing and publication practicesTo maximize the use of actionable evidence for sustainable futures of populations affected by crisis, it is critical to enhance the usability of the existing data. Open and fair data sharing are therefore critical steps for ethical research on EiE and refugee education. Various organizations and scholars collect and use data for different purposes, but these data are very rarely shared or used cross-organizations and cross-projects, while “participants” are sometimes subjected to repeated data collection and sometimes feel obligated to respond to gain access to services. Open and fair sharing of data requires the acceptance of community agreed-upon standards across multiple fields, a process that has been ongoing and accelerating with data publishing requirements for peer reviewed journals. Fortunately, standards already exist for scientific data, one such standard is FAIR: Findable, Accessible, Interoperable, and Reusable. For Education in Emergencies, Refugee Education, and related topics this would mean:

• Findable: Data is stored in well-known and well indexed repositories. Data should also be stored in a repository that is indexed by the major internet search engines. In the social sciences this would include the Inter-university Consortium for Political and Social Research repository and Dataverse.
• Accessible: Data is licensed using a Creative Commons license. The dataset description (metadata) is clear and not licensed. Documentation, codebooks, and where possible, measures, are made available. Embargoes on data sharing are reasonable and the data are archived even if embargoed to make the metadata available for other scholars. When feasible the metadata is multilingual.
• Interoperable: The metadata uses global metadata standards from organizations like NISO and W3C standards to describe time and place. Keywords and other information are taken from community accepted controlled vocabularies like the UNESCO Thesaurus when feasible. This will make finding and combining datasets easier for researchers.
• Reuseable: Data is stored in an open software-agnostic format like csv, tsv, txt, etc so that it can be analyzed using more than one piece of software.

By having the data available we increase the pool of scholars throughout the world who can analyze it. Recent critiques have shown that an existential problem for Education in Emergencies and Refugee Education is one of positionality . The work is being done in the richest and most privileged institutions in the minority world with the support of global forces (UN agencies, funders, INGOs, governments). Making the data available using FAIR standards and creating multilingual datasets will increase the potential for meta-analysis and secondary data analysis by scholars who do not have the resources to collect the data themselves. This opens the field further to new perspectives that are closer to the data respondent. This counter-mapping or alter-use of the data combined with synthesizing existing studies can only help the field better educate children in crisis and conflict areas in the majority world.

The Need for Systematic Reviews and Field Mapping

The fragmentation of research on how to better educate children in crisis

Ha Yeon Kim, Senior Research Scientist Daniel Woulfin, Senior Data Scientist

This is page 3/3 of this section. Click the arrow below to go back to the previous page:

20

19,

18,

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Problem: As academic topics, Education in Emergencies and refugee education aim to shape education in crisis areas of the majority world however the ideas, cultural mores, and values are shaped in the minority world. This section will look at this issue as a whole, while leaning on our expertise and experience working in the Arabic speaking world. Underlying each academic topic are unseen or tacit interpersonal relationships and power structures that are sometimes known as the invisible college. We can bring forward these relationships and metrics using bibliometrics (the scientific study of academic literature and work). For example, according to the October 20, 2023 Web of Science database search and network analysis of coauthors using the R bibliometrix package, the United States of America scores highest in betweenness and PageRank centrality . This means that the USA is the most influential country and connector within the Education in Emergency coauthorship networks:

“Making the invisible visible."

21

Figure 1. Education in Emergency coathorship networks by country

Invest in methods to ensure greater inclusion and integration of tacit knowledge

Joyce Rafla, Senior Research Scientist Daniel Woulfin, Senior Data Scientist

Articles from the United States of America also have the most citations in the Web of Science search. It is more likely that an Education in Emergencies article cites work from the United States of America than any other country (see next page).

This is page 1/4 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

Figure 2. Number of citations in the Education in Emergencies literature by Country

The United States and other minority world countries clearly dominate the academic literature and research regarding Education in Emergencies. This has led to a marginalization of non-English speaking and publishing scholars, where the intended crises to be “fixed” take place. The cultures, mores, and values of the minority world dominate over the local knowledge of the majority world being studied. Making these invisible or tacit power dynamics/structures explicit is the first step to rectifying this power imbalance and ensuring that the research respondents’ perspective, language, and worldview are adequately and accurately represented with the diversity it entails and not simply as monolithic phenomena. Funding and publication disparities between the majority and minority world countries are one of the clearest signs of this unbalanced power structure. For example, there is limited overall funding available for social science programs in universities in Arab countries, and Arab researchers and research tend to be less successful in securing competitive research grants . This “chicken and egg” cycle is the result of a complex of factors, including histories of colonization and economic marginalization. However, through our literature review as well as through our work with Education Quality and Learning for All (EQUAL), several key institutional and systemic challenges that must be addressed to break this cycle are:

• one-off or limited quantitative and qualitative research methodology training opportunities
• limited university or institutional support for proposal development, grants management, and research ethics review processes
• publication biases in minority-world peer-reviewed journals ; and
• lack of recognition for Arabic-language journals.

“Making the invisible visible."

22

Invest in methods to ensure greater inclusion and integration of tacit knowledge

Joyce Rafla, Senior Research Scientist Daniel Woulfin, Senior Data Scientist

23

24

This is page 2/4 of this section. Click the arrow(s) below to go to the next page or back to the previous page:

We can quantify and measure this disparity in the Education in Emergency literature through bibliometrics, which is the statistical or computational analysis of academic literature. Using bibliometrics to understand coauthorship, citation networks, times cited, network analysis and other key indicators, we can measure and reflect on academic outputs and structures. We saw above at a national level at the start of this section, but we could also look at it from an individual, institutional, funder, journal, or other perspective.

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

What Could Work:

• Reducing disparities by elevating local tacit funds of knowledge : Transforming local tacit knowledge into explicit knowledge is a complex social process governed by power, positioning, and trust within social networks. Much of the rich, cultural funds of knowledge that have sustained human development across generations is ignored by minority-world scholars. At the same time, methods that incorporate such tacit knowledge, such as ethnographic methods, lack legitimacy within many Arab communities . One way to respond to this would be to implement Participatory Monitoring & Evaluation methods or Participatory Research Approaches that involve stakeholders including the people who are the target of said services or interventions.

To illustrate this idea, consider a funder who wants to focus on children’s social emotional learning while the implementing partner believes that without adequate literacy and numeracy skills, the targeted children will be at a disadvantage when joining schools. Since the flow of funds is from the funder to the implementer, that would create an imbalance of power that may deter implementers from challenging program designs that they believe will not work. They will ultimately accept what the funder wants to implement, convincing themselves that some impact is better than no impact. This suggests creating engagement around and ownership of a research agenda and activities in the MENA region will require prioritizing relationships as well as the use of diverse forms of knowledge construction and transfer. Without creating such engagement around ownership of culturally grounded research, we risk missing foundational pieces related to the research, one of which is language.

25

26

“Making the invisible visible."

Invest in methods to ensure greater inclusion and integration of tacit knowledge

• Address language disparities within research projects conducted in Arabic-speaking countries: In most of our projects within Arabic-speaking countries, we had to think in English and speak in Arabic. In other words, we envisioned the theory of change (ToC), the survey questions that match such ToC and the brainstorming plans all took place in English. This was mainly because the majority of the researchers on the projects were English-speaking. There were incidents where the majority of the meeting participants were Arabic-speaking, but we still had to switch to English. Once ideas were formulated, we translated to Arabic. We translated the survey questions and prepared the training materials all in Arabic. Language disparities led to a number of inequalities, including:

• Arabic-speaking colleagues having to carry the burden of responsibility for doing this work twice (in Arabic and English) without being compensated in the same way as their non-Arabic speaking colleagues. In our work, we observed that the mere idea of thinking or writing in another language changes the content in a way that makes it “foreign.” For example, in our social-emotional work, translating feelings from English to Arabic has proven difficult. Even among our Arab colleagues, they sometimes resort to English to translate emotional expressions between dialects when talking to each other. For example, the word يعيط in Egyptian means cry and in Levantine dialects means to call for someone. Thinking in a different language is not simply thinking; it is restricting oneself to the lexicon of that language and therefore, detaching oneself from the context we’re supposed to be “researching.” For example, we conduct a lot of parenting or early childhood research. There are Arabic words that reflect a different meaning than their English counterparts. A word like parenting, can be translated to tarbiya. But that word includes being raised as a good human and does not necessarily have to be confined to parents raising the child. For example, the Ministries of Education in Arabic-speaking countries are called Ministry of “Tarbiya” and Education. Similarly, the word “wellbeing” is always hard to translate. What does well-being really mean.

Joyce Rafla, Senior Research Scientist Daniel Woulfin, Senior Data Scientist

This is page 3/4 of this section. Click the arrow(s) below to go to the next page or back to the previous page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

Starting by thinking in the language of the context and then translating reflects a prioritization of the contexts in which we work. We were able to do that in QITABI where two of our colleagues developed the tools in Arabic and translated to English for review from our non-Arabic speaking colleagues.

• Due to the time needed to be spent translating, limited time for these colleagues to focus on their primary job of conducting research, therefore gaining less experience in conducting research.

Conclusion and hope for the futureAs a minority world institution, our role should be to not only listen to but to include local experts and native speakers as equals in EiE projects. We should also evaluate work and evidence from the minority world using advanced techniques designed to expose power structures and tacit assumptions. We can use these techniques to challenge prior conclusions and find strategies that make sense for local contexts. Instead they rely on international academic standards or NGO best practices throughout the research cycle. This is especially true in the Arabic speaking world. Exposing and rebalancing these power structures is critical for the field to move forward and be effective in a local context. Luckily, scholars like Ritesh Shah, Mario Novelli, Birgul Kutan, Francine Menashy, and Zeena Zakharia have already started this work under the umbrella of de-colonialism . Our role extends beyond listening and including global majority experts; we must shift the power structure to those committed to decolonizing the international development sector. This means hiring competent researchers from the global majority who believe in decolonizing the field, not just performative actions. For example, Joyce Rafla led the QITABI project with a decolonized approach, positioning our role as a resource rather than a research partner, and empowering the Lebanese team as experts, even if they lacked the precise terms to express their observations. Having conversations about power structures within international development is a difficult yet necessary step to actually fulfilling the mandates that we claim are important. If we are really about serving children all over the world, then we should not claim that we know what good parenting looks like in Egypt and Lebanon, for example, without having the humility to listen and learn from our colleagues in the majority world countries. It is only by decentering minority world institutions and individuals that EiE can become more effective in the myriad of unique local cultural-linguistic contexts in which it works. In addition, by using bibliometrics and other statistical techniques to measure the relationships within a topic we are also creating benchmarks from which to measure future trends and goals. This allows for initiatives and changes to be piloted and measured to calculate their effect. Transitioning EiE away from the minority world is feasible, but it will mean taking a long look in the mirror, challenging existing power structures, and moving away from internationalization and towards localization, challenging the “ego system” of minority world-driven research.

“Making the invisible visible."

29

28,

27,

Invest in methods to ensure greater inclusion and integration of tacit knowledge

Joyce Rafla, Senior Research Scientist Daniel Woulfin, Senior Data Scientist

This is page 4/4 of this section. Click the arrow below to go back to the previous page:

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: We’ve long joked that the goal of our work at TIES is to work ourselves out of a job; a sentiment that found tangible realization in our role during the UNHCR (HEA), an -funded program led by UNHCR, which aimed to support promising humanitarian education innovations to transition from successful pilots to programs that can operate at scale. TIES provided mentorship support to two cohorts of HEA participants. At the end of 3 years of mentorship that culminated with a 9-month course on adapting measures of holistic learning outcomes for a “finalist” group of three organizations operating in Chad, Jordan, and Uganda, all three organizations had piloted measures we had co-developed during the course in the 3 contexts they operated in and 2 are set to share their measures with the sector as global public goods! As I reflect on the success of this project and many others at TIES, a set of key principles emerge:

• Acknowledging and leveraging privilege: Historically, entities from the Global North have been and continue to be perceived as bastions of scientific expertise, often eclipsing the rich mastery, innovation and potential for further advancement in low-resourced communities. One cornerstone of our success lies in acknowledging the privilege inherent in our origins (individually and institutionally). The realization that equitable collaboration, built on mutual respect and knowledge exchange, forms the bedrock of sustainable progress became evident. Our commitment to bi-directional capacity-building highlights an approach to the sort of work we do that ensures that ownership of the projects and programs lay with the researchers, practitioners and program staff in the contexts in which we work.
• Building replicable training, measurement tools, and analytic methods for long-term impact: A fundamental aspect of our approach involved crafting tools that transcend immediate project goals. By ensuring the development and use of low/no-cost tools and replicable processes (item banks, user-friendly data analysis methods like the k-fold analysis R package, templates for data visualization and evidence reports, etc.) partner organizations are equipped to use them towards tackling future goals and challenges, minimizing the need for external assistance.
• Challenging comorbid inequalities: In pursuit of the enhancement of education systems, some of the moments I have felt our work has been most equitable and meaningful have been those that emerged at the intersections at which inequalities exist (from professional development to immigration to climate change). In doing so, we challenged systemic oppression beyond the direct ways in which our work does. I look back fondly on the occasions when, against resource constraints and administrative nightmares, we made room for early-career scholars' mentorship and professional development and actively challenged disparities like visa restrictions to ensure that stakeholders who are often left out of certain spaces have a literal seat at the table. These intentional initiatives broke the cycle of exclusion that many of us researchers and students from the Global South face, signaling an ardent dedication to not just the goals of the organization, but the wider system that poses multiple, co-occurring inequalities by majority-world present and aspiring researchers.

Our journey during the UNHCR Humanitarian Education Accelerator and many other projects stands as a testament to the power of a community-led, replicable, and holistic approach, emphasizing the indispensability of challenging inequities on multiple fronts. As I reflect on our successes, I recognize that sustainability lies in fostering self-reliant communities capable of navigating the complexities of education and beyond; in transforming educationalparadigms but also dismantling the structural inequalities that perpetuate disparities.

Humanitarian Education Accelerator

Education Cannot Wait

Whose Tools? Whose House?

Reflections on Power, Equity, and Decolonization in Educational Interventions and Innovations

Abiraahmi Shankar, Research Associate

This is page 1/2 of this section. Click the arrow below to go to the next page:

Even so, a gentle reminder echoes. “The master’s tools will never dismantle the master’s house.” - (Lorde, 1984)

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

What Could Work:

• Decenter ourselves and recenter the communities we serve: In reflecting on our interventions, innovations, and technological developments, a critical question persistently rings: Whose needs are we truly satisfying? The intersection of academia and technology often unwittingly perpetuates coloniality and endorses power structures that sustain Western dominance while inducing a bottomless appetite for its values and ideals . Educational innovations and technology, when detached from the lived experiences and cultural nuances of the community they serve, risks perpetuating existing disparities and reinforcing dominant knowledge paradigms. This inadvertently sidelines alternative forms of knowledge and diverse modes of learning, undermining the very essence of educational equality. A shift in the way we conceptualize and execute technological interventions could lead to more equitable innovations, reflecting these principles:
• Identifying ready-made solutions for problems vs identifying problems for home-grown solutions: In the realm of educational technology (which the TIES team journeyed into through supporting the development of a digital formative assessment game), the imposition of innovations from the global North onto non-Western communities is a recurrent challenge. This process, carried out with insufficient consideration for local and/or tacit knowledge (see Rafla and Woulfin’s “Making the invisible visible”: Integrating Tacit Knowledge), practices, and needs, may reify colonial power relations. Innovations in EdTech may be especially prone to perpetuating colonial assumptions and systems for several reasons, including: Western-centric content and pedagogy; digital divide and access issues; data ownership structures; and the economic models that support their development and sustainability. Academic and educational innovations, conceived multiple degrees removed from the communities it is intended to serve, run the risk of being inconsistent with the needs of communities, privileging specific forms of knowledge and learning over others, and setting such innovations up for success through dependency on Western institutions.
• Active community involvement in design, development and maintenance: the process of technology development cannot be a distant, isolated activity. Rather, it needs to be a collaborative endeavor involving the active participation of the communities who will be the end-users. By engaging local communities as co-creators, researchers, designers, and developers and weaving in the stories and imagery of the people they are designed for, we ensure that the technology is not a foreign imposition but a tailored solution that resonates with the unique needs and goals of a context. This requires a fundamental change in the culture of entrepreneurship originating in and funded by the minority world that works in the majority world.
• Ownership: for innovations and interventions to be adopted, communities and end-users must feel a sense of ownership. Technology is often developed in the Global North and then exported to the Global South. The risk inherent in this process is not merely the perpetuation of biases but also the potential creation of innovations that fail to address fundamental barriers to learning and development and ignore the tacit funds of knowledge that have sustained learning for generations in cultural communities. This is particularly harmful in educational settings, where the effectiveness of tools is intrinsically linked to user engagement, and end-users — often teachers/practitioners and/or students — often deal with significant time constraints and a plethora of other EdTech solutions. Fostering a sense of ownership can be a key tenet to ensuring that innovations “stick”; that they are not just temporarily adopted but integrated into daily practice in a meaningful way.

In the pursuit of the decolonization of knowledge, education, and innovation in low- and middle-income contexts, while the narrative is shifting from unilateral innovation to collaborative, community-driven design, it is imperative to place the communities we serve as the leaders of such interventions and ensure their agency in defining educational needs, preferences, and goals. Otherwise, whose house are we building? And with whose tools?

30

Whose Tools? Whose House?

Reflections on Power, Equity, and Decolonization in Educational Interventions and Innovations

Abiraahmi Shankar, Research Associate

This is page 2/2 of this section. Click the arrow below to go back to the previous page:

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: At NYU-TIES, the success of our projects heavily relies on the diverse talents and experiences of our staff, many of whom have lived and worked in the very contexts where our projects are implemented. This direct connection has been instrumental in advancing our localization efforts— a principle increasingly recognized as crucial for sustainable development. Localization, as advocated by agencies like USAID, emphasizes the engagement of local actors from LMICs (Low and Middle-Income Countries) and CACs (Conflict-Affected Contexts) in decision-making processes that impact their communities. This approach not only ensures that development initiatives are relevant and sustainable but also leverage local knowledge and networks which are invaluable in these settings .However, supporting the career pathways of such indispensable staff often hinges on our ability to provide work authorization in the US and effectively transfer funds to individuals and organizations in the contexts in which we work. Despite the infrastructure available at an international university like NYU, this process is resource-intensive and time-consuming, both for the administrative staff of TIES and NYU, and for our staff and partners. Moreover, visa sponsorship, which is crucial for retaining international talent, is not a common practice among organizations in our sector, setting NYU-TIES apart but also highlighting a significant challenge.The Problems:

• Visa Sponsorship: The necessity of visa sponsorship for staff from LMICs and CACs poses a financial burden, as such costs are often not covered by project budgets. Additionally, there is a significant administrative burden and core operational staff are not always funded to take on such work.
• Financial Transactions: The complexity of financial transactions with organizations and individuals in LMICs and CACs can render payments difficult, with high compliance barriers often set by the university and donors, such as requirements for liability insurance and annual audits. These challenges often mean that partners in the Global South can be made to wait months for approvals to come through or may be denied altogether. Livelihoods in low-resource contexts are often placed at risk. Often complicated work-around solutions need to be identified.
• Grant Allocation: Despite a pronounced commitment to local partnerships, a significant portion of funding still flows to larger, international organizations, thereby continuing the imbalance in global development funding.

What Could Work:

• To foster a more inclusive and empowering environment for local actors in global development, we recommend the following strategies:
• Immigration Support: Institutions in the EU and US should proactively sponsor visas and pathways to citizenship for talented employees from LMIC and CAC contexts. This would not only retain crucial talent but also enrich the institutional diversity and capacity with essential insights and experiences. Such funding is not typically allowed within specific grant proposals. In our experience, core flexible funding without restrictions was required to fund visa and immigration support. Core unrestricted funds are much more difficult to fundraise for than project-related grants.
• Capacity Building: There should be a concerted effort to enhance the capabilities of local actors through dedicated training in grant writing and financial management. Equipping local partners with these skills is essential for leveling the playing field and enabling them to compete

effectively for grants.

• Risk Tolerance: Funders and intermediary organizations need to be willing to accept higher levels of risk to support local actors who may

lack a robust administrative infrastructure. Embracing this risk – perhaps in a staged or sequenced way that allows for gradual reduction in institutional risk -- is key to transitioning from a dependency model (which perpetuates the status quo) to one of empowerment and self-reliance.

35

34,

33,

32

31,

“It’s hard to focus, because I might be asked to leave the country”

Invest in hiring and providing visa support for staff from the regions in which we work. Also, reduce administrative barriers to “localization”.

Roxane Caires, Project Managing Director

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: There are various ways in which TIES has developed and expanded partnerships. In some cases, we identify a funding opportunity and reach out to colleagues who we believe would be interested and an asset to the project; sometimes we are asked by a funder or colleague to partner on a new or existing project; at other times we may be asked to continue or expand on a project we are actively engaged in. The impetus for all of our partnerships is the work, but the success of these partnerships often depends on the strength of the relationships among the individuals doing the work. The Problem:

• In a sector where resources are predominantly donor-driven and limited, an inherent competition exists, which can hinder the development of strong relationships unless well managed.
• Funding cycles are usually short, while systemic change requires a long-term commitment. Frequent staff turnover within organizations also impacts the continuity of professional relationships.
• How much we like working together is not a grant deliverable

What Worked: While our partnerships, even in the most successful of projects, have not always been “sunshine and rainbows,” there are a few things we have learned along the way that have contributed to meaningful professional relationships and impactful outcomes.

• Prioritizing the work, then figuring out the funding: The projects that resonate the most are those where a partner approaches us with a specific need, something TIES is uniquely equipped to address. This approach may not always fit neatly within a traditional Request for Proposals but has proven effective in establishing meaningful engagements. While this is clearly a longer-term strategy, this also helps in communicating the significance of our work to donors and ensuring all partners are truly committed to the project’s success. This has worked particularly well when we have been able to maintain a strong partnership and shared vision, and actively fundraise with our partners for continued and future work.
• Changing the backdrop - going shoe shopping in Istanbul. In an anecdote from one of our strong partnerships, we faced significant barriers getting two government offices to collaborate. We decided to hold a workshop in a relaxing setting abroad, which included enjoyable dinners and respectful exchanges. One evening, I accompanied colleagues interested in shopping to a local shoe store and later to a shisha bar for tea and long conversations. This experience not only broke down barriers but also cemented a commitment to collaboration that continues to influence their work. Thankfully, this story is indicative of many of the partnerships we have.
• Listen, really listen - then be willing to adapt. Working globally across various sectors, we invariably encounter a multitude of languages and perspectives. As "experts," we often fall into the trap of thinking we know best, influenced by biases that favor Western academic viewpoints. Recognizing these power dynamics is crucial for ensuring that our partnerships are genuinely equitable, valuing all partners' knowledge and experience equally. Some micro examples of this include: paying attention to who isn’t speaking in meetings or calls; recognizing how power differences affect feedback honesty; understanding incentives: be aware that staff in LMICs may align with minority world led projects to advance careers; consider power dynamics at intersections of socioeconomic status, gender, seniority, age, ethnicity, race, and language; employing indigenous methods like talking circles to reduce speech dominance hierarchies. Moreover, effective science is not about rigidity; it thrives on adaptability. Applied science must be relevant and responsive to its context, necessitating that we, as researchers, tailor our methods to fit

real-world applications. We are grateful for partners who challenge us with "why not?"—their patience and persistence not only enhance our scientific rigor but also enrich our personal growth.

“We care about you first and foremost”: Building Trust-Based Partnerships

Invest in research-practice-policy-donor partnerships, based on mutual trust and capacity strengthening, that challenge power differentials

Roxane Caires, Project Managing Director

This is page 1/2 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

There are also some more technical recommendations that are worth sharing:

• Insist on clear dialogue and agreement of project scope with all engaged partners present prior to committing to participate in the project
• Some resources/processes that can help better clarify roles, responsibilities, and approaches include:
• Discuss different levels of engagement with partners using a tool like the “Wheel of Engagement” to facilitate the conversation
• Develop a Memorandum of Understanding (MoU) that all partners agree to.
• Request a letter of intent to participate in the project from all parties.
• Conduct a key informant interview with the inviting partner to better understand the approach they plan to take and history of the project or partnership with other partners
• Map the pros and cons of continuing the project, and be willing to let the project go if the cons outweigh the pros
• Determine what risks you are willing to take to continue the project, identify clear, observable indicators and decide on responses

“We care about you first and foremost”: Building Trust-Based Partnerships

Invest in research-practice-policy-donor partnerships, based on mutual trust and capacity strengthening, that challenge power differentials

Roxane Caires, Project Managing Director

This is page 2/2 of this section. Click the arrow below to go back to the previous page:

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: For the past three years, as a Research Scientist at NYU Global TIES for Children, I have collaborated with a diverse array of governmental and non-governmental agencies, in designing, strengthening, and analyzing Monitoring and Evaluation Systems (M&E), primarily related to Holistic Children Outcomes. I believe that our approach to designing measurement tools at NYU Global TIES for Children acts as a form of system intervention in and of itself. Below, I identify principles that form the foundations of our design, collection, analysis and dissemination processes. In addition to sharing reflections, challenges, examples, learnings, questions, and resources that my colleagues and I have encountered thus far. I will use the plural form when referring to these topics, as I firmly believe this 'work in progress' is a collective effort. They say it takes a village to raise a child; similarly, it takes a village to develop effective Monitoring and Evaluation systems. Therefore, I adopt a collective perspective in understanding this process. Principles for action in favor of human development, education, and measurementI. Integrating Developmental Science and Applied Research: A Child-Centric Approach: As researchers, we acknowledge and promote the interdependence of decades of robust findings in developmental science and applied work . We prioritize children's holistic development and learning trajectories, recognizing the interconnectedness of various aspects of their development, including family dynamics, and environmental influences . To fully understand their developmental course and co-build equity within systems, we must explore the “why?” . Therefore, our work has aimed to meaningfully design measurement tools that focus on context-relevant developmental, risk and protective processes , to collect, and utilize data to guide policy responses that can build on accumulated research knowledge. This has allowed us to provide children and their families not only with a deeper understanding of children’s flourishing within nurturing (or non-nurturing) systems, but also with insights on enhancing the settings in which they live and transforming their daily life experiences. II. Education is the practice of freedom : We hold that education is both a transformative and empowering tool . It enables individuals to acquire capabilities that allow them to make choices and pursue the life they envision . To be fully prepared to pursue their envisioned life freely, children need to develop fundamental capabilities that allow them to survive, including academic skills such as literacy and numeracy, and social, emotional and cognitive skills, such as critical thinking and empathy . As the COVID global pandemic has underscored, these latter skills are essential for becoming functionally integrated human beings. We believe that educational services should act as a conduit through which children can practice and gain the freedoms necessary to realize their full potential. III. Measurement is a Collaborative Process More Than the Measure Itself: As community-engaged scholars we advocate for reflection-action-reflection processes and community participatory research strategies in our collaborations with governments and local agencies. Existing evidence suggests that a sense of community is positively correlated with levels of participation, interaction, collaboration, and shared resources . This correlation fosters enhanced ownership, empowerment, and networking , which are crucial for the sustainability and strengthening of M&E systems. In our partnerships, we strive to co-produce knowledge, cultivating a sense of community among scholars engaged in creating sensitive and high-quality assessments. The process of co-constructing measurements is as important as the measurements themselves. To avoid the pitfall of merely distributing a list of assessments - akin to storing information in blank containers - without critical examination, we encourage iterative co-creation processes. These processes incorporate feedback loops and a community of learners, thereby facilitating continuous improvement.

“Measurement is an intervention in and of itself.”

36

37

38

40

39,

Prioritize and advocate for process as much as for products

42

41

43,

44

Evelyn Seminario, Research Scientist

45

This is page 1/4 of this section. Click the arrow below to go to the next page:

41

46

48,

49

47,

50

51

52

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Problem: Significant challenges persist despite increased global interest in generating meaningful evidence on children’s holistic learning outcomes. We have identified five key challenges in developing, testing, and implementing a national study of holistic learning:I. Defining 'Holistic Learning': Understanding the Intersection of Social-Emotional and Academic Skills: Social-emotional and academic skills are akin to two sides of the same coin; they are interrelated and cannot exist independently. A failure to recognize this interdependence often leads researchers and policymakers to favor one over the other. Despite the growing focus on children and adolescents' well-being and the widely recognized burden of the pandemic, there is still a significant gap in understanding the nuances of social and emotional skills. These skills are undoubtedly related to well-being but are not inherently the same. For instance, while there is extensive research on academic loss and internalizing and externalizing symptomatology, less is known about how skills such as waiting one's turn, understanding others' feelings, or promoting restorative conflict resolution have been impacted post-pandemic. Although there is consensus on the general impact, the specific nuances remain less understood. II. Aligning Definitions: The Challenge of Establishing Universal Taxonomies: There are no globally or nationally accepted taxonomies of social and emotional skills. What we aim to measure and how we design an instrument highly depends on the study's context and purpose. Undertaking this at scale, which is a relatively new approach in the field, presents specific challenges, such as consistently defining what we intend to measure across multiple target groups and building consensus among various stakeholders. III. Data Collection Amidst a Pandemic: Adapting Traditional Methodologies: SEL assessment tools have traditionally been designed for paper/pencil self-reporting and have targeted respondents who are easily accessible because they are physically present in schools. However, school closures, lockdowns, and social distancing regulations during the pandemic have posed significant challenges to this traditional approach. In many low- and middle-income countries, limited and inequitable internet access restricts our ability to reach many students and families, particularly those in lower-resource contexts. While there are both advantages and disadvantages to every data collection method, one critical challenge we face is that we have developed a set of standardized procedures focusing more on how to collect quality data rather than considering how to collect such data effectively in times of greater need. IV. Timeliness in Tool Development - Balancing Speed and Quality: Developing a tool that fits contextual characteristics, collecting data at scale, generating meaningful evidence of children's holistic learning outcomes, and communicating with various stakeholders are intensive and time-consuming tasks, even under the best of circumstances. However, in the context of pandemics like COVID-19 the window for policy decisions and implementation is often brief, necessitating rapid action. Key steps include reaching agreements on what to measure, designing assessments, allocating responsibilities for data collection, analysis and processing results, and effectively communicating these results to different audiences at specific moments of policy or practice opportunity. However, a prolonged process may miss the opportunity to provide timely information during brief periods of policy decision-making, thereby reducing the potential impact on these decisions. V. Navigating Political Instability - Building Sustainable Relationships: Working at scale necessitates a strong relationship with national education systems, especially with those in leadership and decision-making roles. Over the past two and a half years, many countries have experienced significant political instability. Since our project's inception in Peru, eight ministers under four presidents have overseen the Ministry of Education. Such political instability and high turnover rates in government positions add more burden to the task of designing and implementing measurement tools that align with national interests and are sustainable, given the changing priorities and interests of policymakers across various administrations.

“Measurement is an intervention in and of itself.”

Prioritize and advocate for process as much as for products

Evelyn Seminario, Research Scientist

This is page 2/4 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

What Could Work - Strategies for strengthening educational monitoring systems for social and emotional skills and well-being at national scale: Strengthening monitoring systems is one way to translate applied research into actionable policies. Without timely, continuous, and quality data, policymakers lack the necessary tools to make informed decisions to improve children's holistic development. In our endeavor to address the challenges detailed above, we have developed four strategies:I. Building consensus and clarity around the purpose, content, and timeline of assessment: To overcome the challenge of incorporating a holistic learning approach and aligning various aspects of social-emotional skills, we employed a strategy to define the skills to be assessed and clearly identify the intended purposes of data usage. Such clarity ensures the data's relevance and increases its likelihood of being effectively utilized. Our process for defining priority skills is iterative, drawing on diverse sources like policy documents, curricular guidelines, qualitative data from various stakeholders, and scientific literature. Investing time in building consensus and defining priority skills is crucial for strengthening systems. This approach sets clear expectations for the assessment, defines outcome boundaries, facilitates communication of findings across different offices focusing on social and emotional skills, bridges diverse interests within government agencies, enhances ownership of the results, and promotes accountability for their improvement. For instance, in our collaboration with the Peruvian Government, we aligned constructs of interest with their curriculum. This alignment focused on the primary objective of the measurement initiative, identifying systems that would find the results actionable and determining the extent of their applicability. Our selection of constructs was grounded in evaluating a set of trainable and evidence-based skills to enhance academic learning and help students reach their holistic potential. II. Co-constructing and iteratively refining a mixed-method battery of assessments to provide meaningful data: To increase the ownership of the assessment and in turn, the results, we have found that it must be a co-constructed and scaffolded process. However, despite our best efforts, we acknowledge the impracticality of completely reinventing assessments. Therefore, in our journey, we have learned that co-constructing the M&E system by integrating previously tested assessments with new items, phrasing, and formats tailored to specific efforts significantly enhances both the ownership and quality of the overall measures. In our collaborative construction process, a key component was incorporating free listing questions at the end of each construct. This approach refined the definition of each construct and tailored it to the context, enhancing our ability to capture the concept and increasing content validity. These free-listing questions proved to be the pivotal element needed to advance our collaborative process. To analyze these questions effectively, we developed a streamlined system rooted in grounded theory. For example, in our work in Peru, we developed a script to foster informal conversations, specifically designed and adapted for phone-based data collection. This strategy aimed to enhance engagement, reduce bias, and increase data variability. Additionally, at the end of each set of construct-related questions, we included some free-listing options. For instance, exploring externalizing behaviors, we asked, 'How does your child behave when angry or upset?' ' These open-ended questions allowed us to refine the wording in our items and/or add more, thereby increasing content validity.

“Measurement is an intervention in and of itself.”

Prioritize and advocate for process as much as for products

Evelyn Seminario, Research Scientist

53

This is page 3/4 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

III. Committing to promoting equity, open science, and global public goods throughout the assessment and analytic pipeline: We recognize the urgent need for policymakers to make informed decisions and understand that developing quality monitoring systems requires time, commitment, and collaborative efforts of immense value. To facilitate this, we advocate for the use of open-source resources to enhance the uptake of public goods developed and ensure that processes can be replicated, even in our absence. For example, our partners collected data using KoBo Toolbox, an open-source system for mobile data gathering. For analysis supported by NYU-TIES, we process the data through a pipeline engineered for rapid turnaround. This approach significantly reduces the time required for data cleaning and management – from six months to approximately one-month post-data collection. This efficiency enables access to verified, high-quality data in real-time for analysis and refinement. For instance, this process allows us to support data dissemination across multiple governmental offices. The indicators we developed related to the well-being of Peruvian children were instrumental in rationalizing the decision to reopen schools after two years of closure. IV. Strengthen the capacity of practitioners and policymakers to develop or adapt, implement, and analyze data from high-quality assessments: One of the most significant challenges faced by education systems is the high turnover rate across most civil servant positions. To address this issue, we actively promote learning among our partners and collaborators. We engage with dedicated public servants and policymakers to build, scaffold, and acquire knowledge that is applicable across various settings. These trainings, grounded in learning design principles, utilize diverse resources and interactive learning strategies. They are designed to be flexible and easily adaptable for both remote and on-site forms of data collection. For example, in our work across Lebanon and Peru, we have successfully developed a series of trainings with a foundational emphasis on measurement, the development of measures, and the practical application of statistics. Our goal is to demystify the complexities of psychometrics by awakening our innate curiosity and eagerness to learn with a series of materials that enhance play-learning principles. In summary, the common thread in the process detailed in this paper is that measurement is more than the measure itself. To develop valid, reliable, and fair assessments that provide both quality and timely data, we must view it as a process of intervention. Items, data, and reports are not impactful unless we continuously consider their usefulness, purpose, and—most importantly—the impact of the process of developing the measure itself.

“Measurement is an intervention in and of itself.”

Prioritize and advocate for process as much as for products

Evelyn Seminario, Research Scientist

This is page 4/4 of this section. Click the arrow below to go to the previous page:

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: Data management for time-bound projects in crisis settings is difficult, to say the least. There are frequently few or no guarantees that are in place where off-the-shelf technologies from the "Modern Data Stack" can be employed: usually, there is no guarantee of connectivity to enable live data streaming; data system structure is selected primarily for flexibility over rigidity; and data sources are frequently changing. The greatest difficulty we experienced at Global TIES across projects was that of a lack of investment in data system robustness, creating situations where there were several single points of failure. The following sections will outline several vignettes surrounding common issues we encountered.Challenges:

• Lack of Metadata Infrastructure: Research studies are only as useful as the metadata around them, which include things like program enrollment registers, monitoring and evaluation records, and various other supplemental materials which provide necessary context for the comprehension of the desired data. (especially with external research partners). While there may be investment employed to ensure the rigor of target data collection, we rarely experienced such efforts for metadata. This is not wholly unexpected: all sorts of issues can crop up during the lifetime of a data collection project, so the system to capture the metadata must be flexible and easy to understand to most field team members. This requires extensive collaboration to understand how different sources and types of data may be useful to each partner in the collaboration.
• Unfortunately, too much flexibility can render data useless. For example, we experienced several cases where records of teachers changing sites were overwritten in the Education in Emergencies: Evidence for Action (3EA) project, and we were only able to identify this issue with our own version control system that maintained historical copies of the same file sent to us from the field team. Had we not retained multiple versions of the file on our end, the data structure for this particular project would have been corrupted due to its hierarchical nature. Moreover, we experienced a situation where two different kinds of observational units -- teachers and classrooms -- used the same identifier, implicitly stating that teachers would never be associated with more than one classroom.

• Systemic Brittleness and "Data Extraction": Time pressures of data collection or regulatory issues promote the use of technology with as few guardrails as possible to maximize flexibility. This has the unfortunate side-effect of creating single points of failure which has the seemingly paradoxical effect of creating brittleness over flexibility. Usually, the software of choice for implementing data systems is Microsoft Excel, which is a deceptively powerful tool for some use-cases like analysis but disastrously underpowered when it comes to data management for medium to large scale research. Without being coupled with a version control system or systematic guardrails like implementing Type-2 Slowly Changing Dimensions, there is a significant risk that using Excel can lead to necessary context being lost to time, making interpretability of results for both researchers and practitioners alike much more difficult -- if not impossible.
• This lack of procedural protections is not restricted to the technical realm but also the human realm. Too often there are multiple research projects deployed to the same region by different researchers with no data sharing to understand the questions the individual projects are attempting to answer. The result is that data collection from the same time can yield very different results simply from the fact that the collection methodology was different.
• This problem is, of course, not restricted to the data realm but is a larger issue within research more broadly. These siloed research projects, while nominally trying to better the communities that they study per the humanitarian mandate, contribute to the practice of "data extraction": hoarding data collected from marginalized communities until all potential academic outputs are created for the career advancement of researchers and fundraising capabilities of NGOs. I recall a call to action from a representative of the South Sudanese ministry of education at the 2023 INEE Data and Evidence Summit where he lamented the messaging from partner NGOs and research groups, focusing on their own successes while not offering actual coherent guidance to governments.

“There cannot be a single point of failure.”

Integrating data engineering principles and prioritizing an ethical and open science

Patrick Anker, Data Manager

This is page 1/2 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

• Data Sharing: The data situation for crisis contexts is not all doom and gloom, however. These issues, especially concerning the research duplication efforts and "data extraction", are widely understood in the Education in Conflict and Crisis (EiCC) community. In fact, these two notes were the primary messages in the INEE 2023 Data and Evidence Summit. Thanks to the efforts of several organizations marshaled by UNESCO International Institute for Educational Planning, there are conceptual and guidance notes , to inform practitioners about how to engage governments and other groups operating in similar contexts. These materials are a very welcome first step to pivoting this field to address under investment in data systems for practice and policy in the context of NGOs, governments, and research organizations alike.
• Discussions about the technical aspect of data sharing are still ongoing, however. Institutional sharing is only as useful as the medium through which materials are shared. My former colleague Dan Woulfin and I made these points while being members of the INEE Data Working Group. Specifically, EiCC is a research domain borne out of necessity (SDG 4) and previous efforts in refugee education investigation. However, much of the work since the creation of the field has not referenced continuing efforts outside of EiCC (see Kim and Woulfin’s The Need for Systematic Reviews and Field Mapping). From a technical perspective, there is no common ontology of terms that can connect work done in the EiCC realm to other fields. The lack of this ontology contributes to the project duplication efforts because groups that may not work in the same circles could be performing similar interventions or observations but using differing language. In a field that purports to be interdisciplinary to respond to the many intersectional crises, EiCC has done a fantastic job of staying within its own discipline.

Next steps: This research domain has the unfortunate reality of being couched in academia and the iNGO ecosystem which is slow to make changes that may not mesh with the operational status quo. I worry that there is not enough institutional interest in the academic and iNGO ecosystem to properly engage in developing the necessary but boring infrastructure to adequately handle data sharing and discovery issues. However, if each member understands the structural limitations and practices that hamper these improvements, then progress can be made. As shown with the data sharing efforts from the INEE Data and Evidence summit, the building blocks for addressing these issues identified over my tenure at TIES are being placed. Once institutions agree upon sharing practices to de-duplicate work and (hopefully) limit "data extraction" practices, then work on the technical must be highlighted. This does not mean re-inventing the wheel: there are tools and systems already commonly used, such as the UNESCO Thesaurus in conjunction with the UNICEF Data Team, that could be stitched together to build more FAIR (Findable, Accessible, Interoperable, and Reusable) data. However, robustness must also be considered to reduce single points of failure. This is a collaborative field, which means many people could work together on one element. If there is no backup or system around that element, the whole field comes to a halt. My work at TIES has also shown what can happen when all of these issues are addressed and proper collaboration with governments happens to improve the lives of children. In our 2020/2021 measurement development study partnered with the Ministry of Education of Peru, we were able to rapidly examine remote learning's effects on children and families during the COVID pandemic, and because we invested in these systems, from the technical like metadata management and systemic robustness to the data sharing-focused like training and FAIR data practices, we were able to have our work cited in the prime minister's address in “Ministro Cuenca : ‘La Enseñanza Virtual Puede Ser La Nueva Revolución En La Educación’” (“Virtual Instruction can be the New Revolution in Education”) . Moreover, our work was adopted into the ministry, allowing these efforts to continue helping governments beyond the completion of the project. This is a future I can hope for, and with some institutional investment, it can become a reality broadly

54

“There cannot be a single point of failure.”

Integrating data engineering principles and prioritizing an ethical and open science

Patrick Anker, Data Manager

This is page 2/2 of this section. Click the arrow below to go back to the previous page:

55

Next reflection:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

The Context: When I first joined the team at NYU Global TIES for Children, I was struck by a critical challenge: our groundbreaking research was getting lost in translation. Despite the depth and potential of our findings in education and child development, there was a significant disconnect when it came to communicating these insights to those outside the academic sphere. This communication gap was more than just a stumbling block—it was a barrier preventing us from fully engaging with the stakeholders who could help catalyze real change. Our researchers were experts at conducting thorough and meaningful studies; however, the skills needed to translate this research into accessible, compelling narratives were lacking. Recognizing this, we saw an urgent need to develop communication strategies that could bridge the gap between our research expertise and the practical needs of our diverse audience. This realization led to a concerted effort to enhance our approach, ensuring that the valuable insights gained from our research were not only shared but also understood and utilized by those who needed them most. The Communication Challenges:

1. Building Trust: This gap constituted a barrier to building trust with stakeholders, crucial for empowering decision-makers with research-based evidence for informed policy-making that is culturally and contextually relevant.
2. Maximizing Impact: It also limited the potential impact of our work by restricting the engagement and comprehension of those who stood to benefit most from our research.

In response, we crafted a strategy that extended beyond standard academic articles and presentations aimed solely at academic audiences. We began creating side communication products tailored specifically for funders, practitioners, and the general public. What Worked: We progressively implemented a comprehensive communication strategy that demanded close collaboration with researchers to simplify the complexities of academic studies, remove scientific jargon, and make the findings legible, accessible, and applicable to a wide range of stakeholders. Strategy 1: Enhancing Visibility and Attracting Partnerships. To increase our visibility and influence, it was essential to maintain a robust online presence. At NYU-TIES, we employed a multi-channel communication strategy that encompassed our revamped website, social media channels, and open-access academic publishing. Strategy 2: Tailoring Communications to Diverse Audiences. A core strength of our strategic communication approach was our ability to effectively engage with a diverse audience. This included beneficiaries, funders, collaborators, and a broader informative audience. We simplified complex research findings through engaging content on social media and press releases, which raised public awareness and supported community engagement and advocacy efforts. Each group required a unique engagement strategy that resonated with their specific roles, interests, and traits. We diversified our communication products, including briefs, blogs, explainer videos, infographics, policy briefs, concept notes, podcasts, and press releases. These tools allow us to translate research findings into various formats that effectively speak to each target audience.

“Who is your audience?

Prioritize strategic communications and engagement early, often, and to a range of stakeholders

Douha Boulares, Communications Manager

This is page 1/2 of this section. Click the arrow below to go to the next page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

Strategy 3: Optimizing Dissemination and Visibility. We implemented a strategic approach tailored to the audience and unique features of each platform to optimize our presence and engagement effectively:

• Platform-Specific Content Strategy:
• LinkedIn: Targeting professionals, donors, NGOs, and academics with in-depth articles, research findings, whitepapers, and professional insights.
• Instagram: Catering to a younger, visually-oriented audience with infographics, short videos, stories about fieldwork, and interactive content like quizzes and polls.
• X/Twitter: Focused on real-time updates, news, and quick insights, including brief research highlights and engaging in trending educational topics and discussions.
• Consistent Posting Schedule: A dissemination calendar on ASANA, accessible by all TIES staff, ensured regular posts during peak activity times. We regularly reviewed and adjusted the calendar based on engagement analytics to optimize timings, especially around international days.
• Engagement and Interaction: Actively engaging with the audience through comments, direct messages, and interactive posts.
• Cross-Promotion and Integration: Cross-promoting content across platforms to increase reach, such as sharing LinkedIn articles on Instagram and Twitter.
• Content Diversification and Adaptation: We significantly diversified our communications outputs to appeal to each platform’s unique audience, creating visually engaging content that simplified complex research findings for broader audiences.

Strategy 4: Outreach Amplification. To further enhance our dissemination strategy, we incorporated external partnership communications product development, internal amplification, and strategic outreach efforts. These elements were crucial in elevating our visibility and engagement with our target audience. We collaborated with external organizations and partners for cross-promotion, sharing each other's content, co-creating posts, and featuring each other's initiatives. We also partnered with influencers and thought leaders in our field for guest posts and takeovers, particularly on Instagram and LinkedIn, leveraging their follower base. Internally, we coordinated closely with NYU’s Public Affairs team and the Institute of Human Development and Social Change (IHDSC) to amplify our content across their social channels, websites, and newsletters. We ensured our major announcements and content were included in the university's broader communication efforts. We actively engaged with the Inter-agency Network for Education in Emergencies (INEE) by sharing our content on their platforms and participating in their discussions and forums. This provided us with access to a highly relevant and engaged audience. Conclusion: These efforts have enabled us to reach a broader audience and solidify our standing as thought leaders in the field of education in emergencies. Reflecting on these initiatives, it is evident that our communication efforts have made our research findings both accessible and actionable, serving as a vital resource for those directly involved in implementing educational strategies, influencing policies, and supporting children's development. This strategic engagement has built strong relationships, fostered collaboration, and enhanced the overall impact of our initiatives, significantly advancing our mission to improve education and child development in emergency and crisis-affected environments.

“Who is your audience?

Prioritize strategic communications and engagement early, often, and to a range of stakeholders

Douha Boulares, Communications Manager

This is page 2/2 of this section. Click the arrow below to go back to the previous page:

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

CONCLUSION

Moreover, the importance of strategic communication cannot be overstated. Our efforts to bridge the gap between academic research and practical application have involved developing tailored communication products for diverse audiences. This has included creating accessible and engaging content for funders, practitioners, and the general public, thereby maximizing the impact of our research and building trust with key stakeholders.Finally, addressing the structural inequities and power dynamics within the field is crucial. This involves decentering the dominant narratives and practices of the minority world and recentering the voices and expertise of local actors in the majority world. By doing so, we can ensure that educational interventions are not only contextually relevant but also equitable and empowering for the communities they are intended to serve. In conclusion, the insights gained from our decade-long work underscore the need for a holistic, inclusive, and culturally responsive approach to education in emergencies. By fostering strong local partnerships, leveraging diverse research methodologies, and prioritizing strategic communication and open science, we can significantly enhance the educational outcomes for children in some of the most challenging environments globally. Our ongoing commitment to equity and decolonization in research and practice will continue to drive our efforts towards creating more resilient and effective education systems for all children, regardless of their circumstances.

Based on the extensive review and synthesis of research conducted by NYU Global TIES for Children over the past decade, it is clear that research in low- and middle-income countries (LMICs) and education in emergencies (EiE) requires a multifaceted and culturally responsive approach. Our findings highlight several critical areas for improvement and innovation to better support children's holistic development in conflict and crisis contexts.Firstly, a robust and contextually grounded developmental science is essential. This involves expanding and integrating diverse research methodologies that are sensitive to the unique cultural and contextual needs of the communities we serve. Our work with the EQUAL network project, for instance, demonstrated the value of engaging scholars from sub-Saharan Africa and the MENA region in leading roles, thus fostering local capacity and ensuring that research is both relevant and impactful. Secondly, the transformation of scientific insights into tangible impacts necessitates strong partnerships, data engineering, adherence to open science principles, and strategic communications. By collaborating with a wide range of stakeholders, including local and international organizations, policymakers, and educators, we have been able to implement educational interventions that are both effective and sustainable. Our commitment to open science has also ensured that our findings are accessible and reproducible, fostering a culture of transparency and accelerating the dissemination of knowledge.

We hope that these learnings will inform and inspire the broader field, contributing to a future where every child has the opportunity to thrive, even in the most adverse conditions.

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

ABOUT US

NYU Global TIES for Children (NYU-TIES) is an international research center embedded within New York University. Our mission since 2015 is to contribute to a robust and culturally-grounded science for program and policy action that promotes children’s holistic learning and development in low- and middle-income countries and crisis-affected contexts. The scope of this white paper primarily, though not exclusively, covers projects funded by Porticus or those within the realm of primary education. Consequently, it does not extensively address the broader spectrum of TIES work, particularly our efforts in early childhood development (for a more review of early childhood development work, please review Goodfriend, E., Wuermli, A., Hiott, C, Trang, K., Iqbal, Y., & Castelyn, J. (2022). While many of the efforts reflected in this paper were funded by Porticus the results, interpretations, and recommendations included in this publication are those of the authors and NYU Global TIES for Children, and do not necessarily reflect Porticus’ viewpoints.

Suggested citation: we follow the Contributor Roles Taxonomy (CRediT) model in order to better acknowledge individual author contributions and facilitate collaboration. Carly Tubbs Dolan, Abiraahmi Shankar, Roxane Caires, John Lawrence Aber, Hirokazu Yoshikawa, Ha Yeon Kim, Joyce Rafla, Dan Woulfin, Evelyn Seminario, Douha Boulares. “Supporting Children’s Holistic Development in Contexts of Conflict and Crisis: Lessons from a decade of striving towards research that is effective, equitable and sustainable,” June 2024. NYU Global TIES for Children, New York. CRediT author statement: Conceptualization: Carly Tubbs Dolan, John Lawrence Aber, Roxane Caires Visualization: Carly Tubbs Dolan, Abiraahmi Shankar Writing: Carly Tubbs Dolan, Roxane Caires, Hirokazu Yoshikawa, Abiraahmi Shankar, Ha Yeon Kim, Joyce Rafla, Dan Woulfin, Evelyn Seminario, Douha Boulares, John Lawrence Aber Review & editing: Carly Tubbs Dolan, Roxane Caires, Abiraahmi Shankar, Hirokazu Yoshikawa

55

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

REFERENCES

1. Borazon, Elaine Q., and Chuang, Hsueh-Hua, “Resilience in Educational System: A Systematic Review and Directions for Future Research,” International Journal of EducationalDevelopment 99 (2023): 102761, accessed March 2, 2024, https://doi. org/10.1016/j.ijedudev.2023.102761 2. United Nations Educational, Scientific, and Cultural Organization (UNESCO). “Building Back Resilient: How Can Education Systems Prevent, Prepare for and Respond to HealthEmergencies and Pandemics?” Education sector issue note:. https:// unesdoc.unesco.org/ark:/48223/pf0000375278.3. Dryden-Peterson, Sarah et al., “The Purposes of Refugee Education: Policy and Practice of Including Refugees in National Education Systems,” Sociology of Education 92, no. 4 (2019):346–66, accessed March 2, 2024, https://doi. org/10.1177/0038040719863054 4. Tubbs Dolan, Carly, et al., “Supporting Syrian Refugee Children’s Academic and Social-Emotional Learning in National Education Systems: A Cluster Randomized Controlled Trial ofNonformal Remedial Support and Mindfulness Programs in Lebanon,” American Educational Research Journal (2021): accessed March 2, 2024, https://doi. org/10.3102/00028312211062911.5. Yoshikawa, H., Dryden-Peterson, S., Burde, D., & Aber, J.L. (2022). Education for refugee and displaced children. In M Suárez-Orozco & C. Suárez-Orozco (Eds.), Education: The globalcompact. New York: Columbia University Press.6. UNESCO International Institute for Educational Planning. “Conceptual Framework for Education in Emergencies Data,” International Institute for Educational Planning (2023):https://unesdoc.unesco.org/ark:/48223/pf0000388203/PDF/388203eng.pdf.multi. 7. Amartya Sen, “Development as Capability (Expansion),” in Readings in Human Development, ed. Fukuda-Parr S (New Delhi and New York: Oxford University Press, 2003).8. Goodfriend, E., Wuermli, A., Hiott, C, Trang, K., Iqbal, Y., & Castelyn, J. (2022). Delivering quality research in culturally dynamic, conflict-affected contexts. NYU Global TIES forChildren Center.9. Dammann O. Data, Information, Evidence, and Knowledge:: A Proposal for Health Informatics and Data Science. Online J Public Health Inform. 2019 Mar 5;10(3):e224. doi: 10.5210/ojphi.v10i3.9631. PMID: 30931086; PMCID: PMC6435353.10. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference.

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

REFERENCES

11. NYU-TIES and Uniandes’ HOLAS Assessment Bank: a curated inventory of educational tools/assessments and relevant monitoring and evaluation (M&E) resources designed for useand/or implemented in Colombia and Peru (available in English and Spanish).

12. The MENAT Child Measurement Inventory: an inventory of assessment tools that assess social and emotional learning, academic, physical health, mental skills/behavior, and programimplementation quality. Tools that have been tested in distance education contexts are also included.

13. Roxane Caires et al., “Starting from Square Two: Building a Cohesive, National SEL Framework in Lebanon,” in Educating for the Social, the Emotional and the Sustainable, ed. AndySmart and Margaret Sinclair (NISSEM, 2022), III:146.14. Peru M&E.

15. Kaya course - Introduction to the Measurement Process: Contextualizing and Adapting Measures for Monitoring and Evaluating Education Programs: a capacity-building trainingadapted from NYU-TIES’ curriculum for the UNHCR HEA.

16. Distance Learning Programs Theories of Change: a living framework for thinking and talking about distance education interventions, beginning in low- and middle-income (LMIC) andhumanitarian contexts and expanding over time to include distance education interventions designed for high-income contexts.

17. Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2019). The handbook of research synthesis and meta-analysis. Russell Sage Foundation18. Asare, Samuel, Rafael Mitchell, and Pauline Rose. (2022). “How equitable are South-North partnerships in education research? Evidence from sub-Saharan Africa.” Compare: AJournal of Comparative and International Education 52 (4): 654–673. https://doi.org/10.1080/03057925.2020.1811638. 19. Menashy, Francine, and Zeena Zakharia. (2022). “White Ignorance in Global Education.” Harvard Educational Review 92 (4): 461–485. https://doi.org/10.17763/1943-5045-92.4.461. 20. Shah, R., Boisvert, K., Restrpo Saenz, A.M., Egbujuo, C., Nasralla, M. (2023). Education in emergencies research partnerships through the looking glass. Globalisation, Societies andEducation. https://doi.org/10.1080/14767724.2023.2190877.

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

REFERENCES

21. Aria, M., & Cuccurullo, C. (2017). bibliometrix : An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975.22. Hallinger, P., & Hammad, W. (2019). Knowledge production on educational leadership and management in Arab societies: A systematic review of research. Educational ManagementAdministration & Leadership, 47(1), 20-36.23. Hammad, W., & Al-Ani, W. (2021). Building Educational Research Capacity: Challenges and Opportunities From the Perspectives of Faculty Members at a National University inOman. SAGE Open, 11(3), 21582440211032668. https://doi.org/10.1177/2158244021103266824. Draper, C. E., Barnett, L. M., Cook, C. J., Cuartas, J. A., Howard, S. J., McCoy, D. C., ... & Yousafzai, A. K. (2023). Publishing child development research from around the world: An unfairplaying field resulting in most of the world's child population under‐represented in research. Infant and Child Development, 32(6), e2375. https://doi.org/10.1002/icd.237525. Moll, L. C. (2019). Elaborating funds of knowledge: Community-oriented practices in international contexts. Literacy Research: Theory, Method, and Practice, 68(1), 130-138. 26. Weir & Hutchings, 2025 27. Ritesh Shah, Kayla Boisvert, Ana Maria Restrepo Saenz, Chima Egbujuo & Mai Nasrallah (2023) Education in emergencies research partnerships through the looking glass,Globalisation, Societies and Education, DOI: 10.1080/14767724.2023.219087728. Mario Novelli & Birgul Kutan (2023) The imperial entanglements of ‘Education in Emergencies’: from saving souls to saving schools?, Globalisation, Societies and Education, DOI: 10.1080/14767724.2023.223656629. Menashy, Francine, and Zeena Zakharia. (2022). “White Ignorance in Global Education.” Harvard Educational Review 92 (4): 461–485. doi:10.17763/1943-5045-92.4.461. 30. Appadurai, A. (1990). Disjuncture and difference in the global cultural economy. Theory, culture & society, 7(2-3), 295-310. 31. Fine, P. (2024, February 5). USAID can’t go it alone on localization. Brookings. https://www.brookings.edu/articles/usaid-cant-go-it-alone-on-localization/ 32. Funding the localization agenda: USAID’s progress. (n.d.). https://www.modernizeaid.net/press-room/funding-the-localization-agenda-usaids-progress 33. Funding the Localization Agenda: Measuring progress of United States development and humanitarian assistance to local organizations. (2023, November 30). Oxfam.https://www.oxfamamerica.org/explore/research-publications/funding-the-localization-agenda/34. Local Capacity Strengthening Policy | Basic page | U.S. Agency for International Development. (n.d.). U.S. Agency For International Development.https://www.usaid.gov/policy/local-capacity-strengthening

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

REFERENCES

35. USAID Localization: Challenges, opportunities, and next steps to further development initiatives on the local level | March 9, 2023 | U.S. Agency for International Development. (n.d.).U.S. Agency For International Development. https://www.usaid.gov/news-information/congressional-testimony/mar-09-2023-usaid-localization-challenges-opportunities-and-next-steps-further-development-initiatives-local-level36. Zigler, E. (1998). A place of value for applied and policy studies. Child Development, 69(2), 532-542.37. Bronfenbrenner, U., & Morris, P. A. (2007). The bioecological model of human development. Handbook of child psychology, 1. 38. Luthar SS, Ebbert AM, Kumar NL. Risk and resilience during COVID-19: A new study in the Zigler paradigm of developmental science. Development and Psychopathology. 2021May;33(2):565-80.39. Masten, A. S., Morison, P., Pellegrini, D., & Tellegen, A. (1992). 11 Competence under stress: risk and protective factors. Risk and protective factors in the development ofpsychopathology, 236.40. Sameroff, A. (2006). Identifying risk and protective factors for healthy child development. Families count: Effects on child and adolescent development, 53-76.41. Freire, P. (2020). Pedagogy of the oppressed. In Toward a sociology of education (pp. 374-386). Routledge. 42. Walker, M., & Unterhalter, E. (2007). The capability approach: Its potential for work in education. In Amartya Sen’s capability approach and social justice in education (pp. 1-18). NewYork: Palgrave Macmillan US.43. Sen A. Development as freedom (1999). The globalization and development reader: Perspectives on development and global change. 2014 Oct 13;525.44. Alkire, S. (2002). Dimensions of human development. World development, 30(2), 181-205. 45. Nussbaum, M. C. (2006). Education and democratic citizenship: Capabilities and quality education. Journal of human development, 7(3), 385-395. 46. Shalowitz, M. U., Isacco, A., Barquin, N., Clark-Kauffman, E., Delger, P., Nelson, D., & Wagenaar, K. A. (2009). Community-based participatory research: a review of the literature with strategies for community engagement. Journal of Developmental & Behavioral Pediatrics, 30(4), 350-361. 47. Talò, C., Mannarini, T., & Rochira, A. (2014). Sense of community and community participation: A meta-analytic review. Social indicators research, 117, 1-2 48. Chavis, D. M., & Wandersman, A. (1990). Sense of community in the urban environment: A catalyst for participation and community development. American journal of community psychology, 18(1), 55-81.49. Montero, M. (2004). Comunidad y sentido de comunidad. M. Montero. Introducción a la psicología comunitária: Desarrollo, conceptos y procesos, 197-223.

TABLE OF CONTENTS

INTRODUCTION

EXECUTIVE SUMMARY

REFLECTIONS

REFERENCES

ABOUT US

HOME

REFERENCES

50. Lachapelle, P. (2008). A sense of ownership in community development: Understanding the potential for participation in community planning efforts. Community development, 39(2),52-59.51. McMillan, D. W., & Chavis, D. M. (1986). Sense of community: A definition and theory. Journal of community psychology, 14(1), 6-23.52. Freire, P., Fraser, J. W., Macedo, D. P., & McKinnon, T. (1997). Mentoring the mentor: A critical dialogue with Paulo Freire. 53. Bolton, P., & Tang, A. M. (2002). An alternative approach to cross-cultural function assessment. Social psychiatry and psychiatric epidemiology, 37, 537-543. 54. Kalista, Jane. 2023. “Conceptual Framework for Education in Emergencies Data.” Edited byClaude Ndabananiye. UNESCO International Institute for Educational Planning. https://www.iiep.unesco.org/en/education-emergencies-roadmap-data-driven-resilience-14881.55. “Ministro Cuenca : ‘La Enseñanza Virtual Puede Ser La Nueva Revolución En La Educación’.”n.d. Ministerio de Educación. Accessed February 26, 2024. https://www.gob.pe/institucion/minedu/noticias/500728-ministro-cuenca-la-ensenanza-virtual-puede-ser-la-nueva-revolucion-en-la-educacion.

THE "HOW"

How is science transformed into impact? Partnerships, data engineering and open science principles, communications, research administration

TIES has aimed to transform developmental science into impactful educational interventions that improve children’s academic, social and emotional learning through a multifaceted approach that includes partnerships, data engineering, open science principles, strategic communications, and thought leadership. By forging strong partnerships with diverse stakeholders—including policymakers and local and international organizations—we strive for projects grounded in mutual trust and collective expertise. Our data engineering practices prioritize robust, flexible data systems designed to provide accurate, usable data in the unpredictable conditions of crisis environments. We adhere to open science principles, making our findings accessible and reproducible, which fosters a culture of transparency and accelerates the dissemination of knowledge. Through proactive and targeted communications, we aim to effectively share our research outcomes and engage a broad spectrum of audiences, from local program implementers to global donors. Additionally, our robust research administration ensures that projects are well-coordinated and aligned with both scientific rigor and practical needs. Together, these strategies have allowed TIES to transform scientific insights into real-world impacts, enhancing educational outcomes for children in some of the most challenging environments around the globe.

THE "WHAT"

How can we support a robust, culturally and contextually grounded, and inclusive developmental science for program and policy action?

Since its founding in 2014, the mission of TIES has been to contribute to a robust, culturally and contextually grounded, and inclusive science of human development and education to ensure more equitable opportunities for children and youth in low- and middle-income and conflict-affected countries. We integrate a wide spectrum of types of research—from neurophysiological and developmental insights to field experiments of innovative educational interventions systemic educational policy analysis—to design and implement interventions that are both practical and impactful. By expanding and refining our research methodologies, we strive to ensure that our work remains responsive to the unique cultural and contextual needs of the communities we serve. A cornerstone of our strategy has been partnering with scholars from the majority world through initiatives like the EQUAL network project, which worked to enhance the capacity and visibility of early- to mid-career scholars in sub-Saharan Africa and the MENA region. This approach not only fosters significant advancements in local research capabilities, but also propels many grantees in leadership roles within the educational research community. Through such dynamic and participatory research practices, TIES has worked to transform scientific inquiry into actionable strategies that significantly impact educational outcomes in crisis settings.

Reader's guide

This paper was authored by a group of staff at NYU-TIES who played a leadership role in core center initiatives or multiple initiatives (primarily focused on primary education). Individuals or small groups penned sections that highlight key learnings from their individual perspectives and contributions. While the various sections differ in foci, length, and intended audience they are all broadly structured to cover the “Context”, “Problem(s)”, and either an exposition of “What Worked” for us, or recommendations for potential solutions of “What Could Work”. The document can be read in its entirety or by focusing on sections of interest. We hope that sharing our learnings will serve as a valuable resource for funders, practitioners, and researchers, fostering a deeper understanding and inspiring impactful actions that advance holistic development for children globally.A few things to note:

• You can navigate the report by using the menu at the top of the page, the table of contents, and the arrows that look like this:
• All text in blue are hyperlinks, so you can click them to be directed to an external link with further information.
• All sources have been cited. If you'd like to view them, hover over the superscript and a tooltip will appear with the citation. You can also look for the superscripted number in the References section of this white paper.

THE "WHY"

What motivated us to do this work? Who is “us”? And how have we contributed?

In the face of armed conflicts, the COVID-19 pandemic and human-induced climate change, education leaders from the local to global levels are advocating for more resilient education systems

1,

. These systems need to be able to rapidly adapt to the complexities of cascading conflicts, displacements, and migrations, which disproportionately affect

children from contexts of marginalization and further entrench existing inequities. And in the context of such nested crises, the systems are also called upon to foster holistic learning - both academic knowledge and social and emotional skills and well-being - that can support children to navigate uncertain futures .

3,

4,

More, and better, data and evidence on holistic learning outcomes are central to many notions of resilient education systems . As researchers, we believe in the power of data and evidence to illuminate the educational landscape. Like a painting or a photograph, data and evidence are a reflection and interpretation of our world, with the potential to provide insight and catalyze curiosity and learning. Data and evidence implicitly and explicitly convey our needs, our biases, and our values, and provide a powerful jumping off point for interrogating those worldviews and taking informed action. However, the ability to generate, disseminate, and utilize data and evidence within an education system often depends on privilege structures and power dynamics deeply entrenched in social, cultural, linguistic, and historical contexts. While data and evidence can promote individual, community, and national freedoms to pursue valued “beings and doings", the generation of such data and evidence alone does not catalyze change.

In an effort to ensure that the knowledge that has been gained over a decade of working towards the goal of contributing to a robust and culturally grounded science of human development for program and policy action is available to the funders, researchers, and implementers in our field, insights from our team have been framed under two key questions that have driven our work:

1. What an be done to support a robust, culturally and contextually grounded, and inclusive developmental science in the low- and middle-income contexts we work in? 2. How can such a science meaningfully be transformed into impact? First, we delve into the what, how and why questions that inspired our work. Then, we present researchers’ reflections on them.