Throughout the research, we identified common metrics in the qualitative and quantitative reports of the following organizations: IRIS Core Metrics, IRIS+ SDG Report, EDWEEK Research Center Quality Counts 2021, National Center for Education Statistics Report Card, Michigan School Improvement Timeline and Missouri State Key Performance Indicators. We began our research of NGOs with a broad scope of existing educational NGOs. We identified existing frameworks of measurement and analyzed the metrics used by each organization.
Our survey included the following organizations: Khan Academy, Pratham, The Citizens Foundation, The Sozo Foundation, Villages in Partnerships, The Education Trust, Jumpstart, The New Teacher Project, Room to Read, Learning Forward, Foundation for Environmental Education, The Samburu Project and Vidya.
We mined each NGO organization for the most universal metrics highlighted by the United Nations SDG reports and by the recognized national and global education reporting organizations listed above. We also looked for specific, potentially unique, measurements reported by each NGO to develop a holistic understanding of the factors that must be considered in impact analysis. Our research yielded a comprehensive list of 50+ metrics.
The research highlighted three key difficulties in discerning a measurable impact framework for educational NGOs:
1. The wide variety of methods, stakeholders, and goals incorporated by NGOs to address quality education renders it difficult to construct a universal framework for their diverse and specialized approaches. Thus, we found that NGOs primarily differ in the following aspects: Educational Discipline (Academic v. Trade/Skill v. Emotional v. Environmental Education), Means (Policy work vs. School/Program work), In-Person v. Online Schooling and Location, and Primary Beneficiary (Child v. Adult Learner)
2. We also recognize that rules which enable easier decision-making necessarily exclude some factors or metrics that influence impact. This issue is especially pertinent as quality education significantly depends on other factors including access to electricity and clean water, financial literacy, job access, and rural economic health, among others. Data is costly and pursuing each of these factors will prove complex and unfeasible for investors and most NGOs.
3. As we scoured many financial statements and balance sheets, we realized that the simple dollar-to-impact conversion possible in other fields most often cannot apply to education.
In other words, while $ X = Y trees saved, the majority of qualitative and quantitative metrics of impact deal with the development of individual people, and are thus abstract and subjective. We originally aimed to map the dollar amount of capital allocated to each metric, but this proved infeasible. This aim is also not realistic for the NGOs to report as they do not have data either on donation conversions such as $ dollar = Student test rate improvement (the dollar investment and student impact clearly depend on the child and cannot be closely tied though they correlate). Therefore, our approach compares the NGOs overall summative rating with the NGO's total financial expenditures and how much money the NGO receives.
We mined each NGO for the most universal metrics. Given these difficulties, we argue that education assessment
frameworks should differ based on the questions of Educational Discipline, Means, and Primary Beneficiary. For our purposes, we narrowed our focus to developing a framework for educational NGOs that answer the questions in the following way:
Educational Discipline: Academics
Means: In-Person Schooling
Primary Beneficiary: Child
As stated our approach quantifies the efficiency of a monetary dollar donation. By comparing the NGO's overall summative rating (the sum of the performance ratings of specific metrics) with the dollars spent by the NGO to achieve that rating, we can measure the impact of a donor's dollar. We moved deep into five NGOs that fit the criteria and identified the most valuable assessment metrics. The NGOs are Room To Read, Jumpstart, Pratham, The Sozo Fndation and Vidya.