There is a particular kind of institutional discomfort that sets in when a school’s data dashboard begins telling a story nobody prepared for. Scores are tracked. Cohorts are compared. Year-on-year graphs are generated. And somewhere in the visualisation (usually quietly, between the lines) a gap appears. Then it widens. Then the meeting ends, the report is filed, and the classroom door closes again.
This is not a story about bad schools or indifferent teachers. It is a story about what happens when the systems built to surface a problem are not connected to the systems capable of solving it. And it is increasingly the story that education data is telling, not just in pockets, but at scale.
The Data Is Not the Problem. The Distance From Data to Decision Is.
The argument for regular student assessment has always rested on a clean logic: measure learning, identify gaps, and address them. In practice, the chain breaks at the second link. Data gets collected. Gaps get identified. What follows is often neither swift nor surgical.
In their 2020 study Capacity Building to Bridge Data Use and Instructional Improvement Through Evidence on Student Thinking, published in Teaching and Teacher Education, researchers found that despite over a decade of investment in data-use initiatives, using assessment data effectively to inform instruction remains challenging for most teachers, and identified why: a lack of data literacy, insufficient support structures at the school level, and data-use efforts historically focused on accountability rather than continuous improvement. Crucially, the study found that meaningful data use only occurred when evidence from student thinking was embedded directly into the instructional planning process, not treated as a separate analytical task sitting outside it. The problem, in other words, is not that teachers ignore the data. It is that the systems around them were never designed to make acting on it easy.
This distinction matters enormously because it shifts the conversation. If the failure were motivational, the solution would be exhortation. If it is structural (a gap between data infrastructure and instructional infrastructure), then the solution requires something different: assessment design that is closer to the classroom, more frequent, and built to produce decisions rather than just records.
As Karen Schildkamp argued in her 2019 review of data-based decision-making in schools, published in Educational Research: “Acting without data is often not effective… if the cause or causes of low student achievement lie elsewhere, the problem remains unresolved or may even be exacerbated.” In other words, the wrong kind of data use (or data used too infrequently and too late) can actually cement the problem it was meant to solve.
What Widening Gaps Actually Look Like in the Numbers
The post-pandemic period has offered a grim natural experiment in what happens when learning gaps go undetected, or detected too late, across an entire system.
A meta-analysis drawing on PISA 2022 trend data, synthesised in a population-level study of COVID-19 learning inequalities, estimated an average decline in mathematics equivalent to roughly seven months of learning, with steeper losses in systems that experienced longer school closures. Critically, the research found that learning losses were up to 60% larger for children from less-educated households, meaning the gap between vulnerable students and their peers did not merely persist after schools reopened; it accelerated.
Meanwhile, NWEA’s analysis of 7.7 million students across more than 22,000 schools found that pandemic recovery has not merely stalled. Reported in a July 2024 investigation by Education Week into why students are still losing academic ground, the MAP Growth data showed that students in most year groups are falling further behind pre-COVID benchmarks, not catching up. The gap is not closing. It is moving in the wrong direction.
These are not outliers. They are the aggregate output of thousands of individual classrooms where the data existed: national assessment results, end-of-term tests, and teacher observations. However, the distance between that data and a targeted instructional response was simply too large to bridge in time.
The Quiet Student Problem
There is a particular student that the data tends to miss. Not the one visibly struggling, already flagged, already on a watch list. It is the student who attends regularly, completes his work, and lands somewhere in the middle of every assessment. Nothing about his profile invites a second look. And yet the gap is opening.
Progress data, when collected only annually or only at mandated checkpoints, is poorly suited to catching this student. Summative assessments tell you what happened; they are less useful for telling you when it started happening or what specifically is driving it. A reading score tells you a child is underperforming. It does not tell you whether the problem is decoding, inference, vocabulary, or reading stamina, each of which requires a different instructional response.
This is where the granularity of assessment design becomes a data science problem as much as a pedagogy problem. Diagnostic value requires not just measurement but the right measurement, at the right frequency, broken down into the right sub-domains. Without that, a school is running a data pipeline that produces signals too coarse to act on.
The American Psychological Association’s synthesis of classroom formative assessment research is instructive here: formative assessment can improve educational achievement by 0.2 to 0.3 standard deviations, equivalent to a 50 to 100 per cent increase in the rate of learning for secondary students, but only when the interval between assessment and instructional adjustment is short. The shorter the feedback loop, the stronger the effect.
That finding has direct implications for how schools should think about the infrastructure of monitoring student progress. A system that delivers results broken down by sub-domain, benchmarked against national norms, and aligned to existing national assessments compresses the feedback loop in a way that annual testing cannot. The data becomes actionable. The gap between knowing and doing narrows.
Why This Is a Data Architecture Problem
Education technology has spent a decade promising to close this loop. Learning management systems, dashboards, and analytics platforms have proliferated. Yet the problem persists. Why?
Part of the answer is that most school-facing data tools were built for reporting, not for decision-making. They aggregate. They visualise. They compare cohorts. What they rarely do is connect a specific student’s specific gap to a specific next instructional step and make that connection legible to a classroom teacher who has twenty-eight other students and forty minutes until the bell.
The missing layer is not more data. It is assessment design that is purposefully built to produce diagnostic outputs (sub-domain breakdowns, progression tracking over time, curriculum-aligned benchmarks) delivered with enough frequency and speed that the insight is still relevant when the teacher receives it.
This is a systems design challenge that the data science community understands well. In any other applied analytics context (logistics, healthcare, financial risk) a signal that arrives six months after the event it is measuring would be considered operationally useless. Education has largely normalised that latency. The cost is paid by the students who slip through in the interval.
What the Data Is Actually Asking For
Progress data that shows widening learning gaps is not, in itself, a failure. It is information. The failure is what happens (or does not happen) between the information and the response.
Schools that close gaps consistently tend to share a pattern: assessment is frequent enough to be formative, granular enough to be diagnostic, and connected to reporting systems that make differentiated action possible. The data is not treated as a verdict. It is treated as a prompt.
For teachers and curriculum coordinators navigating the aftermath of a disrupted few years, that distinction is more than semantic. The question is not whether to collect data. The question is whether the data being collected is close enough to the classroom (and fast enough in its return) to change what happens in it.
When it is not, the gap on the dashboard is not the problem. It is the symptom.