Improving Student Math Outcomes: Beyond Curriculum and Data Meetings

TL;DR: Summary

Changing the mathematics curriculum is often seen as a cost-effective solution, but recent research indicates that it does not significantly impact student outcomes on its own. Instead, improving teacher quality and providing content-specific professional learning (PL) can be more effective. This type of PL involves enhancing teachers' content knowledge and pedagogical skills and fostering collaboration. Additionally, while data-informed education is popular, merely studying assessment data without directly improving instructional techniques has limited impact. The GROWTH framework, developed by MathTrack Institute, offers a comprehensive approach to improving math teaching by deepening teachers' understanding and collaboration without changing the curriculum.

Let's Define Improving Mathematics Learning

It is important to acknowledge that school leaders and curriculum directors must make hard decisions about sustaining or improving all of their programs, as well as their facilities, operations, and community engagement, on a budget. That is a lot! We focus on Science, Technology, Engineering, and Mathematics (STEM) programs, and more precisely, mathematics for several reasons. The broader STEM, which we quip is all applied mathematics, is our focus because of its importance to the national economy (Atkinson & Mayo, 2010). Mathematics education is a cornerstone for students' intellectual development, providing fundamental knowledge and skills for intellectual behavior in our complex world (Skovsmose, 2010).

So, what does it mean to improve mathematics learning? The simple answer is that with changes, you see better, higher-quality mathematics teaching happening in the classroom, and the students are growing and learning as assessed by standardized and localized assessments.

From our analysis of recent educational research, the choices that a school leader or curriculum director has available to improve mathematics learning outcomes fall into three main categories:

• Choice of Curriculum: Leaders and curriculum directors have various options to ensure the curriculum aligns with state or national standards. This choice incorporates the need to ensure the curriculum is challenging and has relevant mathematical content promoting critical thinking. The choice often comes down to keeping the current curriculum or choosing a new curriculum for mathematics.
• Collecting and Using Assessment and Student Data: Leaders and curriculum directors seek to monitor student progress and inform instruction using regular formative and standardized assessments that produce large volumes of student data. These choices involve scheduling time, allocating a data expert, and promoting teachers' ability to use assessment data to identify areas of need, track progress, and tailor interventions.
• Teacher Quality and Instruction: Leaders and curriculum directors seek to hire and retain teachers with strong mathematical knowledge and teaching skills. This choice often relies on the available talent pool in the unique school context. It is also about finding and implementing continuous, high-quality professional development focused on effective mathematics instruction and content knowledge.

We will examine each of these in order, discussing what research says about spending time on each and what the outcomes might be from a complexity lens of influencing the probability of success.

Is Changing Math Curriculum the Most Cost-effective Solution for Improving Mathematics Learning?

The choice of the "right" textbook and curriculum for mathematics has a widespread impact on the exposure to mathematics that your students and teachers utilize. Few school leader decisions have far-ranging implications for mathematics learning in your school community. For example, a recent survey found that 94% of elementary schools reported using the official district-adopted textbook or curriculum in more than half of their lessons (Blazar et al., 2020).

Suppose you are in charge of the math curriculum and want to see student outcomes improve. The choice comes down to switching your curriculum or keeping your current curriculum. Switching curriculum materials has been positioned for the last 40 years in research as a way of getting a significant return on your investment as a school leader (Kirst, 1982; Whitehurst, 2009). Chingos and Whitehurst. (2012) explained that improving teacher quality and expertise is expensive and time-consuming, whereas the typical choice of instructional materials should be relatively easy, inexpensive, and quick. Here's a quote that we like to use in our education innovation discussions:

“There is always an easy solution to every human problem—neat, plausible, and wrong." ~ H.L. Mencken

The most recent meta-analysis research has one conclusion about the curriculum choice as the place you invest in positively influencing your mathematics program. The results? The curriculum is essential, but you already know that. You will need more than the choice to change it alone to improve your mathematics program. Despite considerable interest and attention to textbooks as a low-cost, "silver bullet" intervention for improving student mathematics outcomes, recent research concludes that the adoption of a new textbook or set of curriculum materials, on its own, is unlikely to achieve this goal (Blazar et al., 2020). Your curriculum choice is important, but if you have one that ranks relatively well on the EdReports rubric, you may find more success placing your limited resources elsewhere.

Is Studying Assessment and Student Data the Most Cost-Effective Solution for Improving Mathematics Learning?

The new era in technology for curriculum, adaptive curriculum, assessments, and standardized assessments has brought a wave of "data-informed education." Results from a recent national survey (Banilower et al., 2018) have specific recommendations for school leaders and curriculum directors to focus on when improving the use of academic assessment and data. Specifically, STEM educators are recommended to devote allocated professional learning time to:

1. Studying state standards
2. Learning about assessment
3. Studying student data

The same study suggests that schools invest significantly in this recommendation (Banilower et al., 2018). What we need to know from this research is the extent to which such activities improve students' academic outcomes in mathematics. Is there evidence that studying student assessment data is the place to invest your teachers' limited time?

The results here may be surprising or even disturbing. Based on evidence in the literature, especially the recent work from Harvard University (Hill et al., 2020), eliminating data team meetings is recommended. You read that right. These meetings are defined as time spent where the mathematics team would study assessments and student data to individualize and improve instruction. In short, schedule time to do what is recommended!

“There is always an easy solution to every human problem—neat, plausible, and wrong." ~ H.L. Mencken

The researchers based these findings on their meta-analysis research and a qualitative survey from other researchers (Barmore, 2018; Goertz, Oláh, L. N., & Riggan, 2009). This data, as clearly as meta-analysis and qualitative research can conclude, suggests that teachers studying data does not itself lead to new or improved instructional techniques or learning outcomes. It also states that these meetings often ascribe poor student learning outcomes to factors other than instruction. Straightforwardly, teachers usually talk about student behaviors and the dynamic nature of the classroom during these meetings rather than what they were designed to accomplish. This means that recent research concludes that making time to study student data without directly impacting the content-specific instructional techniques to influence factors gleaned from that data is ineffective for improving mathematics learning.

Is Teacher Quality and Quality Instruction the Most Cost-Effective Solution for Improving Mathematics Learning?

Many agree that teacher talent is the single most influential school factor in student achievement in our schools (Aaronson et al., 2007; Chetty et al., 2014; Kane & Staiger, 2008). The thousands of school leaders and curriculum directors we talk with cite this as one of their most significant anxieties, especially at different stages throughout the academic year. Chingos and Whitehurst (2012) explain that improving teacher quality and expertise is expensive and time-consuming, and often, because of this narrative, it is not chosen as the method for improving schools' mathematics programs. That is, beyond the intense initial hiring stage of the teaching team. What is the evidence that improving your teaching team's quality is the right choice for improving your mathematics program?

Returning to the Harvard study (Hill et al., 2020), they found that teacher professional development that is non-content specific showed no measurable impact on student learning, especially for STEM and mathematics. Contrastingly, they reported that programs aimed at improving teachers' content knowledge and pedagogical content knowledge (Ball et al., 2008; Berkopes, 2014; Davis & Simmt, 2006; Lampert, 2001; Shulman, 1986) tended to outperform those that did not focus at all on these topics. Further, programs benefited from teacher collaboration, or lesson-study (Chokshi & Fernandez, 2004) styled "implementation meetings" — chances for teachers to collaborate during the program after experimenting in their classrooms. This type of teacher collaboration allows the natural experimentation and feedback loops with your educators to capitalize on their diverse experiences and expertise.

Concentrating on high-quality professional learning that is content-specific is a complex solution. It will take time for teacher professional growth, which is best aligned with while gaining classroom experience collaboratively. As a complex solution, it recognizes the constraints of budget, time, and the ability to limit unintended consequences while heightening positive unintended consequences. It also can easily be combined with the other two recommendations. For example, Kathleen Lynch and others at Harvard University found that STEM professional learning programs that focus on helping teachers learn to use curriculum materials they have were more effective for student learning growth than programs that featured professional learning alone. Combining professional learning with curriculum materials was also more effective than programs where teachers received new curriculum materials. Teachers studying their student data while training on improving their instructional capacity about the topics that emerge as problematic for their students can be impactful.

How Can I Ensure That Teacher Professional Learning is Content Specific to Math?

It's a great question, one that we continue to find is tough to answer with on-market professional learning programs. Due to this need, we developed the GROWTH framework from my personal research and an evidence base of over 40 years of study. It was designed by MathTrack Institute to help educators deepen their conceptual understanding of mathematics, specifically for teaching. This programmatic approach was intended to improve instructional practice without requiring our partner schools or teachers to change the curriculum that they are using. We reference this mathematical understanding as an applied mathematics skill called mathematics-for-teaching, correlated in research as highly effective for improving student learning.

GROWTH is an acronym, and each letter outlines a category of questioning and activities that guide mathematics teachers' professional learning, following the most cutting-edge research on how to do this and ensuring a high probability of improving student outcomes.