Scale What?

Student impact at scale is the panacea for those of us who have devoted our careers to improving K-12 education. We desperately want to make a difference in the lives of all students, and although it’s always heartening to see small pockets of success in individual schools and classrooms, we are always seeking ways to catalyze a massive, nationwide improvement in student learning. Scale must be the solution.

As it relates to the emerging area of Personalized Learning (PL), I am often asked how to achieve massive scale, to which I respond, “Scale what?” The reality is that this work is nascent, and while there are promising approaches emerging that combine blended instruction with competency-based progressions, none of the models are yet ready to scale across thousands of schools and millions of students. The reason is that our bar for massive scale must be sustained, replicated results across a diverse array of schools over a period of several years, which we do not (yet) have.  

Even though no single PL model, aside from 1:1 mastery-based tutoring which is prohibitively expensive, is ready to be scaled, we should not just sit on our hands and wait for magic to happen. Instead, states, districts and charter networks should consider a framework for approaching scale. I suggest a three-tiered approach:

  1. Start by scaling only foundational elements (a.k.a. building blocks) that allow all schools to pursue innovations if they so choose, while at the same time piloting bold PL designs in a limited number of early adopting schools,
  2. As data emerges from the early pilots AND those schools that experimented on their own, begin to spread/replicate the most promising approaches (defined by impact on student learning) to more schools (note: how to do this effectively is worthy of another blog post, if not a book), and
  3. Scale only the instructional practices or school models that produce sustained results over a period of multiple years.

With this framework in mind, below is a way that states, districts and charter leaders might divide those activities that are ready for scale versus those that are not:

Scale Now

  • Design principles that provide some direction and guardrails around the innovations that you are seeking from early adopting schools and educators.
  • A comprehensive, trusted system-wide assessment and performance measurement system for schools with two key components: (1) a precise, multi-faceted definition of college and career readiness for each student that includes, but is not limited to, academic knowledge and skills and (2) the ability to accurately and adequately measure each student’s individual growth/progress toward that definition of college and career readiness on an annual basis, if not much more frequently.
  • Research and evaluation focused acutely on understanding the degree to which schools and the instructional approaches they adopt are having an impact on each student’s progress toward college and career readiness.
  • Technology infrastructure (e.g. broadband, Wi-Fi) that can adequately support a range of technology-mediated instructional models.
  • IT capacity to manage multiple types of devices (e.g. tablets, laptops).
  • Reasonable policy changes to allow innovative schools to pursue different types of models (e.g. proficiency-based graduation, seat time waivers, lifting onerous human capital rules).
  • Reasonable changes to procurement rules and adoption processes to give schools, teachers and perhaps families more agency in selecting digital and non-digital instructional materials.
  • Clear guidelines for schools that define the autonomies (e.g. budget, materials, assessment, staffing) they have and the outcomes they are being held to.
  • Software infrastructure (e.g. LMS, middleware) that creates the enabling conditions necessary for schools to adopt PL without forcing them into a specific instructional approach. Note: this one is particularly tricky because too often software locks schools into a particular instructional model.

Not Ready for Massive Scale Now (If Ever?)

  • Unproven instructional models: as I suggest above, few if any of the PL models that have emerged thus far are ready for widespread scale, particularly in the context of existing schools that have legacy systems and structures that often make change difficult. For that reason, I would caution any district or charter network against picking a single instructional approach and attempting to scale it across many schools.
  • Specific devices (e.g. laptops, tablets): the classic mistake districts and networks make is to scale a single type of device to all schools before they have designed the instructional model(s) that those devices will support. See Hardware is Not a Strategy for more of my perspective on this.
  • Specific digital content (e.g. adaptive learning programs): as with devices above, requiring all schools to implement content from a specific set of providers will inevitably stifle school-level innovation. That said, some districts (e.g. NYC) have centrally procured content from multiple providers and made it available to a group of schools, which is an interesting approach that retains school-level autonomy without sacrificing centralized purchasing power.

Seed Now, Spread with Results, Scale Only When Proven 

  • A limited number of whole-school designs: offer incentives for a coalition of the most visionary and eager among your school leaders – either new or existing schools – to pursue bold new approaches to PL. The expectation must be that these schools will pursue breakthrough results in learning growth and ultimately college readiness rates, but be realistic about the time required to iterate to achieve those results. Consider using the Next Generation Learning Challenges model for inspiration on how to structure a competitive school selection process.
  • Partnerships with promising 3rd party providers that have had success elsewhere.
  • Educator fellowships that allow a coalition of the willing to implement new instructional approaches in their classrooms. The expectation is that successful pilots in this initiative will ultimately lead to school-wide implementations in future years. CityBridge Foundation in D.C. and Chicago Public Education have sponsored these types of fellowships for teachers. 
  • Permission for school leaders and educators to test new approaches without being part of a formal system-wide program.

In sum, states, districts and charter networks can take critical steps today to encourage PL by setting the stage for schools and educators to innovate without jumping too quickly to scaling unproven instructional practices. Challenging as it may be to proceed cautiously, waiting to scale based on proven results is the wise approach for this movement and the students we aim to serve through it.


Reblogging Stacey Childress’ post on student outcomes in personalized learning. Join the discussion – what do you think the goals should be?

Thinking Out Loud

A couple of weeks ago I blogged about our support for schools that are redesigning their instructional models in order to personalize learning for students. I listed design principles shared by these schools:

  • Student Centered: designed to meet the diverse learning needs of each student every day
  • High Expectations: committed to ensuring that every student will meet clearly defined, rigorous standards that will prepare them for success in college and career
  • Self-Pacing & Mastery-Based Credit: enables students to move at their own optimal pace, and receive credit when they can demonstrate mastery of the material
  • Blended Instruction: optimizes teacher- and technology-delivered instruction in group and individual work
  • Student Ownership: empowers students with skills, information and tools they need to manage their own learning
  • Financial Sustainability: sustainable on public per-pupil revenue within four years
  • Scalable: designed to serve many more students if it demonstrates impact

In the earlier post I wrote…

View original post 413 more words

ITHAKA Releases Results from Randomized Blended Learning Study: Similar Results at Lower Cost and in Less Time

ITHAKA released a report this week titled Interactive Learning Online at Public Universities: Evidence from Randomized Trials. This study focused on the implementation of Carnegie Mellon’s Open Learning Initiative (OLI) Statistics course across six campuses. A summary of the findings are below:

We find that learning outcomes are essentially the same—that students in the hybrid format “pay no price” for this mode of instruction in terms of pass rates, final exam scores, and performance on a standardized assessment of statistical literacy. These zero-difference coefficients are precisely estimated. We also conduct speculative cost simulations and find that adopting hybrid models of instruction in large introductory courses have the potential to significantly reduce instructor compensation costs in the long run.

While blended learning advocates may be disappointed that the learning outcomes were similar, the fact that the study showed similar results at a lower price is still promising. Furthermore, it is important to note that students in the blended learning course spent on average 25% less time taking the course. In other words, they not only achieved the same results at lower cost, but they did so in significantly less time. While promising, keep in mind that this study focused only on a statistics course, so these results may not necessarily be generalizable across other courses and subject areas.

This example illustrates an important point that we should consider as we weigh the relative benefits of new instructional models compared with legacy approaches. The three basic questions that any study should be asking are: to what degree are these models driving …

  • Effectiveness: did students master the intended learning objectives?
  • Efficiency: how quickly did they master them?
  • Cost: how much did it cost?

In this study, the findings were that effectiveness was held constant while efficiency increased and costs were lowered. We could imagine other models demonstrating different combinations of these three variables that could be equally or more compelling, including the ideal combination – greater effectiveness, greater efficiency and lower costs.

Download Report

Public Impact’s Checklist for Policymakers Interested in the Reach of Effective Teachers

Bryan and Emily Hassell at Public Impact are doing some fascinating work on extending the reach of excellent teachers through their Opportunity Culture initiative. They do not focus exclusively on technology as a means of facilitating teacher reach, but they do see it as an important tool in several of their models.

One document that they’ve produced that I feel may be especially helpful to practitioners and policymakers who are looking for sage guidance in this area is a two-page policy checklist. The fact that technology is promoted as a means to an end – to extend the reach of effective teachers in order to support higher achievement for more students – as opposed to as the end itself is an extremely important message.

A Prediction (and a Word of Caution) about Scaling Blended Learning in Elementary Schools

My good friend, Alex Hernandez, at Charter School Growth Fund, speaks regularly about the potential for blended learning to improve upon a widely-accepted instructional practice in many elementary school classrooms: guided instruction (which includes guided reading). In that model, a teacher provides direct instruction to a small group of students while her remaining students work independently, with another adult (e.g. a paraprofessional) or in small, self-directed groups. The groups rotate, typically every 20-30 minutes, such that every student experiences small-group instruction with his teacher daily.

The obvious benefit relative to whole-group instruction is that guided instruction allows a teacher to target instruction to fewer students at a time. The challenge for a teachers is ensuring that students outside her immediate purview remain on task and receive on-demand  support. For example, if a student has a question during independent practice, the teacher cannot offer immediate assistance because she is focused intently on the students directly in front of her.

To mitigate some of these challenges, schools such as KIPP Empower Academy in Los Angeles have turned to digital content with promising early results. They have taken the basic in-classroom guided instruction model and  replaced much of the analog independent practice with digital content. The idea is that the adaptive digital products can provide personalized instruction and embedded support, allowing a student to work autonomously for a block of time. Michael Horn at the Innosight Institute, the author of The Rise of K12 Blended Learning, refers to these as a “rotational” blended learning models for obvious reasons.

This week, I was on a school visit to KIPP Chicago where they are also implementing this rotational approach in their kindergarten and first-grade classrooms. During that visit (which was great thanks to Executive Director April Goble and technology guru Anirban Bhattacharyya), I had an epiphany that now seems obvious to me but had not been until then. My prediction: the in-classroom rotation model will scale dramatically within existing elementary classrooms in the next few years. Here is why:

  • The classroom rotation structure is already in place in many elementary schools, so this change does not require any fundamental changes to classroom design or teacher roles.
  • Budget pressures have and will continue to force many schools to increase class sizes. Teachers will demand solutions that help them to differentiate instruction to larger numbers of students.
  • Engaging digital content should make a teacher’s job easier: any activity that engages students and allows them to work autonomously lets her focus on other students; also, using digital content in place of paper-based assignments may reduce the amount of lesson planning, content curating, grading and printing that she has to do.
  • A number of digital products that meet schools’ needs, particularly in math, exist already and more are emerging rapidly. Popular products include DreamBox, ST Math, Compass Learning, and iStation, to name a few. See for a reviews of these products and others.
  • Hardware requirements are relatively modest because not every student needs a computer at once; many schools already have some equipment  and those that do not can enter it with a relatively limited investment in inexpensive laptops.

While I believe this transition of existing classrooms to blended learning holds tremendous promise, both in terms of student learning and teacher support, I also have concerns about this prediction:

  • Data on the effectiveness of digital content and blended learning in general is limited. Without data on the efficacy of various products and approaches, teachers and schools cannot make informed purchasing decisions.
  • The easiest blended model to implement may not be the most effective. Other approaches, such as Rocketship and New Classrooms (which current serves only grades 6-8 but could conceivably port its model to K-5), have design elements that would make it more challenging for existing schools to implement but may ultimately be more effective. Time and data will tell.
  • Potential for data overload for teachers; making sense of all of the data that these programs generate and adapting instruction accordingly is an inevitable challenge.
  • Implementing multiple products is often a logistical challenge; integrating them into existing systems is still more expensive than it should be. Schools must decide whether to pay extra for an emerging 3rd party solution or deal with complexity such as a lack of single sign on and disparate data sources.

In closing, I am convinced now more than ever that some form of blended learning – good, bad and otherwise – will rapidly become the norm rather than the exception in elementary schools within the next five years. My hypothesis is that this rapid proliferation will occur largely within the in-classroom rotational model because it fits neatly into their current structure and it alleviates acute pain points within it. Putting my cautious optimism hat on, I would advocate for a measured approach to adopting in-classroom blended learning: pilot new programs, measure results, consistently tweak the model and expand based on results. Separately, we should also inspire and support a wave of innovators with bolder approaches to personalized, mastery-based learning in elementary schools, even if these new approaches do not fit neatly into traditional classroom structures. Ultimately, my hope is that the most effective (not the easiest to implement) approaches win.

EdTechResearcher on

Justin Reich, a doctoral student at the Harvard Graduate School of Ed, is now blogging as the EdTechResearcher at In his intro post, Justin says he’ll be focused on the following:

1) Recent Reports: Analysis of newly published white papers and articles in scholarly journals, translated and summarized for a wide audience

2) Ask a Researcher: Where I answer emails that I receive from educators and fellow researchers about my work or anything else

3) In the News: Examining news articles reporting on education and education technology

4) Works in Progress: Updates on research in progress from my work or colleagues

5) From the Field: Stories and questions from educators that I meet and work with

6) Up for Debate: Joining in conversation with fellow ed tech researchers and practitioners on questions that are engaging people

7) Questions worth Exploring: Looking at research questions that someone should answer; a good category for new doctoral students looking for research ideas!

I’m looking forward to seeing this. The ed tech space is certainly in need of folks that can connect research and practice.