Why Personalized Learning Keeps Failing

In my inbox sat a New York Times article sent by my girlfriend's mother: Silicon Valley Came to Kansas Schools. That Started a Rebellion. The article tells the story of an 8,000-person town of Wellington all but lighting the school on fire after the Chan and Zuckerberg Foundation's personalized learning software was introduced — software developed alongside $99.1 million dollars of grants. "Isn't that what you do Elliott?" asked Natalie.

I write this response in part to save relationship face, but also to bring to light why this outcome is not surprising at all from my research. Take a deep breath. It is going to be a bumpy ride.

As I tell the edtech companies I advise: "First off, you're not alone. I have watched hundreds of hours of elementary school kids use digital software, and I have yet to see one child learn a new math concept." For the last five years, I observed Boys and Girls Club students fail at using educational software — with help from eye tracking and emotion sensors. A child will listen to a song or mini lecture about finding the area of a triangle for the first time. But afterwards, they consistently fail at solving that area. And then they give up and think math is boring.

When we translate our educational theories to the digital world they falter. Most educational editors are copying what they did as a child: an adult tells you what to do and then you repeat. But these digital designers do not include all of the subtle things a teacher does. Children care a lot about their teacher so they are fixated on her. But when educational videos come up, children's eyes consistently drift away, looking for other, more emotional stimuli. A teacher gives feedback and understands when the child is confused — teaching is more of a dialog than a lecture. Computers are agnostic to where the child is at.

The Boys and Girls Clubs of Denver have a reading room where children can choose to read after school. Daily, children line up at my research station and ask if I would read to them. One of the most impactful things a volunteer can do at the club is read with a child. Sit with them. Ask them questions. Sound out words together. Learning is social.

ThinkCERCA capitalizes on this: students come together as a group and discuss articles they read on the computer. The whole second grade class debates whether we should get rid of roads to save animals. You can see the students thinking and reacting to one another, applying what they read.

In our prototypes, sometimes we let students work together — despite our client's request to avoid it. When children are allowed to work together, their learning is amplified. They come alive. Children laugh at jokes together, support one another in hard problems, and celebrate shared successes. I wish we would design software that was meant to be used as a group.

One Summit Learning parent described the software this way. I believe this is a byproduct of how we view computers: PCs are designed to be "personal," for the individual. In one school I visited, the most social interaction students received was by logging out of the educational program and sharing YouTube videos with one another. Teachers checked their email for the entirety of the computer lab hour. The New York Times article stated teachers would only check in with students two minutes a week.

At first I blamed the teachers. But now I understand how these programs are designed. It feels rude to peep over a student's shoulder. No questions say "Ask the teacher or a friend…" In an effort to make things more independent, the programs boot classmates and teachers out of the experience — those same classmates and teachers who are the foundation of strong learning.

In the New York Times article, John Payne of the RAND Corporation states "There has not been enough research." But I think research is the wrong word. For the most part, educational theories about how to teach the area of triangles digitally have failed students. I already have plenty of video evidence of disgruntled students not learning, and now Summit Learning has rebelling schools as a clear indicator of failure.

We need to move past validative research: spending large amounts of resources to prove old methods are not working well. Instead, the Chan and Zuckerberg Foundation should invest in generative research that looks to innovate new ways of teaching digitally. New research should be messy — trying new things, making mistakes, and starting in classrooms with students who might someday use the software. Educational research should feel more like a hackathon than a standardized test.

I tell my clients that academic research is simply a pool of ideas. Do you want to teach with a growth mindset angle, a constructivist foundation, or a more didactic method? All of those ideas are fine, but they should be considered only as potential ideas — nothing more. Take those years of research at Stanford, build something out of it in two days, and test it for a week at a school. In that one week you will learn more about what is effective in your unique environment than from those ten years of research.

In a recent project, I was asked to better engage middle school students with taking tests. I gave them four different methods — including Growth Mindset approaches — before a math test at the Boys and Girls Club. All four digital methods crashed and burned. Students quit after about three to five minutes.

I then co-created with the students. They showed me how their peers motivated them more than any computer program. When a high school student with your same skin color talks, that means something. I don't know what academic theory they were tapping into, but that was the winning idea. We created videos of high school students explaining how they double-checked their answers. It worked. Students wanted to take the test.

One of the largest challenges personalized learning faces is that there is no clear working model to copy from. I cannot point at a piece of educational software and say "just do that." Because there is no working method to copy, designers of personalized learning tools need to recognize they are taking stabs in the dark.

To make matters worse, kids are often a voiceless consumer. My clients seldom ask children what they think or what is broken. If you take time to observe and listen to what kids want, they will paint a different world for how digital education should look. The same goes for teachers — they have needs going unmet too. As long as funding and permission comes from school boards and admins, edtech companies are likely to miss their real consumers. When you fail to listen to your customers, your students are going to drop out and teachers are going to post protest signs.

In the New York Times article, the Summit team came off as distant — claiming parents and students are nostalgic and resistant to change. A more human-centered approach would be to listen and embrace those complaints. If a child does not like your program, there is something there for you to learn.

I understand why a human-centered design process is hard. When I spoke with the Charter School Growth Fund, I asked how they knew their charter model would work. We had a heart to heart about how hard it is to get permission from all the stakeholders and funders to experiment. As an innovator, you are diving in, hoping all the pieces work on the first try. They seldom do. I advised the fund to take one class and rapidly iterate on the experience — make it work the way they want it to before replicating to other schools. This process would involve daily iterations and heavy co-creation with teachers and students. They agreed, but that is not how funding works. You have to sell a perfectly baked solution from the get-go.

The New York Times article all but said personalized learning is a failure. I used to think the same. However, after years in the digital classroom, I see potential — potential that keeps me writing. Personalized learning allows students to:

But for today, these promises still remain at a distance. These features are difficult to implement in a non-ideal system and require deep, intentional user research and design thinking. Hence why they are more opportunities than realities.

I do not personally know the Summit Learning software. And I do not know how much of the New York Times article was true or slanted. But all of the issues cited are the same as what my other personalized learning clients experience. The digital education problem is not a Chan and Zuckerberg Foundation problem. It is a design problem affecting all of education.

I believe with strong human-centered design and strong research, Summit Learning has the potential to reinvent how we all learn. Here's hoping they learn from Wellington and do.