Megan Hall: When we see a friend’s face, how do we instantly know who they are? Doris Tsao looks closely at the brain patterns of monkeys to help unravel this mystery. This year, she received The Kavli Prize in Neuroscience with Nancy Kanwisher and Winrich Freiwald, for identifying a specialized region of the brain where facial recognition happens.
Scientific American Custom Media, in partnership with The Kavli Prize, spoke with Doris to learn more about her discoveries and how she’s using them to unlock a bigger question – how do our brains represent the world?
As a kid, Doris Tsao was surrounded by science. Her mother was a computer programmer and her father is a mathematician.
Doris Tsao: I always had grew up with the sense that being a scientist was the most noble life calling. That really came from my parents talking to them. It was part of our family.
Megan Hall: But Doris didn’t think she’d be a scientist.
Doris Tsao: I didn’t think of myself as particularly interested in science. I like math. My parents gave me geometry problems, and I loved that. I certainly didn’t think about the brain when I was a kid. I liked to play. I played with Barbie dolls. I loved to read biographies.
Megan Hall: That all changed when she was in sixth grade.
Doris Tsao: I remember just waking up one morning and, suddenly, for no real reason, wondering if space is infinite or not. Because it seemed like if space is infinite, that seems incredible. I’d never thought about infinity before. And if it wasn’t, how could that be? Right? So, I just kept going in these loops, and I remember obsessing about this for days.
Megan Hall: She revisited this question in high school as she started reading about artificial intelligence and neuroscience. Books by philosophers like Immanuel Kant made her think about how our minds perceive space. Why do you think that question gripped you so much?
Doris Tsao: It’s kind of funny, I always thought I was special, but my kids, they’re like six years old and they ask me that nowadays. So, I think it’s such a natural question. Maybe every kid wonders about this at some point.
Megan Hall: But Doris kept wondering about it. Still, she couldn’t pinpoint exactly what she was looking for. She says she went to the California Institute of Technology for college because she liked the idea of being a scientist.
Doris Tsao: And I had read all these books about the brain, and so on. So, I had romantic notions about that, but it was like sort of a fantasy about what my life could be like rather than motivated in a question about the world.
Megan Hall: Then something pretty common happened. She was on a camping trip with her dad and he asked her to proofread one of his academic papers. His first language is Chinese, so…
Doris Tsao: He would give me his papers to correct English mistakes. And I did this starting in middle school, high school, and I had no idea what his papers were – they were like gobbledygook – but I could figure out that the verb was not agreeing with the subject.
Megan Hall: But this time was different. With her training from Caltech, Doris actually understood what he was writing about.
Doris Tsao: It was kind of astonishing to me, like, the idea.
Megan Hall: The idea in her father’s paper described how our brains help us see the world in three dimensions.
Doris Tsao: I just thought it was so beautiful. It was like this idea that the brain is creating our perception, suddenly it had mathematical clothes. It was like a real framework for explaining how this works.
Megan Hall: That’s when everything fell into place.
Doris Tsao: Suddenly, my dream to understand 3D space and how’s infinite space possible, the sense of beauty that I could discover something beautiful about the brain, and then just this connection with my father. We’re an immigrant family. We always had a sense like we have to prove ourselves in this country, in this new land. So, all of those thoughts came together and it’s like, wow, I can go prove this amazing theory and yeah, I’ll go win a big prize.
Megan Hall: You did!
Doris Tsao: Yeah! It didn’t quite work out that way, but…
Megan Hall: Doris went on to graduate school at Harvard University to prove her dad’s theory about how our brains process the 3D world. And to do that, she spent years working with macaque monkeys. That’s because…
Doris Tsao: The visual system is almost identical to the human visual system in all these details. They’re just like replicas. It’s just a beautiful thing to behold.
Megan Hall: So, she gave the monkeys 3D goggles, showed them different images and used electrodes to measure what was happening in their brains – one neuron at a time. For three years…
Doris Tsao: I got nothing. I tell this to my students to reassure them. I tell them science is highly nonlinear. You can make zero progress, and then all of a sudden things take off.
Megan Hall: Things finally took off when she joined a research group at Massachusetts General Hospital. There, she helped with a project that measured blood flow across a monkey’s entire brain, using what’s called functional magnetic resonance imaging or fMRI.
Doris Tsao: fMRI gives you a bird’s-eye view of all the areas that are activated and how much they’re activated by a particular stimuli. So, it seemed an exciting opportunity to me.
Megan Hall: Around this time, Doris heard about the work of her fellow laureate Nancy Kanwisher.
Doris Tsao: She was the hot new professor at MIT back then. And I read her paper on this discovery of a face area in the human brain. It reported that there’s an area in the brain that just responds to faces.
Megan Hall: Doris was shocked and puzzled by this idea.
Doris Tsao: Because I didn’t feel from introspection that faces are all that different from anything else, and it just seems so understandable and simple. How can you have an area that just cares about faces? I thought it’d be some horribly complicated code, right?
Megan Hall: So, Doris decided to replicate Nancy Kanwisher’s human experiment with monkeys.
Doris Tsao: Show them pictures of faces and non-face objects, and compare – are there any voxels in a monkey’s brain that respond more to faces than other objects? It was a very low-risk experiment because the experiment’s so easy. It would be like one night of scanning. If it didn’t work, it’s totally fine.
Megan Hall: And during that night of scanning…
Doris Tsao: We saw this region light up to faces. We scanned the monkey again, and same region would light up again. And later, we got a better coil and we saw six of these regions light up, and it would always be the exact same six regions. Right? And then we scanned another monkey and it showed the same pattern, roughly. And also, the regions were located in very similar locations across the two hemispheres. And so, it was clear they weren’t random.
Megan Hall: After three unproductive years, Doris had stumbled on something new and replicable: six nearby patches of brain tissue that recognized faces.
Doris Tsao: It was the first time I had traction. I understood what this loop in science is, right? Before, it was just me chasing a fantasy. I had no idea what I was doing, but now it was like a closed loop. You have a new finding, now you have new questions.
Megan Hall: Her first question was what are the individual cells in these brain areas doing? Do all of them have something to do with faces?
Doris Tsao: I didn’t know what to expect. My expectations were not that high.
Megan Hall: Around this time, she teamed up with her fellow laureate Winrich Freiwald. They went back to the approach she’d used earlier in her research – implanting electrodes in a monkey’s brain to monitor how cells reacted to different images.
Doris Tsao: I remember our first experiment. We lowered the electrode. We got to the face area, where we expected the face area to be, and we could hear the electrical activity. We showed pictures of faces and other objects, and the cell would just go off to the faces. And it was very clear, there was a selectivity for faces. And then we recorded another cell and it was exactly the same. It responded more to faces.
Megan Hall: When this part of the research was done…
Doris Tsao: I never in my wildest imagination imagined what we actually found, which is that all the cells in this region were face-selective.
Megan Hall: That discovery opened up a whole new set of questions. How do all of these different brain areas relate to each other? What specifically is each cell doing when it comes to recognizing faces?
Doris Tsao: We just have to go figure it out like this treasure chest, like, what’s happening there?
Megan Hall: Doris spent the next 20 years digging deeper and deeper into these questions. Early on, she and her team discovered that some of these brain cells responded to specific parts of faces.
Doris Tsao: Some of them required the eyes and the hair to be present. Others required the nose and the mouth. Almost all of them required eyes. The eyes were a very, very effective feature.
Megan Hall: Other cells responded to where a face was looking.
Doris Tsao: So, cells in this one patch, we call it ML, or the middle face patch, the cells were very selective for view. So, if it responded to cells facing left, like left-profile faces, then it did not respond to faces from the lower to the right.
Megan Hall: Doris and her team also stimulated the six brain areas, or face patches, one at a time to see how all of them worked together.
Doris Tsao: And the results were just so beautiful. We’d stimulate one face patch, the other face patches would light up, as well as a few other areas. And so, it really gave us a sense of how these regions are connected.
Megan Hall: The discoveries just kept coming. Over time, Doris and her team built a comprehensive dictionary decoding how these face-recognizing cells work. At this point, she feels like she understands them completely, so much so that she can conduct her experiments in reverse.
Doris Tsao: We’ve done a demonstration where we can just listen to the neurons. We have no idea what the monkey’s seeing. And just knowing the dictionary and listening to the response of neurons, we can create a reconstruction of what we think the monkey’s seeing. And if you look at the reconstructions, they look exactly like what the monkey’s actually seeing.
Megan Hall: Whoa! So, you can draw a picture just based on what neurons are firing?
Doris Tsao: Yeah! Yeah.
Megan Hall: Doris received The Kavli Prize for this work, but it wasn’t really her passion…
Doris Tsao: I honestly had no particular interest in how we see faces.
Megan Hall: She still wanted to answer those bigger questions from that camping trip with her dad. How do we see 3D images? How does the brain represent the world? Luckily, what she’s learned about faces is helping Doris expand her research.
Doris Tsao: I’ve slowly realized that this problem of seeing a face is really a microcosm of the problem of seeing the world. It turns out that there’s a whole set of networks in this part of the brain, and they’re organized exactly like the face-patch network.
Megan Hall: Except, instead of faces, they recognize different types of shapes.
Doris Tsao: Something like spiky things versus stubby things, and curvy things versus blocky things. A spider would be a spiky thing. A helicopter is a spiky thing. A USB stick is a stubby thing. A radio is a stubby thing. So, every single object that you can think of, you can put it somewhere in this space.
Megan Hall: So, you are building this more and more detailed map of the brain, it sounds like?
Doris Tsao: Yeah! And now, the big question that we have now is how do we represent the entire scene?
Megan Hall: So far, all of these experiments have involved flashing images in front of monkeys and seeing how their brains respond. But that’s not how anyone actually sees the world.
Doris Tsao: In real-world vision, there’s so many different objects, and you have to represent not just what you’re looking at, but where other stuff is, right? So, you know where to look at next, for example. And so, that’s a big question that we’re tackling.
Megan Hall: And that’s the next frontier for you, in terms of what you’re trying to figure out?
Doris Tsao: Yeah. Yeah. We want to understand how the brain represents the world. That was always my dream.
Megan Hall: Back to that camping trip.
Doris Tsao: In sixth grade. Even before that, yeah.
Megan Hall: Decades after Doris had those original epiphanies, she’s finally back to where she wanted to be. She even published a paper with her dad about 3D vision.
Doris Tsao: We’ve been on a camping trip, and I was exposed to this beautiful idea, and then he and I have been discussing these ideas ever since.
Megan Hall: Doris says now she’s in the best place possible. She has her own lab, brilliant students and postdocs, and the freedom to delve deeper and deeper into the mystery of how our minds show us the world.
Doris Tsao is a professor of neuroscience at the University of California Berkeley and an investigator of the Howard Hughes Medical Institute. This year, she shares The Kavli Prize in Neuroscience with Nancy Kanwisher and Winrich Freiwald.
The Kavli Prize honors scientists for breakthroughs in astrophysics, nanoscience and neuroscience, transforming our understanding of the big, the small, and the complex. The Kavli Prize is a partnership among the Norwegian Academy of Science and Letters, the Norwegian Ministry of Education and Research, and the US-based Kavli Foundation.
This work was produced by Scientific American Custom Media and made possible through the support of The Kavli Prize.
