Pixar’s Senior Scientist explains how math makes the movies and games we love

Tony DeRose wanders between rows at New York’s Museum of Mathematics. In a brightly-colored button-up T-shirt that may be Pixar standard issue, he doesn’t look like the stereotype of a scientist. He greets throngs of squirrely, nerdy children and their handlers — parents and grandparents, math and science teachers — as well as their grown-up math nerd counterparts, who came alone or with their friends. One twentysomething has a credit for crowd animation on Cars 2; he’s brought his mom. She wants to meet the pioneer whose work lets her son do what he does.

“It’s wonderful to see such a diverse crowd,” he says. “How many of you have seen a Pixar film?” he asks after taking the podium. The entire room’s hands go up. “How many of you have seen three? Five?” He pauses. “How many of you have seen all of them?” Dozens of people raise their hands, maybe a quarter of the room. “Wow,” he says. He smiles, to himself and the crowd. This gig is not one bit bad.

ASPIRING ANIMATORS AND GAME DESIGNERS, STUDY YOUR CALCULUS AND COMBINATORICSThe topic of DeRose’s lecture is “Math in the Movies.” This topic is his job: translating principles of arithmetic, geometry, and algebra into software that renders objects or powers physics engines. This process is much the same at Pixar as it is at other computer animation or video game studios, he explains; part of why he’s here is to explain why aspiring animators and game designers need a solid base in mathematics. As Pixar’s Senior Scientist, DeRose has more than a solid base: PhD in computer science, specialty in computational physics, a decade as a professor of computer science and engineering at the University of Washington. This is the first instance of the Math Encounters lecture series at MoMath’s new campus in midtown Manhattan, but DeRose given a version of this talk many times before, continually updating it as Pixar’s technology improves and fans want to hear about the latest films.

Hair, cloth, fluids, and gaseous phenomena like clouds, smoke, and fire all have their own physics at Pixar. These basic engines are then augmented to try to produce specific outcomes. “Simulating water is easy,” says DeRose. “What’s hard is, how do you make water more directable?” For Brave, DeRose explains, Merida’s voluminous, bright red, highly animated curls required building an entirely new physics engine. The studio’s animators had to figure out how to make Merida’s hair beautiful, expressive, and even more living than lifelike. DeRose and his team of scientists had to engineer a model that makes that animation computationally possible.

HAIR, CLOTH, LIGHT, FLUIDS, AND GASES ALL HAVE THEIR OWN PHYSICS“In the real world, hair keeps its bounciness and volume by constantly colliding with itself,” DeRose says. Merida’s hair is made of 100,000 individual elements. “If you know any combinatorics, you know that if you have n objects, you have n² possible collisions,” he says, or 10 billion. How can you render so many collisions quickly enough to be usable? You have to create a new spatial data structure that culls extraneous collisions without being too lossy. Instead of a quick-and-dirty compression algorithm like MP3 or JPEG, Pixar has to create the equivalent of the PNG or the FLAC for animating hair.

Computer animation, DeRose says, frequently deals with modeling objects at greater scale and detail than even physicists typically deal with in their computations. Much of his work involves finding better algorithms to intelligently approximate this kind of scale without sacrificing detail. “Directors will say, ‘oh, it’s just a small thing in the background, we’ll never see it.’ Directors lie,” DeRose explained. And if every time a director changed his or her mind, objects or characters had to have their underlying physics redesigned from scratch, it would be impossible for Pixar to make a movie every year, with four teams working at once.

FROM POLYGONS TO PARABOLASDeRose’s most important contribution to computer animation has come from new ways of quickly generating smooth curves with high fidelity. “It’s all about turning complicated shapes into some form that computers will deal with.” For years, in both computer animation and video games, this meant mapping three-dimensional objects with planar facets, or polygons. But the problem with polygons is that at close detail, you can see every one of them — a fatal problem when the illusion depends on ignoring individual frames and pixels. The trend has been is to replace polygons with parabolas, curving surfaces that are continuous at arbitrary levels of detail. But you still need to define these curves quickly to match a finite number of points or planes. So mathematicians have worked to develop different methods for quickly generating smoothly curved surfaces. These are typically called subdivision surfaces because of how they’re calculated, by repeatedly splitting and averaging the midpoint of a line.

The first time subdivision surfaces were used extensively in computer animation was in the Oscar-winning 1997 Pixar short Geri’s Game. The leap between this film and earlier polygon-based animation is startling. It simply looks like Pixar. DeRose adapted his scholarly work on calculating wavelets on multidimensional surfaces to create a new curve-generating algorithm. And while it was first used only to solve especially thorny problems in animating characters — the complex shape of an old man’s nose, the particular warp and woof and movement of cloth — it’s now used for nearly every object in Pixar’s films. DeRose shows a still from The Incredibles, and gestures to the background. “You might be surprised that the building, its windows, all of the details are generated with subdivision surfaces,” he says. What started with a single short — and before that, applied research by DeRose and other computer scientists — has become an industry standard.

PIXAR, OPEN-SOURCE, AND THE FUTURE OF ANIMATIONDeRose and the research team at Pixar continue to publish papers and apply new techniques to their software engines, but the studio doesn’t have the same lead in R&D and proprietary software it once had. Controlling lighting and shading or defining the parameters for a character marionette used to be huge challenges to mathematically define and then engineer code. Now, says DeRose, open-source software like Blender can do almost everything Pixar’s software can do. Last summer, Pixar even open-sourced its subdivision surface code library. “We had a competitive advantage for ten years,” DeRose says, “but now we get more value by letting everyone contribute.”

Pixar’s biggest competitive advantage now is its ability to use this math-driven technology not to make better shapes but to tell better stories. DeRose and Pixar aren’t sitting on their laurels. “Somewhere out there, a brilliant kid and their friends are working in their garage” using and improving on tools like Blender, DeRose tells the assembled children and adults at MoMath. “They will be the next Pixar.”