Visionary Images: The Lost Fractals of Benoît Mandelbrot

Many people know Benoît Mandelbrot from the computer screensavers of a pre-LCD era. Others have a deeper understanding of his mathematics, the repeating geometries that earned him the sobriquetFather of Fractals. Less appreciated, though, is the process underlying his work: Mandelbrot relied as much for guidance on visual imagery as whiteboard formulae. Primitive computer printouts were his maps to uncharted mathematical terrain, their dot-matrix patterns a “here be dragons” for the exploration of dynamical systems and chaos theory.

In 2008, fascinated by the interplay between imagery and scientific investigation, art historian Nina Samuel spent two weeks interviewing Mandelbrot in his Cambridge, Massachusetts home. After Mandelbrot passed away in 2010, she was allowed entry to his office, collecting some 300 printouts, sketches and notebook scribbles now on display in The Islands of Benoît Mandelbrot: Fractals, Chaos, and the Materiality of Thinking, an exhibition at the Bard Graduate Center in Manhattan.

“There is such an organic quality to these images,” said Samuel. “These are the images the scientists used when they were working, and not what was found on magazine covers or popularized in screensavers.”

The exhibition runs until Jan. 27, but for readers not fortunate enough to visit, Samuel took Wired on a guided virtual tour. Each entry is followed by links to high-resolution versions of the images.

Below:

Becoming the Mandelbrot Set

The image below comes from a series of 120 prints, some composed of seemingly stray dots and others almost completely blank, that preceded Mandelbrot’s discovery of the fractal set that bears his name(visualized above, as produced by a modern pattern generator).

“If you look at the shadows, you can see the resemblance,” Samuel said of the early image. “Later on, he could have seen the shapes that became famous.” But at the time, Mandelbrot saw only an undefinedsomething, a hint of what needed next to be done.

“Mandelbrot would never say that an image was a proof, but an image would lead to conjecture, open up the imagination, and then you could prove something with formulas,” said Samuel.

Images: 1) Geek3/Wikimedia Commons [high-resolution] 2) Benoît Mandelbrot and Mark Laff, programmer. Collection Aliette Mandelbrot. [high-resolution]

Mandelbrot at IBM

In 1958, Mandelbrot took a job at IBM, where he’d stay for the next 35 years. In part he went because they had the best, most powerful computers, capable of displaying the graphics that would be so crucial to his work, but also because of the intellectual freedom that existed within the company’s research divisions.

Educated in post-World War II France, Mandelbrot chafed at the restrictions of a mathematical community that largely frowned on imagery as a tool, preferring to conduct their work in traditional, non-visual formulae.

“He had such a visual way of thinking,” Samuel said. “Being raised within these completely non-visual, very formalist mathematics was a pain for him. At IBM he had possibilities he could never have in academia. He could concentrate completely on his visual research.”

The image above is a press photograph produced by IBM in the mid-1980s. On the computer is an image produced by programmer Richard Voss and Mandelbrot for The Fractal Geometry of Nature, which was printed in 1982 and famously described how fractal math could be seen throughout the natural world, from snowflakes to mountainsides.

Image: IBM Corporation. [high-resolution]

Into the Unknown

An early entry in the series of experimental images that led to the Mandelbrot set, and looking almost like a the mathematician channeled Jackson Pollock, this print underscores the role of image in Mandelbrot’s work: There’s a pattern, but apprehended less through formal analysis than by intuition.

“They understood something only by looking at these computer pictures,” said Samuel of Mandelbrot and his colleagues. “Before printing these structures, nobody knew what they would look like. Every image was a surprise.”

Image: Benoît Mandelbrot. Collection Aliette Mandelbrot.[high-resolution]

On the Edge

Notated printouts are common in the exhibition, but the handwriting of Mandelbrot (or an assistant, as in the image below) mark more than the usual note-taking. They often describe points at which mathematics pushed beyond the technical capacities of an era’s computers, with theoretical details subsumed and distorted in pixelated, grayscale blocks.

The essential challenge of Mandelbrot and his peers, said Samuel, was “to try to make something visible where you didn’t know what it should look like, or what you could expect, and it wasn’t easy to get anything on paper.”

Images: 1) Benoît Mandelbrot. Collection Aliette Mandelbrot. [high-resolution] 2) Benoît Mandelbrot. Collection Aliette Mandelbrot. [high-resolution]

The Importance of Drawing

Often Mandelbrot and his peers — the drawing above was penned by Adrien Douady and John Hubbard, who made vital contributions to the mathematics which took Mandelbrot’s name — eschewed computers in favor of drawing by hand. Though computer limitations played a role in this choice, there was something cognitively powerful about having a direct, manual connection to their work.

“In order to relate the visual shapes that appeared on a computer’s screen to mathematical theory, they had to be worked with,” explained Samuel. “The hand was used to understand the structure.”

Image: Adrien Douady & John Hubbard. Private collection. [high-resolution]

Filling the Gaps

Look closely and you can see that portions of this print, made by Mandelbrot and programmer Sig Handelman, have been filled in with blue pen. Often Mandelbrot and his colleagues would draw directly onto their printouts, completing in by hand what they intuited but couldn’t yet describe.

“It was like a sculptural process. The pictures on the sheets of paper had to be sculpted,” Samuel said. “The mathematicians told me that the moment they had these drawings, they understood the formula and the equations. They understood what they yet had to prove.”

Image: Benoît Mandelbrot and Sigmund Handelman, programmer. Collection Aliette Mandelbrot. [high-resolution]

Bug or Breakthrough?

Relying as they did on relatively rudimentary computers and visualizing technologies, Mandelbrot and his colleagues were often confronted with a conundrum: Visual inconsistencies might result from human mistake or program error, but they could also represent mathematical reality.

In the image above, a representation intended to resemble the figure below, “it’s clear that something went wrong,” said Samuel. “But often, at the beginning of the process, one could not know: Was this a bug? Or was it supposed to look like this?”

Images: 1) Benoît Mandelbrot and Sigmund Handelman, programmer. Collection Aliette Mandelbrot. [high-resolution] 2) Benoît Mandelbrot and Sigmund Handelman, programmer. Collection Aliette Mandelbrot. [high-resolution]

Natural Nature

Convincingly natural computer-generated landscapes are common nowadays, expected in all but the lowest-budget movies and videogames. Just a few decades ago, though, they were a programmer’s pipe dream, realized only when Mandelbrot and colleagues described the underlying math.

“We have to see these through historical glasses. It was totally not given or normal at the time. It was the first time in history that these landscapes could be generated mathematically,” said Samuel of the now-dated renderings above and below, which were made by Mandelbrot and Richard Voss.

Images like these represent a transition from theoretical questions to commercial applications. Though few viewers at the time realized it, Mandelbrot’s popular debut came via the landscapes of 1982’s Star Trek II: The Wrath of Khan.

Images: 1) Benoît Mandelbrot and Richard Voss. Collection Aliette Mandelbrot. [high-resolution] 2) Benoît Mandelbrot and Richard Voss. Collection Aliette Mandelbrot. [high-resolution]

Accidental Art

A photographic screengrab of a landscape rendering’s early iterations, the image above “looks like design, like something that could be early 20th century art,” said Samuel. “But it’s just the calculating process, part of the production.” The completed landscape can be seen below.

Images: 1) Benoît Mandelbrot and Richard Voss. Collection Aliette Mandelbrot. [high-resolution] 2) Benoît Mandelbrot and Richard Voss. Collection Aliette Mandelbrot. [high-resolution]

Mandelbrot’s Vision

Mandelbrot assembled this collage as a suggested cover design for his 1975 treatise Fractals: Form, Chance, and Dimension. The profusion of straight lines is an abstract interpretation of stars, while at right are the natural forms Mandelbrot tried to simulate with his own mathematics.

“My idea is that he put these two things together to show the difference between straight lines, Euclidean geometry, and his new invention which was fractal geometry, the broken lines,” speculated Samuel. “It was not taken. The editors said, ‘This is too much image. It looks too crazy!'”