Research Reveals the Optimal Way to Optimize

The original version of this story appeared in Quanta Magazine.

In 1939, upon arriving late to his statistics course at UC Berkeley, George Dantzig—a first-year graduate student—copied two problems off the blackboard, thinking they were a homework assignment. He found the homework “harder to do than usual,” he would later recount, and apologized to the professor for taking some extra days to complete it. A few weeks later, his professor told him that he had solved two famous open problems in statistics. Dantzig’s work would provide the basis for his doctoral dissertation and, decades later, inspiration for the film Good Will Hunting.

Dantzig received his doctorate in 1946, just after World War II, and he soon became a mathematical adviser to the newly formed US Air Force. As with all modern wars, World War II’s outcome depended on the prudent allocation of limited resources. But unlike previous wars, this conflict was truly global in scale, and it was won in large part through sheer industrial might. The US could simply produce more tanks, aircraft carriers, and bombers than its enemies. Knowing this, the military was intensely interested in optimization problems—that is, how to strategically allocate limited resources in situations that could involve hundreds or thousands of variables.

The Air Force tasked Dantzig with figuring out new ways to solve optimization problems such as these. In response, he invented the simplex method, an algorithm that drew on some of the mathematical techniques he had developed while solving his blackboard problems almost a decade before.

Nearly 80 years later, the simplex method is still among the most widely used tools when a logistical or supply-chain decision needs to be made under complex constraints. It’s efficient and it works. “It has always run fast, and nobody’s seen it not be fast,” said Sophie Huiberts of the French National Center for Scientific Research (CNRS).

At the same time, there’s a curious property that has long cast a shadow over Dantzig’s method. In 1972, mathematicians proved that the time it takes to complete a task could rise exponentially with the number of constraints. So, no matter how fast the method may be in practice, theoretical analyses have consistently offered worst-case scenarios that imply it could take exponentially longer. For the simplex method, “our traditional tools for studying algorithms don’t work,” Huiberts said.

Eleon Bach is a coauthor of the new result.

Photograph: Courtesy of Eleon Bach

But in a new paper that will be presented in December at the Foundations of Computer Science conference, Huiberts and Eleon Bach, a doctoral student at the Technical University of Munich, appear to have overcome this issue. They’ve made the algorithm faster, and also provided theoretical reasons why the exponential runtimes that have long been feared do not materialize in practice. The work, which builds on a landmark result from 2001 by Daniel Spielman and Shang-Hua Teng, is “brilliant [and] beautiful,” according to Teng.

“It’s very impressive technical work, which masterfully combines many of the ideas developed in previous lines of research, [while adding] some genuinely nice new technical ideas,” said László Végh, a mathematician at the University of Bonn who was not involved in this effort.

Optimal Geometry

The simplex method was designed to address a class of problems like this: Suppose a furniture company makes armoires, beds, and chairs. Coincidentally, each armoire is three times as profitable as each chair, while each bed is twice as profitable. If we wanted to write this as an expression, using a, b, and c to represent the amount of furniture produced, we would say that the total profit is proportional to 3a + 2b + c.

To maximize profits, how many of each item should the company make? The answer depends on the constraints it faces. Let’s say that the company can turn out, at most, 50 items per month, so a + b + c is less than or equal to 50. Armoires are harder to make—no more than 20 can be produced—so a is less than or equal to 20. Chairs require special wood, and it’s in limited supply, so c must be less than 24.

The simplex method turns situations like this—though often involving many more variables—into a geometry problem. Imagine graphing our constraints for a, b and c in three dimensions. If a is less than or equal to 20, we can imagine a plane on a three-dimensional graph that is perpendicular to the a axis, cutting through it at a = 20. We would stipulate that our solution must lie somewhere on or below that plane. Likewise, we can create boundaries associated with the other constraints. Combined, these boundaries can divide space into a complex three-dimensional shape called a polyhedron.