Escalating complexity, a shortage of trained workers, and crass politicization mean that most programs to develop new military systems fail to meet expectations
Robert N. Charette
Spectrum, Institute of Electrical and Electronics Engineers
November 2008
In the vast and varied world of advanced military technology, the U.S. Army's proposed Aerial Common Sensor aircraft was neither the biggest nor the sexiest. But it was very important to the U.S. Army. The Army desperately needed the ACS to replace its Guardrails, a fleet of small, piloted reconnaissance aircraft that first began flying in the mid-1970s.
The Army was so keen on the ACS, in fact, that it showcased the program to Congress and the American public as an example of cost-efficient and timely procurement. The service was still smarting from several high-profile failures, including the cancellations of the Crusader howitzer and the Comanche helicopter. And so, in August 2004, when the Army awarded a US $879 million five-year contract for the ACS to a team from Lockheed Martin and the Brazilian aircraft company Embraer, it had high hopes. Lockheed did too: it stood to earn an additional $7 billion once the plane entered production.
Almost immediately, things started to go wrong. Barely four months after signing the paperwork, the contractors revealed that their aircraft, based on an Embraer commercial jet, was rated for only 9 gs of force, not the 16 that the Army wanted. Fixing the problem drove up the plane's weight, putting it 1400 kilograms over its safety threshold.
A panel brought in to assess the ACS concluded that the program was "un-executable" and estimated it would cost at least another $900 million and two more years to get it back on track. So the Army canceled the program 18 months after it formally began, paying Lockheed $200 million for its trouble. The Army is now faced with spending $462 million over the next eight years to keep those old Guardrails flying.
To summarize: a big, basic technological problem emerged long after such an obstacle should have been spotted and resolved, in a program that would have been hugely over budget and behind schedule had it been left to run its course. But those factors aren't what made the ACS unusual. The Government Accountability Office, the U.S. Congress's investigative arm, recently scrutinized 72 major defense programs and found that only 11 of them were on time, on budget, and meeting performance criteria. No, what set the ACS apart was that it was not allowed to linger for years, racking up more costs and delays-it was actually canceled. Less than 5 percent of major defense programs ever suffer that fate, the GAO says.
Problems in defense acquisitions have existed for decades. What's new is the economic scale. The Pentagon now spends about $21 million every hour to develop and procure new defense systems. As recently as the mid-1990s, the largest of these cost tens of billions of dollars. Today, some, like the Army's Future Combat Systems, now range in the hundreds of billions. Topping the charts is the F-35 jet fighter, which is expected to cost taxpayers an astounding $1 trillion to develop, purchase, and operate. That sum is close to what the United States spent to fight both the Korean and Vietnam wars.
The spiraling costs are linked to schedule delays that are equally troubling. In 2007, the GAO estimated that current programs in development were experiencing an average delay of 21 months, with a few programs nearly a decade behind schedule. A 2007 GAO report to Congress noted that because of such delays and cost overruns, "not only is the buying power of the government reduced and opportunities to make other investments lost, but the warfighter receives less than promised."
Behind the deterioration is a convergence of factors, say analysts both inside and outside the Defense Department New military systems are more technologically complex than ever before, and they rely increasingly on unproven technologies. Defense programs are now "so massive and so fanciful we don't know how to get there," says Katherine Schinasi, the GAO's managing director of acquisitions and sourcing management. And engineers, scientists, and technicians skilled enough to design, build, and debug such complex systems are scarce. The ranks of DOD acquisitions managers have also been thinned, even as the number of major military programs has risen.
Too often, also, politics trumps technology and even common sense. DOD managers and service brass aren't the only people who have a stake in which military systems get developed and which don't: congressmen, defense contractors, lobbyists, and economic development officials are all aggressive players in the weapons-acquisition process, all pushing for their own pet projects. The result is a proliferation of development efforts that the Pentagon cannot fully fund and that are nearly impossible to cancel. Politics also leads contractors to overpromise on what they can deliver and leads DOD staffers to turn a blind eye when those promises aren't met.
The situation has become so dire that even former top defense officials are weighing in with uncharacteristic pessimism and alarm. "The [U.S.] military has been living in a rich man's world," says Jacques Gansler, a former undersecretary of defense for acquisition, technology, and logistics. "Very little attention has been paid to cost. We're coming up against a very serious fiscal crisis."
This month, U.S. voters will elect a new president. Whoever takes office in January will need to make reforming the weapons-acquisition process a high priority. "The next president is going to need to set the tone very, very quickly," says David M. Walker, who was head of the GAO until March. "If we don't make some fairly dramatic and fundamental acquisition reforms soon, we could find ourselves in a situation in the not-too-distant future where we have a lot of things that people wanted but not enough of what we really need."
Army Gen. Edward Hirsch was a blunt-talking man who spent many years after retirement trying to improve the acquisition process. Among other things, he was instrumental in establishing the School of Program Managers at the Defense Acquisition University. The DAU, at Fort Belvoir, Va., provides mandatory training for the Defense Department's 140 000 military and civilian acquisitions personnel.
Shortly before he passed away last year, at 85, Hirsch told me that he was deeply troubled by the present state of affairs. He felt that the process the Pentagon uses to develop and acquire new military systems actually works against those who try to follow it. "Put good people into a system that is designed for failure--it isn't going to work," Hirsch said.
On the surface, at least, it's hard to see why Hirsch felt the way he did: the defense-acquisition process looks perfectly logical. Acquiring a major defense system generally consists of nine steps, each step having multiple criteria that must be met before moving on to the next step. A "major" program in the DOD realm is any acquisitions effort whose research, development, testing, and evaluation costs exceed $365 million or whose procurement costs exceed $2.19 billion, in FY 2000 constant dollars. There are now roughly 95 such programs on the books. It's a traditional top-down approach that should be familiar to any engineer:
First, you determine and validate your national and military strategies and ensure that they align;
Second, identify the missing defense capabilities you will need to carry out your strategies;
Third, identify alternative approaches and their technical feasibility;
Fourth, select the best approach in terms of technology, cost, and schedule;
Fifth, get the budget and schedule for your approach approved;
Sixth, design and then implement the system;
Seventh, test the system in operational conditions;
Eighth, produce the vetted system in the quantities needed; and
Ninth, support and upgrade the system until its retirement from service.
Repeat each step as necessary.
The process is intentionally long and iterative, each step aimed at reducing the risk of failure and increasing the likelihood of meeting cost, schedule, and technical promises. You might expect then that the normal outcome would be a successful defense system. In fact, though, the process has nearly the opposite effect.
It now takes years-more than 110 months on average-for a major military program, once funded, to wend its way through this process. Some programs last even longer: the Marines' EFV amphibious vehicle is projected to take more than 20 years before it gets fielded. While the weapons program makes its way slowly and methodically through the nine steps, the defense strategy that gave rise to it moves on, in response to new threats, shifting geopolitics, and changing imperatives.
So why does the Pentagon stick with such a slow and flawed system? Because many in the DOD believe that following "hundreds and hundreds and hundreds of regulations that say, 'do this and not that' and 'check this over five times'รข¿¿" means that acquisition risk is under control, says the GAO's Schinasi.
Moreover, the best way to protect yourself from accusations of failure is to simply follow the process, says Thomas Lassman, a historian with the Pentagon-sponsored Defense Acquisition History Project. "Then you can't be blamed," Lassman says, "because you followed procedures correctly."
The complexity of military systems stems increasingly from their interconnectedness to other systems. One of the Pentagon's biggest, most ambitious, and most controversial programs is the Future Combat Systems (FCS), a sprawling effort to digitally link battlefield vehicles, sensors, and communications gear and improve their interoperability. The FCS consists of 14 individual systems, including manned and robotic ground and air vehicles, software radios, and satellites, as well as an overarching network and operating system tying those components together. Developing such enormous "systems of systems" poses technical and management problems that are neither well defined nor well understood. The software alone-95 million lines of code for the FCS, at last count-poses a daunting challenge. Nobody has yet figured out a way to develop reliable, secure software for much smaller projects. Yet now the DOD is contemplating systems beyond the FCS that would require more than a billion lines of code.
$21 million: Amount the Pentagon Spends Per Hour To Procure New Defense Systems
Another factor contributing to program failure is the shortage of technically trained people, especially systems engineers. A systems engineer translates technical needs into an overall system architecture that creates the best operational capability at the most affordable cost. As a project proceeds and goals or needs shift, systems engineers have to determine the difficult but necessary cost, schedule, and performance trade-offs to keep everything on track. As programs get bigger and more complex, the need for rigorous systems engineering increases.
But the ranks of DOD systems engineers have not increased. "One of the reasons we have such problems in systems engineering today is that we don't have people with the training and experience," says Paul Kaminski, a former U.S. undersecretary of defense for acquisition and technology who recently headed up a National Research Council study on the state of systems engineering in defense. "Gaining the needed expertise," he adds, "is not a 3-year proposition but a 10- to 15-year proposition." In canceling the contracts for the U.S. Navy's Littoral Combat ship program in 2007, Pentagon officials cited inadequate systems engineering as one of the main causes of the huge cost overruns that prompted the cancellations.
Systems engineers are not the only professionals in short supply. Over the past two decades, the DOD has outsourced much of its scientific and engineering expertise to industry. In many of its programs, the Pentagon now has one private contractor for every full-time civilian employee. In some cases, the DOD has admitted, contractors are doing jobs that should be performed only by federal employees, such as weapons procurement and contract preparation. The DOD has also outsourced the systems integration and management of some of its most critical programs, including the FCS.
The Pentagon believes outsourcing saves money, but the practice has depleted the ranks of technical, managerial, and contractual personnel who can provide effective oversight of the department's defense acquisitions. As Norman Augustine, former chairman of Lockheed Martin, told me, "If you're not smarter than your suppliers, you can't manage them effectively."
The situation will only get worse. This year alone, nearly one-fourth of the United States' 637 000 aerospace workers are eligible to retire. Both Northrop Grumman and Lockheed Martin, two of the biggest U.S. defense contractors, report that well over half of their workers will reach retirement age within the next five to 10 years. The same is true of DOD federal employees. Younger workers are not rushing in to replace them. Many students who pursue scientific and technical disciplines don't want careers in defense-related work. The implications for the DOD are grim.
Though personnel skilled in the arts of managing acquisition programs are on the wane, the programs themselves are on the rise, along with the Pentagon's budget. The FY 2009 defense budget of $488 billion is the largest in real terms since World War II and 6 percent higher than this year's budget. Meanwhile, the total projected development costs for the 95 or so major weapons systems currently in the pipeline have more than doubled in the last seven years, from $790 billion in 2000 to $1.6 trillion in 2007.
There are no quick or easy solutions to labor shortages and escalating costs, but the standard response in the private sector is to lean more heavily on automation and information technologies. But the DOD seems not to have benefited from technological advances that would make development less expensive. "One might have thought that more efficient production methods, including computer-aided design and manufacturing, microminiaturization of components, and the employment of greater computing power, all would have reduced costs or at least held them level," former DOD comptroller Dov Zakheim told a Washington, D.C., audience in January 2005. The fact that they haven't, he said, is "not easy to fathom."
Definition: Next-war-itis n: The creation of complex and expensive machinery for possible future conflicts while ignoring the present need for affordable weapons.
Ironically, the solution supported by the DOD and military service chiefs, as well as some members of Congress, is not to make acquisitions more efficient but to spend even more money. They advocate setting aside an amount equal to at least 4 percent of the annual U.S. gross domestic product for defense. At the current GDP of about $16 trillion, that would mean an annual defense budget of $640 billion.
Would even that much be enough? The wars in Iraq and Afghanistan have revealed that U.S. military strategy still focuses on winning conventional force-on-force wars and that it lacks the systems to fight asymmetrical or irregular wars against, for example, insurgents and militias. Secretary of Defense Robert Gates has criticized his own agency for what he calls "next-war-itis." The military services and defense contractors, he says, are too focused on creating complex and expensive machinery for possible conflicts far into the future and not sufficiently attentive to providing affordable weapons that the military can use right now.
So, given the fast-changing military imperatives of these times, what constitutes a successful acquisition program? There is no straightforward answer to that question, partly because in the realm of military acquisitions, there is no universally agreed upon definition of a "successful" program Jacques Gansler, now a professor of public policy at the University of Maryland, has a definition that seems as good as any other: one that "satisfies a military need for an intended cost and intended schedule."
By that definition, most major defense programs would be failures. The fact that only 5 percent ever do get canceled means that the defense community doesn't hold itself to a high standard.
"The definition of success in DOD is to start a program," the GAO's Schinasi says. "That turns on the program's funding. [Success] has nothing to do with the eventual fielding" of a system.
Ronald Fox, professor emeritus at Harvard Business School, adds that after a program gets funded, the definition of success changes again. "A successful program is one that hasn't been canceled," says Fox, who has studied defense acquisitions for over 40 years.
Defined that way, "success" can look an awful lot like what many people would call failure. Seven years ago, the U.S. Air Force awarded a $3.9 billion contract to Boeing to outfit its C-130 cargo aircraft with digital cockpits, which are equipped with monitor screens rather than analog gauges. But Boeing grossly underestimated how much engineering work it would require to modify the C-130's many different configurations. By last year, the program had gone so far over budget that it triggered a congressionally mandated review. The Air Force's response was not to cancel the program but rather to cut the number of planes getting the upgrade from 519 to 222, thereby "saving" a projected $560 million. Nevertheless, the total program still came in $1.4 billion over budget.
Why not just cancel such a program? For one thing, cancellation means lost jobs-and votes. Loren Thompson, a defense analyst at the Lexington Institute, puts it this way: "Most of the time what [the acquisition process] is trying to achieve is only partially 'equipping a soldier in the field.' It is also concerned with getting a congressperson reelected, advancing the career of a bureaucrat, and making certain that the defense-industrial base is sustained during periods of low demand." Politicians, urged on by lobbyists and defense contractors, routinely support programs that should have been killed or should never have been funded, he says.
That sort of collective conspiracy extends to the wildly optimistic promises that contractors make to win funding. Such optimism usually takes the form of "understating the cost [of a program] and overstating the technical requirements," says Fox. For example, company A claims it can produce its widgets for $1 apiece and that they will accomplish X, Y, and Z; it will win out over company B, which pitches its widgets at a more realistic $5 and says they will do only X.
The contractors are not solely to blame for this shell game. "Before you know whether the system will work, you have to define the price of all the units you expect to buy," notes Ron Kadish, former director of the DOD's Missile Defense Agency and now a vice president at consulting firm Booz Allen Hamilton. The cost estimate is always going to be wrong, he says, but everyone, including the DOD and military service procurement officials and Congress, pretends that it is correct. This intellectual dishonesty leads to expectations that can never be met.
"The bureaucratic incentives at work in the acquisition system militate fairly strongly against honesty," says the Lexington Institute's Thompson. "Until a weapon system is put into operational test and then must perform, there are lots of rewards for understating costs, for understating technical challenges, and for exaggerating the speed at which costs and technical problems can be overcome."
To be fair, part of that exaggeration stems from the real engineering problem of designing a system that has to meet some theoretical threat 5 or 10 or 15 years from now. If you had to design a car of the future, what technology would you put in it? Would it rely on just what's available today, or would it need to accommodate a power source or steering mechanism that doesn't yet exist? Even when you settle on a design, innovations will inevitably arise during the many years that your system is in development.
Dependence on unproven technology is anathema in the commercial world, but it's common in defense programs. The design for the Army's Crusader howitzer, for instance, relied on 16 "critical" technologies, including advanced armaments, ammunition handling, and mobility. But only six of those technologies had ever been demonstrated outside the laboratory when the Crusader entered development in 1994. Subsequent problems with those untested technologies contributed to the doubling of the program's development cost-and ultimately to its cancellation in 2002.
In their landmark 1962 book, The Weapons Acquisition Process: An Economic Analysis , Harvard professors Merton Peck and Frederic Scherer noted that the fundamental objective of acquiring any military system is that it either create a qualitative superiority over an enemy's weapon system or neutralize the enemy's superiority-not only today but into the future. Thus the eternal push for better fighting technology, from sharpened stones to GPS-guided bombs.
Each technological advance a country achieves should not only enhance its own military position but should also degrade the enemy's. For example, making a bomber stealthy enables it to be more destructive to the enemy's key installations. This situation is quite different from the commercial world. Buying an iPhone may rock your world, but it won't have any ill effects (except maybe envy) on your friend who owns a Motorola Razr.
This mind-set tacitly encourages the DOD to demand, and contractors to propose, ever more sophisticated technology. As one former senior military program manager put it, "to sell a program today, you need to claim that it is 'transformational' in some way."
But that quest for the "transformational" is now colliding with the hard reality that many of the fundamental technologies in today's weapons systems are already very advanced. The engines, avionics, and flight controls in military aircraft, for instance, are all close to the limits of what is possible. Even incremental advances come at enormous cost.
And so the infatuation with immature and exotic technologies, with their high costs and risks. James Finley, deputy undersecretary of defense for acquisition and technology, admits that in many programs "technology is being pushed too fast, too soon."
The widespread inability to meet promises creates a vicious circle: lowballed cost projections allow too many programs to be approved; as the projections for each program repeatedly get revised upward, the defense budget balloons; eventually, cuts have to be made, resulting in what military critic Chuck Spinney has termed the "defense death spiral." In congressional testimony in 2002, Spinney described the spiral as "shrinking combat forces, decreasing rates of modernization, aging weapons inventories, with the rising cost of operations creating continual pressure to reduce readiness."
Over the last six decades a dozen or so blue-ribbon panels and at least a hundred initiatives have called for detailed, concrete reforms in defense acquisitions. So there isn't much doubt that something is fundamentally wrong. What is most disheartening is that everyone knows it and nobody-not DOD management, not the military services, not Congress-has done much about it. "The problem in Washington isn't what we don't know but what we don't want to know," says defense analyst Thompson.
The most recent of these reform efforts was the 2005 Defense Acquisition Performance Assessment, led by Ron Kadish. The panel came up with 35 recommendations in all, some repeating earlier committees' findings, some new to that report. Among the people I interviewed who had read the Kadish report, nearly everyone said that three of the recommendations in particular would have an immediate impact on the acquisitions process, without having to change existing regulation or win congressional approval.
25 percent of the 637 000 U.S. Aerospace Workers are currently eligible to retire
The first of these involves trying to break the defense death spiral by returning to a "block program" approach, in which systems would be built incrementally in capability, instead of trying to satisfy every mission requirement in the first increment. The second recommendation is that programs be "time certain," meaning that most programs would have to deliver some useful operational capability within five years. That requirement would force contractors to use only technologies that are essentially mature, rather than ones that would need to be invented on a schedule.
The third recommendation is to fund only those programs that have an 80:20 cost confidence level, meaning the program has an 80 percent chance of meeting its estimated cost target. Traditionally, the DOD has aimed for a 50:50 confidence level, and some programs don't even reach that cutoff. The Army's FCS program, for instance, was approved in 2003 with "somewhere down around [a] 28 percent chance of success," according to then Army chief of staff Peter J. Schoomaker.
If the DOD were to implement all three recommendations-funding only those programs that it deemed to have an 80 percent chance of succeeding, could deliver operational capability in five years, and could be developed incrementally-it would effectively cut the number of new programs by up to 25 percent. Unfortunately, the Pentagon and the military services have shown no great willingness to scale back, even though it would likely mean getting systems out into the field more quickly.
Still, the DOD's Finley insists that the department is at least attempting to implement all of the Kadish panel's recommendations. But will they in fact fix a broken system? Finley is quite optimistic, but even he admits, "I can't predict the will for change."
Indeed, if there's one sentiment that has been repeated more often than any other in the past 60 years of failed attempts at acquisition reform, it is the need for "the will to change." That quality always seems to be in short supply.
In this election season, neither of the two presidential candidates has had much to say about reforming defense acquisitions. That's disappointing but not surprising. No politician wants to be accused of not supporting the troops. Any suggestion that defense spending be reined in has become, like Social Security, a political third rail. Most defense experts are skeptical that the Pentagon or Congress or the White House will fundamentally alter the current way of doing business. There is too much money and too many jobs at stake.
But reform will have to come. Each day that the acquisition process continues to operate ineffectively and inefficiently is another day that the troops are not getting what they need, the country is less secure, and much-needed programs, both civilian and military, don't get funded.
The next administration will need to choose wisely, and soon.
ROBERT N. CHARETTE, an IEEE Spectrum contributing editor, is a self-described "risk ecologist" who investigates the impact of the changing concept of risk on technology and societal development. In writing the cover story, "What's Wrong With Weapons Acquisitions?", he spent over 18 months interviewing dozens of industry and government defense-acquisition experts. "Everyone knows the acquisition process desperately needs to change," he says. "Unfortunately, no one I spoke to believes it will change until there is a major national crisis."
Citation: Robert N. Charette, What's Wrong with Weapons Acquisitions?, IEEE Spectrum (November 2008)
Robert N. Charette
Spectrum, Institute of Electrical and Electronics Engineers
November 2008
In the vast and varied world of advanced military technology, the U.S. Army's proposed Aerial Common Sensor aircraft was neither the biggest nor the sexiest. But it was very important to the U.S. Army. The Army desperately needed the ACS to replace its Guardrails, a fleet of small, piloted reconnaissance aircraft that first began flying in the mid-1970s.
The Army was so keen on the ACS, in fact, that it showcased the program to Congress and the American public as an example of cost-efficient and timely procurement. The service was still smarting from several high-profile failures, including the cancellations of the Crusader howitzer and the Comanche helicopter. And so, in August 2004, when the Army awarded a US $879 million five-year contract for the ACS to a team from Lockheed Martin and the Brazilian aircraft company Embraer, it had high hopes. Lockheed did too: it stood to earn an additional $7 billion once the plane entered production.
Almost immediately, things started to go wrong. Barely four months after signing the paperwork, the contractors revealed that their aircraft, based on an Embraer commercial jet, was rated for only 9 gs of force, not the 16 that the Army wanted. Fixing the problem drove up the plane's weight, putting it 1400 kilograms over its safety threshold.
A panel brought in to assess the ACS concluded that the program was "un-executable" and estimated it would cost at least another $900 million and two more years to get it back on track. So the Army canceled the program 18 months after it formally began, paying Lockheed $200 million for its trouble. The Army is now faced with spending $462 million over the next eight years to keep those old Guardrails flying.
To summarize: a big, basic technological problem emerged long after such an obstacle should have been spotted and resolved, in a program that would have been hugely over budget and behind schedule had it been left to run its course. But those factors aren't what made the ACS unusual. The Government Accountability Office, the U.S. Congress's investigative arm, recently scrutinized 72 major defense programs and found that only 11 of them were on time, on budget, and meeting performance criteria. No, what set the ACS apart was that it was not allowed to linger for years, racking up more costs and delays-it was actually canceled. Less than 5 percent of major defense programs ever suffer that fate, the GAO says.
Problems in defense acquisitions have existed for decades. What's new is the economic scale. The Pentagon now spends about $21 million every hour to develop and procure new defense systems. As recently as the mid-1990s, the largest of these cost tens of billions of dollars. Today, some, like the Army's Future Combat Systems, now range in the hundreds of billions. Topping the charts is the F-35 jet fighter, which is expected to cost taxpayers an astounding $1 trillion to develop, purchase, and operate. That sum is close to what the United States spent to fight both the Korean and Vietnam wars.
The spiraling costs are linked to schedule delays that are equally troubling. In 2007, the GAO estimated that current programs in development were experiencing an average delay of 21 months, with a few programs nearly a decade behind schedule. A 2007 GAO report to Congress noted that because of such delays and cost overruns, "not only is the buying power of the government reduced and opportunities to make other investments lost, but the warfighter receives less than promised."
Behind the deterioration is a convergence of factors, say analysts both inside and outside the Defense Department New military systems are more technologically complex than ever before, and they rely increasingly on unproven technologies. Defense programs are now "so massive and so fanciful we don't know how to get there," says Katherine Schinasi, the GAO's managing director of acquisitions and sourcing management. And engineers, scientists, and technicians skilled enough to design, build, and debug such complex systems are scarce. The ranks of DOD acquisitions managers have also been thinned, even as the number of major military programs has risen.
Too often, also, politics trumps technology and even common sense. DOD managers and service brass aren't the only people who have a stake in which military systems get developed and which don't: congressmen, defense contractors, lobbyists, and economic development officials are all aggressive players in the weapons-acquisition process, all pushing for their own pet projects. The result is a proliferation of development efforts that the Pentagon cannot fully fund and that are nearly impossible to cancel. Politics also leads contractors to overpromise on what they can deliver and leads DOD staffers to turn a blind eye when those promises aren't met.
The situation has become so dire that even former top defense officials are weighing in with uncharacteristic pessimism and alarm. "The [U.S.] military has been living in a rich man's world," says Jacques Gansler, a former undersecretary of defense for acquisition, technology, and logistics. "Very little attention has been paid to cost. We're coming up against a very serious fiscal crisis."
This month, U.S. voters will elect a new president. Whoever takes office in January will need to make reforming the weapons-acquisition process a high priority. "The next president is going to need to set the tone very, very quickly," says David M. Walker, who was head of the GAO until March. "If we don't make some fairly dramatic and fundamental acquisition reforms soon, we could find ourselves in a situation in the not-too-distant future where we have a lot of things that people wanted but not enough of what we really need."
Army Gen. Edward Hirsch was a blunt-talking man who spent many years after retirement trying to improve the acquisition process. Among other things, he was instrumental in establishing the School of Program Managers at the Defense Acquisition University. The DAU, at Fort Belvoir, Va., provides mandatory training for the Defense Department's 140 000 military and civilian acquisitions personnel.
Shortly before he passed away last year, at 85, Hirsch told me that he was deeply troubled by the present state of affairs. He felt that the process the Pentagon uses to develop and acquire new military systems actually works against those who try to follow it. "Put good people into a system that is designed for failure--it isn't going to work," Hirsch said.
On the surface, at least, it's hard to see why Hirsch felt the way he did: the defense-acquisition process looks perfectly logical. Acquiring a major defense system generally consists of nine steps, each step having multiple criteria that must be met before moving on to the next step. A "major" program in the DOD realm is any acquisitions effort whose research, development, testing, and evaluation costs exceed $365 million or whose procurement costs exceed $2.19 billion, in FY 2000 constant dollars. There are now roughly 95 such programs on the books. It's a traditional top-down approach that should be familiar to any engineer:
First, you determine and validate your national and military strategies and ensure that they align;
Second, identify the missing defense capabilities you will need to carry out your strategies;
Third, identify alternative approaches and their technical feasibility;
Fourth, select the best approach in terms of technology, cost, and schedule;
Fifth, get the budget and schedule for your approach approved;
Sixth, design and then implement the system;
Seventh, test the system in operational conditions;
Eighth, produce the vetted system in the quantities needed; and
Ninth, support and upgrade the system until its retirement from service.
Repeat each step as necessary.
The process is intentionally long and iterative, each step aimed at reducing the risk of failure and increasing the likelihood of meeting cost, schedule, and technical promises. You might expect then that the normal outcome would be a successful defense system. In fact, though, the process has nearly the opposite effect.
It now takes years-more than 110 months on average-for a major military program, once funded, to wend its way through this process. Some programs last even longer: the Marines' EFV amphibious vehicle is projected to take more than 20 years before it gets fielded. While the weapons program makes its way slowly and methodically through the nine steps, the defense strategy that gave rise to it moves on, in response to new threats, shifting geopolitics, and changing imperatives.
So why does the Pentagon stick with such a slow and flawed system? Because many in the DOD believe that following "hundreds and hundreds and hundreds of regulations that say, 'do this and not that' and 'check this over five times'รข¿¿" means that acquisition risk is under control, says the GAO's Schinasi.
Moreover, the best way to protect yourself from accusations of failure is to simply follow the process, says Thomas Lassman, a historian with the Pentagon-sponsored Defense Acquisition History Project. "Then you can't be blamed," Lassman says, "because you followed procedures correctly."
The complexity of military systems stems increasingly from their interconnectedness to other systems. One of the Pentagon's biggest, most ambitious, and most controversial programs is the Future Combat Systems (FCS), a sprawling effort to digitally link battlefield vehicles, sensors, and communications gear and improve their interoperability. The FCS consists of 14 individual systems, including manned and robotic ground and air vehicles, software radios, and satellites, as well as an overarching network and operating system tying those components together. Developing such enormous "systems of systems" poses technical and management problems that are neither well defined nor well understood. The software alone-95 million lines of code for the FCS, at last count-poses a daunting challenge. Nobody has yet figured out a way to develop reliable, secure software for much smaller projects. Yet now the DOD is contemplating systems beyond the FCS that would require more than a billion lines of code.
$21 million: Amount the Pentagon Spends Per Hour To Procure New Defense Systems
Another factor contributing to program failure is the shortage of technically trained people, especially systems engineers. A systems engineer translates technical needs into an overall system architecture that creates the best operational capability at the most affordable cost. As a project proceeds and goals or needs shift, systems engineers have to determine the difficult but necessary cost, schedule, and performance trade-offs to keep everything on track. As programs get bigger and more complex, the need for rigorous systems engineering increases.
But the ranks of DOD systems engineers have not increased. "One of the reasons we have such problems in systems engineering today is that we don't have people with the training and experience," says Paul Kaminski, a former U.S. undersecretary of defense for acquisition and technology who recently headed up a National Research Council study on the state of systems engineering in defense. "Gaining the needed expertise," he adds, "is not a 3-year proposition but a 10- to 15-year proposition." In canceling the contracts for the U.S. Navy's Littoral Combat ship program in 2007, Pentagon officials cited inadequate systems engineering as one of the main causes of the huge cost overruns that prompted the cancellations.
Systems engineers are not the only professionals in short supply. Over the past two decades, the DOD has outsourced much of its scientific and engineering expertise to industry. In many of its programs, the Pentagon now has one private contractor for every full-time civilian employee. In some cases, the DOD has admitted, contractors are doing jobs that should be performed only by federal employees, such as weapons procurement and contract preparation. The DOD has also outsourced the systems integration and management of some of its most critical programs, including the FCS.
The Pentagon believes outsourcing saves money, but the practice has depleted the ranks of technical, managerial, and contractual personnel who can provide effective oversight of the department's defense acquisitions. As Norman Augustine, former chairman of Lockheed Martin, told me, "If you're not smarter than your suppliers, you can't manage them effectively."
The situation will only get worse. This year alone, nearly one-fourth of the United States' 637 000 aerospace workers are eligible to retire. Both Northrop Grumman and Lockheed Martin, two of the biggest U.S. defense contractors, report that well over half of their workers will reach retirement age within the next five to 10 years. The same is true of DOD federal employees. Younger workers are not rushing in to replace them. Many students who pursue scientific and technical disciplines don't want careers in defense-related work. The implications for the DOD are grim.
Though personnel skilled in the arts of managing acquisition programs are on the wane, the programs themselves are on the rise, along with the Pentagon's budget. The FY 2009 defense budget of $488 billion is the largest in real terms since World War II and 6 percent higher than this year's budget. Meanwhile, the total projected development costs for the 95 or so major weapons systems currently in the pipeline have more than doubled in the last seven years, from $790 billion in 2000 to $1.6 trillion in 2007.
There are no quick or easy solutions to labor shortages and escalating costs, but the standard response in the private sector is to lean more heavily on automation and information technologies. But the DOD seems not to have benefited from technological advances that would make development less expensive. "One might have thought that more efficient production methods, including computer-aided design and manufacturing, microminiaturization of components, and the employment of greater computing power, all would have reduced costs or at least held them level," former DOD comptroller Dov Zakheim told a Washington, D.C., audience in January 2005. The fact that they haven't, he said, is "not easy to fathom."
Definition: Next-war-itis n: The creation of complex and expensive machinery for possible future conflicts while ignoring the present need for affordable weapons.
Ironically, the solution supported by the DOD and military service chiefs, as well as some members of Congress, is not to make acquisitions more efficient but to spend even more money. They advocate setting aside an amount equal to at least 4 percent of the annual U.S. gross domestic product for defense. At the current GDP of about $16 trillion, that would mean an annual defense budget of $640 billion.
Would even that much be enough? The wars in Iraq and Afghanistan have revealed that U.S. military strategy still focuses on winning conventional force-on-force wars and that it lacks the systems to fight asymmetrical or irregular wars against, for example, insurgents and militias. Secretary of Defense Robert Gates has criticized his own agency for what he calls "next-war-itis." The military services and defense contractors, he says, are too focused on creating complex and expensive machinery for possible conflicts far into the future and not sufficiently attentive to providing affordable weapons that the military can use right now.
So, given the fast-changing military imperatives of these times, what constitutes a successful acquisition program? There is no straightforward answer to that question, partly because in the realm of military acquisitions, there is no universally agreed upon definition of a "successful" program Jacques Gansler, now a professor of public policy at the University of Maryland, has a definition that seems as good as any other: one that "satisfies a military need for an intended cost and intended schedule."
By that definition, most major defense programs would be failures. The fact that only 5 percent ever do get canceled means that the defense community doesn't hold itself to a high standard.
"The definition of success in DOD is to start a program," the GAO's Schinasi says. "That turns on the program's funding. [Success] has nothing to do with the eventual fielding" of a system.
Ronald Fox, professor emeritus at Harvard Business School, adds that after a program gets funded, the definition of success changes again. "A successful program is one that hasn't been canceled," says Fox, who has studied defense acquisitions for over 40 years.
Defined that way, "success" can look an awful lot like what many people would call failure. Seven years ago, the U.S. Air Force awarded a $3.9 billion contract to Boeing to outfit its C-130 cargo aircraft with digital cockpits, which are equipped with monitor screens rather than analog gauges. But Boeing grossly underestimated how much engineering work it would require to modify the C-130's many different configurations. By last year, the program had gone so far over budget that it triggered a congressionally mandated review. The Air Force's response was not to cancel the program but rather to cut the number of planes getting the upgrade from 519 to 222, thereby "saving" a projected $560 million. Nevertheless, the total program still came in $1.4 billion over budget.
Why not just cancel such a program? For one thing, cancellation means lost jobs-and votes. Loren Thompson, a defense analyst at the Lexington Institute, puts it this way: "Most of the time what [the acquisition process] is trying to achieve is only partially 'equipping a soldier in the field.' It is also concerned with getting a congressperson reelected, advancing the career of a bureaucrat, and making certain that the defense-industrial base is sustained during periods of low demand." Politicians, urged on by lobbyists and defense contractors, routinely support programs that should have been killed or should never have been funded, he says.
That sort of collective conspiracy extends to the wildly optimistic promises that contractors make to win funding. Such optimism usually takes the form of "understating the cost [of a program] and overstating the technical requirements," says Fox. For example, company A claims it can produce its widgets for $1 apiece and that they will accomplish X, Y, and Z; it will win out over company B, which pitches its widgets at a more realistic $5 and says they will do only X.
The contractors are not solely to blame for this shell game. "Before you know whether the system will work, you have to define the price of all the units you expect to buy," notes Ron Kadish, former director of the DOD's Missile Defense Agency and now a vice president at consulting firm Booz Allen Hamilton. The cost estimate is always going to be wrong, he says, but everyone, including the DOD and military service procurement officials and Congress, pretends that it is correct. This intellectual dishonesty leads to expectations that can never be met.
"The bureaucratic incentives at work in the acquisition system militate fairly strongly against honesty," says the Lexington Institute's Thompson. "Until a weapon system is put into operational test and then must perform, there are lots of rewards for understating costs, for understating technical challenges, and for exaggerating the speed at which costs and technical problems can be overcome."
To be fair, part of that exaggeration stems from the real engineering problem of designing a system that has to meet some theoretical threat 5 or 10 or 15 years from now. If you had to design a car of the future, what technology would you put in it? Would it rely on just what's available today, or would it need to accommodate a power source or steering mechanism that doesn't yet exist? Even when you settle on a design, innovations will inevitably arise during the many years that your system is in development.
Dependence on unproven technology is anathema in the commercial world, but it's common in defense programs. The design for the Army's Crusader howitzer, for instance, relied on 16 "critical" technologies, including advanced armaments, ammunition handling, and mobility. But only six of those technologies had ever been demonstrated outside the laboratory when the Crusader entered development in 1994. Subsequent problems with those untested technologies contributed to the doubling of the program's development cost-and ultimately to its cancellation in 2002.
In their landmark 1962 book, The Weapons Acquisition Process: An Economic Analysis , Harvard professors Merton Peck and Frederic Scherer noted that the fundamental objective of acquiring any military system is that it either create a qualitative superiority over an enemy's weapon system or neutralize the enemy's superiority-not only today but into the future. Thus the eternal push for better fighting technology, from sharpened stones to GPS-guided bombs.
Each technological advance a country achieves should not only enhance its own military position but should also degrade the enemy's. For example, making a bomber stealthy enables it to be more destructive to the enemy's key installations. This situation is quite different from the commercial world. Buying an iPhone may rock your world, but it won't have any ill effects (except maybe envy) on your friend who owns a Motorola Razr.
This mind-set tacitly encourages the DOD to demand, and contractors to propose, ever more sophisticated technology. As one former senior military program manager put it, "to sell a program today, you need to claim that it is 'transformational' in some way."
But that quest for the "transformational" is now colliding with the hard reality that many of the fundamental technologies in today's weapons systems are already very advanced. The engines, avionics, and flight controls in military aircraft, for instance, are all close to the limits of what is possible. Even incremental advances come at enormous cost.
And so the infatuation with immature and exotic technologies, with their high costs and risks. James Finley, deputy undersecretary of defense for acquisition and technology, admits that in many programs "technology is being pushed too fast, too soon."
The widespread inability to meet promises creates a vicious circle: lowballed cost projections allow too many programs to be approved; as the projections for each program repeatedly get revised upward, the defense budget balloons; eventually, cuts have to be made, resulting in what military critic Chuck Spinney has termed the "defense death spiral." In congressional testimony in 2002, Spinney described the spiral as "shrinking combat forces, decreasing rates of modernization, aging weapons inventories, with the rising cost of operations creating continual pressure to reduce readiness."
Over the last six decades a dozen or so blue-ribbon panels and at least a hundred initiatives have called for detailed, concrete reforms in defense acquisitions. So there isn't much doubt that something is fundamentally wrong. What is most disheartening is that everyone knows it and nobody-not DOD management, not the military services, not Congress-has done much about it. "The problem in Washington isn't what we don't know but what we don't want to know," says defense analyst Thompson.
The most recent of these reform efforts was the 2005 Defense Acquisition Performance Assessment, led by Ron Kadish. The panel came up with 35 recommendations in all, some repeating earlier committees' findings, some new to that report. Among the people I interviewed who had read the Kadish report, nearly everyone said that three of the recommendations in particular would have an immediate impact on the acquisitions process, without having to change existing regulation or win congressional approval.
25 percent of the 637 000 U.S. Aerospace Workers are currently eligible to retire
The first of these involves trying to break the defense death spiral by returning to a "block program" approach, in which systems would be built incrementally in capability, instead of trying to satisfy every mission requirement in the first increment. The second recommendation is that programs be "time certain," meaning that most programs would have to deliver some useful operational capability within five years. That requirement would force contractors to use only technologies that are essentially mature, rather than ones that would need to be invented on a schedule.
The third recommendation is to fund only those programs that have an 80:20 cost confidence level, meaning the program has an 80 percent chance of meeting its estimated cost target. Traditionally, the DOD has aimed for a 50:50 confidence level, and some programs don't even reach that cutoff. The Army's FCS program, for instance, was approved in 2003 with "somewhere down around [a] 28 percent chance of success," according to then Army chief of staff Peter J. Schoomaker.
If the DOD were to implement all three recommendations-funding only those programs that it deemed to have an 80 percent chance of succeeding, could deliver operational capability in five years, and could be developed incrementally-it would effectively cut the number of new programs by up to 25 percent. Unfortunately, the Pentagon and the military services have shown no great willingness to scale back, even though it would likely mean getting systems out into the field more quickly.
Still, the DOD's Finley insists that the department is at least attempting to implement all of the Kadish panel's recommendations. But will they in fact fix a broken system? Finley is quite optimistic, but even he admits, "I can't predict the will for change."
Indeed, if there's one sentiment that has been repeated more often than any other in the past 60 years of failed attempts at acquisition reform, it is the need for "the will to change." That quality always seems to be in short supply.
In this election season, neither of the two presidential candidates has had much to say about reforming defense acquisitions. That's disappointing but not surprising. No politician wants to be accused of not supporting the troops. Any suggestion that defense spending be reined in has become, like Social Security, a political third rail. Most defense experts are skeptical that the Pentagon or Congress or the White House will fundamentally alter the current way of doing business. There is too much money and too many jobs at stake.
But reform will have to come. Each day that the acquisition process continues to operate ineffectively and inefficiently is another day that the troops are not getting what they need, the country is less secure, and much-needed programs, both civilian and military, don't get funded.
The next administration will need to choose wisely, and soon.
ROBERT N. CHARETTE, an IEEE Spectrum contributing editor, is a self-described "risk ecologist" who investigates the impact of the changing concept of risk on technology and societal development. In writing the cover story, "What's Wrong With Weapons Acquisitions?", he spent over 18 months interviewing dozens of industry and government defense-acquisition experts. "Everyone knows the acquisition process desperately needs to change," he says. "Unfortunately, no one I spoke to believes it will change until there is a major national crisis."
Citation: Robert N. Charette, What's Wrong with Weapons Acquisitions?, IEEE Spectrum (November 2008)