Both the people and the machines at the National Counterterrorism Center have trouble digesting the huge volumes of data that the other intelligence agencies collect.
Saturday, Jan. 30, 2010
by Shane Harris, National Journal
When the news reached Washington that a Northwest Airlines passenger had been taken into custody in Detroit after, authorities alleged, he tried to set off a bomb aboard the plane, the staff at the National Counterterrorism Center outside Washington quickly huddled. During any critical terrorist incident, the NCTC becomes the central clearinghouse for information and updates across the government.
Within hours of the Christmas Day attack, officials organized a secure video teleconference that included more than 20 agencies that would have to respond to the incident. They began sharing what they knew and asking what they needed to find out. Using established protocols, the NCTC's staffers -- experts drawn from the CIA, the FBI, and other intelligence agencies -- tried to get their hands around the rapidly developing story line.
According to an official with knowledge of the day's events, once the NCTC analysts had a name, they began looking for whatever information they had in their databases on Umar Farouk Abdulmutallab, a young Nigerian man who would later assert he had been trained by Al Qaeda in Yemen to blow up the plane with a bomb hidden in his underwear. The analysts hunted for links to reporting on terrorist activity, which the center routinely collects. They first had to determine whether this was an isolated event or a part of a larger plot.
The White House directed the dozens of agencies participating in the secure teleconference, and soon they began reporting in. The FBI would start an investigation. The Homeland Security Department would step up airline passenger screening. The Federal Aviation Administration would have to adjust to the disruption in air travel schedules. The NCTC analysts began working the plot, just as they had after the July 2005 bombing of the London subway and bus system and the 2006 plan to sneak liquid bombs onto trans-Atlantic airliners. They searched for connections, leads, and clues, those notorious "dots" that made up the full narrative of an attack that, until that moment, had not been fully understood.
Crisis control is not the center's usual routine. It also has a portfolio of longer-term duties, including helping plan global counterterrorism strategy and keeping track of terrorism trends. Director Michael Leiter has said that the task that has grown the most in the past two years involves deep analysis of the root causes of radicalization. Academics have praised the NCTC for its terrorism research. If the government had a think tank for terrorism, this would be it.
The center's most immediate role, though, is in defending against attacks. It is a data hub, the one place in the government where streams of terrorist reports -- and warnings -- from dozens of agencies are supposed to come together. Since the Christmas Day attempt, many critics have questioned why the NCTC's analysts, who examine this information every day, failed to detect Abdulmutallab's plan. According to an Obama administration review, the analysts had received or were privy to intelligence about him from at least three agencies. They were also aware, more broadly, about a burgeoning terrorist threat in Yemen.
Last April, in a speech at the Aspen Institute, Leiter described a "resurgence of Al Qaeda in the Arabian Peninsula which is quite worrisome... and a stated desire in their propaganda to expand their reach beyond Yemen into the Kingdom of Saudi Arabia and elsewhere." Last year, the National Security Agency intercepted phone traffic of Qaeda operatives in Yemen talking about a "Nigerian" who was engaged in a new plot. And, as most of the country now knows, in November, Abdulmutallab's father, a prominent Nigerian banker, walked into the U.S. Embassy in Abuja and told officials he was afraid that his son had gone to Yemen and had become an Islamic radical.
In hindsight, if any one outfit was in a position to put all of these pieces together, it was the NCTC. That is, after all, why federal officials set it up in 2004, to be a "fusion center" for the intelligence community. But to describe the center as the place where all the dots are connected -- implying, as some unnamed officials have in recent press reports, that its staff are supposed to be able to predict the next attack -- is at best a misunderstanding of the NCTC's capabilities and at worst an attempt to pin the intelligence failures of Christmas Day on a single organization.
In the global fight against terrorism, the NCTC is mostly a passive participant. It cannot investigate terrorist suspects in the United States. (That's the FBI's job.) It cannot legally conduct covert operations overseas. (That's the CIA's domain.) It cannot intercept foreign communications. (That's the NSA's bailiwick.) And it cannot revoke visas. (That's up to the State Department.)
"NCTC, it's important to remember, is just an aggregator of information," said Rick (Ozzie) Nelson, who worked in the center's strategic operational planning directorate and is now at the Center for Strategic and International Studies. "Certainly, some criticism can be leveled that they didn't connect all the dots. But that can't fall all on NCTC's shoulders."
In his Aspen Institute speech, Leiter said that every threat report that his analysts write is available to more than 17,000 people at the federal, state, and local government levels. But even many in that vast network of colleagues, he said, don't always understand the NCTC's role. "I think it's important to remind people what we do, because, frankly, even the people with whom I work on a daily basis aren't always sure."
Drowning In The Flood
So what does the NCTC do? For starters, it tries to keep up. From within their headquarters at a new secure facility in Northern Virginia, the center's analysts can access at least 28 computer networks maintained by other agencies. They're supposed to pan these streams of dots for clues. But, according to former NCTC employees and current intelligence officials, that is a mostly manual process, and a time-consuming one.
In a 2006 interview with CBS News, Russell Travers, a deputy director responsible for the center's major database of terrorist names, said that most NCTC analysts need six or seven computers on their desks to plug into the different systems around the government. A Defense Department analyst at the center, for example, can read raw data from the CIA or the FBI. The NCTC is the only place where that kind of cross-agency sharing, a key recommendation of the 9/11 commission, actually happens, Travers said.
The data come into the center in huge floods, however. After the 9/11 attacks, the FBI, the CIA, and electronic spies such as the National Security Agency went into collection overdrive. If there was a way to get more information, they found it -- from interrogations, from computers, from the pocket detritus of Qaeda fighters killed in battle. After the NCTC was established, the intelligence agencies turned their data streams on the center with the ferocity of a rushing river. From 2004 to 2005, the master database of known or suspected terrorists doubled in size, Travers said. Then, it doubled again in 2006. Today, officials estimate that the system contains half a million names, all of which were poured in by other agencies.
"Right now, we are getting on a daily basis many thousands of names," Travers told CBS. (Outside experts have put the number between 4,000 and 8,000.) "Lots of them are undoubtedly duplicates. Some of them aren't really terrorists." Just to keep up with the names, Travers said, he employed "about 80 analysts," who spent their days sifting through "fragmentary," "ambiguous," and "conflicting" information. That is the nature of intelligence, but American analysts have never had to manage data on this scale. Travers seemed to be "drowning" in information, the CBS reporter commented.
"It's probably the single biggest challenge that the community's got right now," Travers replied. "The information that's out there is coming in a deluge."
Another senior intelligence official concurred. Ron Sanders, the top personnel official for the national intelligence director, told reporters this month, "The amount of information, the amount of dots that need to be connected, the needles in the haystack, it's just mind-numbing."
Disconnected Data
The master list is called the Terrorist Identities Datamart Environment, or TIDE. Before someone ends up on a no-fly list, or gets pulled aside at an airport for questioning or a special pat down, his name goes into this catchall repository. Other government agencies can add a name, but it's the NCTC analysts' job to cull through and suggest who needs further inspection.
If someone deserves blame for not keeping Abdulmutallab from boarding a plane, he or she probably doesn't work in the NCTC. Analysts there pass along names from the master database to another group, the Terrorist Screening Center, run by the FBI, which compiles the much-trumpeted no-fly list. After a review of the Christmas attack, President Obama said that had the government put together all the known information about Abdulmutallab, it "would have placed the suspect on the 'no-fly' list." The NCTC might have synthesized that information, but preventing the suspect from boarding an airliner would have been the FBI's call.
And there's more: Still another agency maintains the no-fly list that the FBI creates. The Transportation Security Administration, established after the 9/11 attacks, shares names with the airlines. The airlines control passenger manifests, and it's up to them to check their records against the no-fly list.
The airlines' track record in complying with TSA directives is flawed. Just recently, a tuberculosis patient whom the TSA had alerted airlines not to allow onboard any plane was permitted to take a U.S. Airways flight from Philadelphia to San Francisco. According to the Centers for Disease Control and Prevention, the flight was too short for the patient to have infected other passengers. But he should have been kept off the plane. The TSA defended itself, saying that it was the airlines' job to check and abide by the agency's lists. "We are just a conduit," an agency statement said. "We receive information and provide it to the airlines."
To be sure, names by themselves are practically useless. "If there's a name in a database with no other information, it's just a name in a database," Nelson said. "Unless there's a trigger, something that tells you to look at this individual, you're not going to do that." At the NCTC, he said, "there aren't enough analysts, and there are already so many other issues that they're working."
Abdulmutallab's father's report to the U.S. Embassy in Nigeria that his son might have become a radical in Yemen is arguably a triggering event. And in this case, CIA officials worked up a dossier on Abdulmutallab, and the embassy passed his name on to the counterterrorism center via the TIDE database. There was no failure to share information, to move around reports and names, to log entries in a computer. But no system existed to check what analysts were actually doing with that information. Who was following up on the CIA dossier? Did anyone bother to check the entry in the TIDE database against visa records?
According to current and former government officials who are familiar with the NCTC's tools, analysts can find information by searching databases for keywords. But no system is in place to alert them to potentially important linkages between those keywords. The best tool they have is similar to a Google News alert, which automatically searches particular databases for keywords and returns any results. But such a crude method for mining fragmentary, often incomplete, data does not produce very refined results.
Using the search tools and databases they have, it's difficult for analysts to conduct so-called dynamic searches, which involve looking for multiple variables and terms that are scattered across databases. To do that -- a fundamental requirement for "connecting the dots" -- analysts have to join the various systems in a technically demanding process that can take hours or days.
Further complicating the effort, many of the records in intelligence databases contain additional information in the form of written comments or notes attached to the main record. Sources familiar with these kinds of records said that the notes and comments aren't indexed and so the kinds of keyword searches that analysts perform can't find them. That's a problem because it's often these subsets of information that contain the nuance, context, and interpretation that might actually connect the dots.
Obama seemed to grasp the underlying problem when he said that the Christmas attack "was not a failure to collect intelligence; it was a failure to integrate and understand the intelligence that we already had." But why the failure? It's not because the analysts are incapable of fusing different pieces of a story. It's just that it is exceedingly difficult to do so using the tools now at their disposal. Compounding this problem, current and former officials say, is the lack of an auditing system. Agencies are throwing information into databases at a dizzying rate, but no one is keeping track of who uses the data and how, or whether proper follow-through takes place.
It's not clear whether the immediate corrections that the president has called for involve a new tool for keeping track of where those dots are in the system and who is looking at them. But Leiter told the Senate Homeland Security and Governmental Affairs Committee on January 20 that technology improvements -- including the ability to simultaneously search multiple databases -- is one of four areas on which Obama wants the center to direct its attention.
Enemies Within
If the NCTC has never quite become the seamless conductor for the government's vast terrorism-fighting apparatus, the shortcomings might have something to do with the agency's tortured history. The center was born amid internecine dissent. The Bush administration's plan was to carve the counterterrorism divisions out of the FBI and the CIA and "co-locate" them under one roof as the Terrorist Threat Integration Center. The year was 2003, and the administration was under public pressure to tear down the bureaucratic walls separating the many counterterrorism agencies. The TTIC was a prime example of the new order.
From a practical standpoint, it made sense to put the lead agencies for domestic and foreign counterterrorism in one house. But almost immediately, the G-men and the spies resisted the arrangement, fearful that the new center, and its director, would rob them of their coveted powers. Civil libertarians, meanwhile, warned that putting the FBI, a domestic law enforcement agency, in such proximity to spies, who labor under fewer and different laws when working abroad, was a recipe for abuse.
The essential compromise was that the two counterterrorism units would still report back to their respective bosses -- the directors of the FBI and the CIA. But another turf war erupted over who should oversee the center itself. Officials at the Homeland Security Department wanted the job, and some congressional overseers backed their bid. But the Bush administration planned to give control to the CIA director, who, at the time, was still the overall manager of the entire intelligence community.
Congress had passed legislation calling for a new "fusion center" at the Homeland Security Department. Sen. Joe Lieberman of Connecticut, then the top Democrat on the Senate committee that oversaw the department, accused the administration of doing nothing to break up the historic rivalries between the CIA and the FBI that had caused some of the missteps before the 9/11 attacks.
"CIA clearly has a lever of control; that is not going to help these entrenched rivalries," a Lieberman spokeswoman said at the time, referring to the White House plan to put the spy agency in charge. The administration didn't budge. When Congress established the Office of the Director of National Intelligence in 2005, the baton simply passed to that new spy chief. The National Counterterrorism Center, as it is now called, may be home to employees from across the government, but at its core, it is an intelligence organization. The Homeland Security Department's inability to get control of all the terrorist reporting -- all those "dots" -- damaged its standing in the pecking order and reinforced a growing perception that the department wasn't a major player in the war on terrorism.
Today, the NCTC still has detractors and resisters. According to a former official, the CIA, the State Department, and Homeland Security are the top three carpers. "They've all done everything they can to ensure that NCTC has a difficult time working," the official said.
The CIA, the former official said, still resents having its counterterrorism group operating out of the NCTC, and Homeland Security still smarts over not having control. The State Department, meanwhile, has asserted that it has the ultimate legal authority to operate overseas.
"The CIA station and the embassies are very, very clear that they understand the threat because they're in the field," the former official said. "They always say, 'You people back in D.C. don't understand.' Now what's happening is that these individuals had a threat in Nigeria, and they're blaming D.C. for not figuring it out."
Still, the center has touted significant progress in making more information available to agencies that might need it. The NCTC operates a classified website that's open to more than 5,000 intelligence analysts around the world. "If the information is available to other agencies, it goes up on that website," former NCTC Director John Scott Redd told journalist Ronald Kessler for his book The Terrorist Watch: Inside the Desperate Race to Stop the Next Attack. Redd said, "Anybody with the right clearance can get on there and say, 'What do you make of this?' Or, 'How does this compare with that?' Nothing like this existed prior to 9/11."
In his Aspen Institute address, NCTC Director Leiter took offense at those who think "it has been an accident that we weren't attacked over the past eight years. I think that is flatly and completely false," he said. "We have disrupted plots, we have watched people, we have put things in place which make it less likely we will be attacked today than we were on 9/11."
But he cautioned that the system isn't perfect, and he seemed to predict the narrative that would overtake his organization in the event of another attack on the United States. "We've increased the odds in our favor against the terrorists. So, is it a success story that we haven't been attacked? I think, absolutely. If we are attacked, will it be a failure? Well, in some absolute sense, it will absolutely be a failure." But intelligence, Leiter said, "is an imperfect business. So it does not mean necessarily that the system failed if there is an attack. It means we had a failure, but it doesn't mean the system is a failure."